René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"retry ChunkedEncodingError by default","articleBody":"Per https://github.com/psf/requests/issues/4771, ChunkedEncodingError is a kind of transport-layer error. They don't classify it as a connection error because it's caused by a \"misbehaving server\", but in our case it's likely due to a transient issue somewhere in the server stack. Retrying is likely to help.\r\n\r\n#### Stack trace\r\n\r\nSee: https://github.com/googleapis/python-bigquery-storage/issues/242\r\n\r\n```\r\n[POST]\r\nTraceback (most recent call last):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 696, in _update_chunk_length\r\n    self.chunk_left = int(line, 16)\r\n  ValueError: invalid literal for int() with base 16: b''\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 436, in _error_catcher\r\n    yield\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 763, in read_chunked\r\n    self._update_chunk_length()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 700, in _update_chunk_length\r\n    raise httplib.IncompleteRead(line)    \r\n  http.client.IncompleteRead: IncompleteRead(0 bytes read)\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/requests/models.py\", line 751, in generate\r\n    for chunk in self.raw.stream(chunk_size, decode_content=True):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 571, in stream\r\n    for line in self.read_chunked(amt, decode_content=decode_content):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 792, in read_chunked\r\n    self._original_response.close()\r\n  File \"/opt/python3.9/lib/python3.9/contextlib.py\", line 135, in __exit__\r\n    self.gen.throw(type, value, traceback)\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/urllib3/response.py\", line 454, in _error_catcher\r\n    raise ProtocolError(\"Connection broken: %r\" % e, e)\r\n  urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py\", line 2447, in wsgi_app\r\n    response = self.full_dispatch_request()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py\", line 1952, in full_dispatch_request\r\n    rv = self.handle_user_exception(e)\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py\", line 1821, in handle_user_exception\r\n    reraise(exc_type, exc_value, tb)\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py\", line 39, in reraise\r\n    raise value\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py\", line 1950, in full_dispatch_request\r\n    rv = self.dispatch_request()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py\", line 1936, in dispatch_request\r\n    return self.view_functions[rule.endpoint](**req.view_args)\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/__init__.py\", line 152, in view_func\r\n    function(data, context)\r\n  File \"/workspace/common/functions.py\", line 61, in wrapper\r\n    raise e\r\n  File \"/workspace/common/functions.py\", line 54, in wrapper\r\n    response = wrapped_func(*args, **kwargs)\r\n  File \"/workspace/main.py\", line 533, in controller\r\n    status = create_files(contex_bdp)\r\n  File \"/workspace/main.py\", line 346, in create_files\r\n    for row in result:\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/page_iterator.py\", line 214, in _items_iter\r\n    for page in self._page_iter(increment=False):\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/page_iterator.py\", line 253, in _page_iter\r\n    page = self._next_page()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/page_iterator.py\", line 382, in _next_page\r\n    response = self._get_next_page_response()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/cloud/bigquery/table.py\", line 1541, in _get_next_page_response\r\n    return self.api_request(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/cloud/bigquery/client.py\", line 686, in _call_api\r\n    return call()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/retry.py\", line 285, in retry_wrapped_func\r\n    return retry_target(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/retry.py\", line 188, in retry_target\r\n    return target()\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/cloud/_http.py\", line 473, in api_request\r\n    response = self._make_request(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/cloud/_http.py\", line 337, in _make_request\r\n    return self._do_request(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/cloud/_http.py\", line 375, in _do_request\r\n    return self.http.request(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/google/auth/transport/requests.py\", line 482, in request\r\n    response = super(AuthorizedSession, self).request(\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/requests/sessions.py\", line 530, in request\r\n    resp = self.send(prep, **send_kwargs)\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/requests/sessions.py\", line 685, in send\r\n    r.content\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/requests/models.py\", line 829, in content\r\n    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''\r\n  File \"/layers/google.python.pip/pip/lib/python3.9/site-packages/requests/models.py\", line 754, in generate\r\n    raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))\r\n```\r\n","author":{"url":"https://github.com/tswast","@type":"Person","name":"tswast"},"datePublished":"2021-07-23T14:47:02.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/235/python-api-core/issues/235"}