René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Reading using open api fails with 416 error code in google-resumable-media","articleBody":"  - OS type and version: Ubuntu\r\n  - Python version: 3.7.5\r\n  - pip version: 9.0.1\r\n  - `google-cloud-storage` version: 1.37.0\r\n  - `google-resumable-media` version: 1.2.0\r\n\r\nWhile trying to copy a gcs file using blob.open method to read and write, it fails at the last read step with the following error:\r\n```\r\ngoogle.resumable_media.common.InvalidResponse: ('Request failed with status code', 416, 'Expected one of', \u003cHTTP\r\nStatus.OK: 200\u003e, \u003cHTTPStatus.PARTIAL_CONTENT: 206\u003e)\r\n```\r\n\r\n#### Code example\r\n\r\nIn the snippet below i am using shutil.copyfileobj, but same error happens if I i loop and call read with a chunk size.\r\n\r\n```python\r\nimport shutil\r\n\r\nfrom google.cloud import storage\r\n\r\nclient = storage.Client()\r\nbucket: storage.Bucket = client.bucket(\"gcp-public-data-landsat\")\r\nblob = bucket.get_blob(\"index.csv.gz\")\r\nsrc = blob.open(\"rb\", chunk_size=40 * 1024 * 1024)\r\ndest = open(\"index.csv.gz\", \"wb\")\r\nshutil.copyfileobj(src, dest, 40 * 1024 * 1024)\r\n```\r\n\r\n#### Stack trace\r\n```\r\nOriginal exception was:\r\nTraceback (most recent call last):\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/client.py\", line 728, in downloa\r\nd_blob_to_file\r\n    checksum=checksum,\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py\", line 961, in _do_downl\r\noad\r\n    response = download.consume(transport, timeout=timeout)\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/requests/download.py\", line 16\r\n8, in consume\r\n    self._process_response(result)\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/_download.py\", line 186, in _p\r\nrocess_response\r\n    response, _ACCEPTABLE_STATUS_CODES, self._get_status_code\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/_helpers.py\", line 104, in req\r\nuire_status_code\r\n    *status_codes\r\ngoogle.resumable_media.common.InvalidResponse: ('Request failed with status code', 416, 'Expected one of', \u003cHTTP\r\nStatus.OK: 200\u003e, \u003cHTTPStatus.PARTIAL_CONTENT: 206\u003e)\r\nDuring handling of the above exception, another exception occurred:\r\nTraceback (most recent call last):\r\n  File \"storage-test.py\", line 10, in \u003cmodule\u003e\r\n    shutil.copyfileobj(src, dest, 40 * 1024 * 1024)\r\n  File \"/usr/lib/python3.7/shutil.py\", line 79, in copyfileobj\r\n    buf = fsrc.read(length)\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/fileio.py\", line 97, in read\r\n    start=fetch_start, end=fetch_end, **self._download_kwargs\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py\", line 1307, in download\r\n_as_bytes\r\n    checksum=checksum,\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/client.py\", line 731, in downloa\r\nd_blob_to_file\r\n    _raise_from_invalid_response(exc)\r\n  File \"/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py\", line 4061, in _raise_f\r\nrom_invalid_response\r\n    raise exceptions.from_http_status(response.status_code, message, response=response)\r\ngoogle.api_core.exceptions.RequestRangeNotSatisfiable: 416 GET https://storage.googleapis.com/download/storage/v\r\n1/b/gcp-public-data-landsat/o/index.csv.gz?generation=1616911592995216\u0026alt=media: Request range not satisfiable:\r\n ('Request failed with status code', 416, 'Expected one of', \u003cHTTPStatus.OK: 200\u003e, \u003cHTTPStatus.PARTIAL_CONTENT: \r\n206\u003e)\r\n```\r\n\r\n","author":{"url":"https://github.com/amardeep","@type":"Person","name":"amardeep"},"datePublished":"2021-03-28T06:56:41.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/399/python-storage/issues/399"}