René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Started getting random ValueError: Bad response:  not multi-part exceptions ","articleBody":"Thanks for stopping by to let us know something could be better!\r\n\r\n**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.\r\n\r\nPlease run down the following list and make sure you've tried the usual \"quick fixes\":\r\n\r\n  - Search the issues already opened: https://github.com/googleapis/python-storage/issues\r\n  - Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python\r\n\r\nIf you are still having issues, please be sure to include as much information as possible:\r\n\r\n#### Environment details\r\n\r\n  - OS type and version: Debian GNU/Linux 10 (buster)\r\n  - Python version:  Python 3.7.7\r\n  - pip version: pip 21.0.1\r\n  - `google-cloud-storage` version: 1.37.0\r\n\r\n#### Steps to reproduce\r\n\r\n  1. I cannot reproduce this because it is happening periodically, and only in my production flask app, not even on the same docker image running in staging with less traffic.  This makes me think its related to some threading related issue, but I can't isolate the problem. \r\n  2. This problem started without any code changes on our side, and without changing the docker image, so something changed in the google cloud storage server response behavior that started triggering this sometime starting on March 30, and then ramping up in frequency on April 1st and 2nd. \r\n\r\n#### Code example\r\n\r\nI don't have a concise code example. But it's being triggered with\r\nget_blob(key).\r\n\r\n\r\n#### Stack trace\r\n```\r\ntraceback: [\r\n0: \"Traceback (most recent call last):\r\n\"\r\n1: \"  File \"/usr/local/lib/python3.7/site-packages/flask/app.py\", line 1950, in full_dispatch_request\r\n    rv = self.dispatch_request()\r\n\"\r\n2: \"  File \"/usr/local/lib/python3.7/site-packages/flask/app.py\", line 1936, in dispatch_request\r\n    return self.view_functions[rule.endpoint](**req.view_args)\r\n\"\r\n3: \"  File \"/usr/local/lib/python3.7/site-packages/middle_auth_client/decorators.py\", line 171, in decorated_function\r\n    return f(*args, **kwargs)\r\n\"\r\n4: \"  File \"/usr/local/lib/python3.7/site-packages/middle_auth_client/decorators.py\", line 253, in decorated_function\r\n    return f(*args, **kwargs)\r\n\"\r\n5: \"  File \"./pychunkedgraph/app/meshing/v1/routes.py\", line 75, in handle_get_manifest\r\n    return common.handle_get_manifest(table_id, node_id)\r\n\"\r\n6: \"  File \"./pychunkedgraph/app/meshing/common.py\", line 204, in handle_get_manifest\r\n    flexible_start_layer=flexible_start_layer\r\n\"\r\n7: \"  File \"./pychunkedgraph/meshing/meshgen_utils.py\", line 152, in get_highest_child_nodes_with_meshes\r\n    existence_dict = stor.files_exist(filenames)\r\n\"\r\n8: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/storage/storage.py\", line 584, in files_exist\r\n    self.wait(desc)\r\n\"\r\n9: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/threaded_queue.py\", line 234, in wait\r\n    self._check_errors()\r\n\"\r\n10: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/threaded_queue.py\", line 191, in _check_errors\r\n    raise err\r\n\"\r\n11: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/threaded_queue.py\", line 153, in _consume_queue\r\n    self._consume_queue_execution(fn)\r\n\"\r\n12: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/threaded_queue.py\", line 180, in _consume_queue_execution\r\n    fn()\r\n\"\r\n13: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/storage/storage.py\", line 575, in exist_thunk\r\n    results.update(interface.files_exist(paths))\r\n\"\r\n14: \"  File \"/usr/local/lib/python3.7/site-packages/cloudvolume/storage/storage_interfaces.py\", line 240, in files_exist\r\n    result[file_path] = self._bucket.get_blob(key)\r\n\"\r\n15: \"  File \"/usr/local/lib/python3.7/site-packages/google/cloud/storage/batch.py\", line 296, in __exit__\r\n    self.finish()\r\n\"\r\n16: \"  File \"/usr/local/lib/python3.7/site-packages/google/cloud/storage/batch.py\", line 281, in finish\r\n    responses = list(_unpack_batch_response(response))\r\n\"\r\n17: \"  File \"/usr/local/lib/python3.7/site-packages/google/cloud/storage/batch.py\", line 334, in _unpack_batch_response\r\n    raise ValueError(\"Bad response:  not multi-part\")\r\n\"\r\n18: \"ValueError: Bad response:  not multi-part\r\n```\r\n\r\nMaking sure to follow these steps will guarantee the quickest resolution possible.\r\n\r\nThanks!\r\n","author":{"url":"https://github.com/fcollman","@type":"Person","name":"fcollman"},"datePublished":"2021-04-02T21:45:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":9},"url":"https://github.com/404/python-storage/issues/404"}