René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"google-cloud-datastore no longer works with freezegun","articleBody":"As of `google-cloud-datastore` 1.0.0 it no longer works together with [freezegun](https://github.com/spulec/freezegun).\r\n\r\nWorkaround: downgrade to `google-cloud-datastore` 0.23.0\r\n\r\n1. OS type and version:\r\n\r\nUbuntu 17.04, but it was also reproduced on 16.04.\r\n\r\n2. Python version and virtual environment information `python --version`\r\n\r\n```\r\n$ python --version\r\nPython 3.6.0+\r\n```\r\n\r\n3. google-cloud-python version `pip show google-cloud`, `pip show google-\u003cservice\u003e` or `pip freeze`\r\n\r\n```\r\n$ pip list | grep google-cloud\r\ngapic-google-cloud-datastore-v1 0.15.3 \r\ngoogle-cloud-core               0.24.0 \r\ngoogle-cloud-datastore          1.0.0  \r\ngoogle-cloud-storage            1.0.0  \r\nproto-google-cloud-datastore-v1 0.90.3\r\n```\r\n\r\n4. Stacktrace if available\r\n\r\n\u003cdetails\u003e\u003csummary\u003epy.test --tb native\u003c/summary\u003e\r\n\r\n```python\r\nTraceback (most recent call last):\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py\", line 120, in inner\r\n    return to_call(*args)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py\", line 68, in inner\r\n    return a_func(*updated_args, **kwargs)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/grpc/_channel.py\", line 507, in __call__\r\n    return _end_unary_response_blocking(state, call, False, deadline)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/grpc/_channel.py\", line 455, in _end_unary_response_blocking\r\n    raise _Rendezvous(state, None, None, deadline)\r\ngrpc._channel._Rendezvous: \u003c_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, Received RST_STREAM with error code 2)\u003e\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 74, in _catch_remap_gax_error\r\n    yield\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 173, in commit\r\n    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/gapic/datastore/v1/datastore_client.py\", line 345, in commit\r\n    return self._commit(request, options)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 419, in inner\r\n    return api_caller(api_call, this_settings, request)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 407, in base_caller\r\n    return api_call(*args)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 368, in inner\r\n    return a_func(*args, **kwargs)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py\", line 126, in inner\r\n    ' classified as transient', exception)\r\ngoogle.gax.errors.RetryError: GaxError(Exception occurred in retry method that was not classified as transient, caused by \u003c_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, Received RST_STREAM with error code 2)\u003e)\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"/home/remco/example/test.py\", line 16, in test_put_frozen\r\n    client.put(entity)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/client.py\", line 384, in put\r\n    self.put_multi(entities=[entity])\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/client.py\", line 411, in put_multi\r\n    current.commit()\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/batch.py\", line 273, in commit\r\n    self._commit()\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/batch.py\", line 249, in _commit\r\n    self.project, mode, self._mutations, transaction=self._id)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 173, in commit\r\n    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)\r\n  File \"/usr/lib/python3.6/contextlib.py\", line 100, in __exit__\r\n    self.gen.throw(type, value, traceback)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 82, in _catch_remap_gax_error\r\n    six.reraise(error_class, new_exc, sys.exc_info()[2])\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/six.py\", line 685, in reraise\r\n    raise value.with_traceback(tb)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 74, in _catch_remap_gax_error\r\n    yield\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/datastore/_gax.py\", line 173, in commit\r\n    return super(GAPICDatastoreAPI, self).commit(*args, **kwargs)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/cloud/gapic/datastore/v1/datastore_client.py\", line 345, in commit\r\n    return self._commit(request, options)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 419, in inner\r\n    return api_caller(api_call, this_settings, request)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 407, in base_caller\r\n    return api_call(*args)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/api_callable.py\", line 368, in inner\r\n    return a_func(*args, **kwargs)\r\n  File \"/home/remco/.local/share/virtualenvs/tmp-35dac56d418be2c/lib/python3.6/site-packages/google/gax/retry.py\", line 126, in inner\r\n    ' classified as transient', exception)\r\ngoogle.cloud.exceptions.InternalServerError: 500 Received RST_STREAM with error code 2\r\n```\r\n\u003c/details\u003e\r\n\r\n5. Steps to reproduce\r\n\r\nDecorate a function using `freezegun.freeze_time` from [freezegun](https://github.com/spulec/freezegun) and put a gcloud datastore entity.\r\n\r\n6. Code example\r\n\r\n```python\r\nimport google.cloud.datastore\r\nimport pytest\r\nfrom freezegun import freeze_time\r\n\r\n\r\n# Works\r\ndef test_put():\r\n    client = google.cloud.datastore.Client()\r\n    entity = google.cloud.datastore.Entity(key=client.key('spam'))\r\n    client.put(entity)\r\n\r\n\r\n# Fails\r\n@freeze_time('3216-08-04')\r\ndef test_put_frozen():\r\n    client = google.cloud.datastore.Client()\r\n    entity = google.cloud.datastore.Entity(key=client.key('spam'))\r\n    client.put(entity)\r\n```\r\n\r\nThe datastore was launched in the background using the following code:\r\n\r\n```python\r\ndef start_datastore():\r\n    \"\"\"\r\n    Start an in memory gcloud datastore emulator.\r\n\r\n    The datastore and all of its subprocesses are killed when the Python\r\n    interpreter is shut down.\r\n\r\n    Raises:\r\n        RuntimeError: If the datastore failed to start.\r\n\r\n    \"\"\"\r\n    tmpdir = tempfile.TemporaryDirectory()\r\n    atexit.register(partial(tmpdir.cleanup))\r\n    command = (\r\n        'gcloud',\r\n        'beta',\r\n        'emulators',\r\n        'datastore',\r\n        'start',\r\n        '--project', os.environ['GOOGLE_CLOUD_PROJECT'],\r\n        '--no-store-on-disk',\r\n        '--data-dir', tmpdir.name,\r\n        '--consistency', '1.0'\r\n    )\r\n    proc = subprocess.Popen(\r\n        command,\r\n        stderr=subprocess.PIPE,\r\n        preexec_fn=os.setsid)\r\n    atexit.register(partial(os.killpg, os.getpgid(proc.pid), signal.SIGTERM))\r\n    logs = []\r\n    for lineno, line in enumerate(proc.stderr):\r\n        line = line.decode()\r\n        logs.append(line)\r\n        if 'Dev App Server is now running' in line:\r\n            return tmpdir\r\n        # At the time of writing, 22 lines were printed before the\r\n        # server was up and running. A small buffer is allowed in case\r\n        # changes are introduced in the logging output.\r\n        if lineno \u003e 40:\r\n            break\r\n    raise RuntimeError('Unable to start datastore\\n' + ''.join(logs))\r\n```","author":{"url":"https://github.com/remcohaszing","@type":"Person","name":"remcohaszing"},"datePublished":"2017-04-11T08:08:44.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":11},"url":"https://github.com/3284/google-cloud-python/issues/3284"}