René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Calls to `create_dataset` and `get_table` are timing out","articleBody":"We're getting some frequent test failures in the pandas integration tests due to timed out requests, possibly due to https://github.com/googleapis/google-cloud-python/pull/10219 which added non-`None` default timeout.\r\n\r\nIt seems 60 seconds is not enough for these calls (but strange that so many API requests are timing out).\r\n\r\n#### Environment details\r\n\r\nContext: https://github.com/pydata/pandas-gbq/issues/309\r\n\r\n#### Steps to reproduce\r\n\r\n  1. Make frequent calls to `create_dataset`.\r\n  2. Observe flakey tests.\r\n\r\n#### Stack trace\r\n\r\n```\r\n        if new_retry.is_exhausted():\r\n\u003e           raise MaxRetryError(_pool, url, error or ResponseError(cause))\r\nE           urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bigquery.googleapis.com', port=443): Max retries exceeded with url: /bigquery/v2/projects/pandas-travis/datasets/pydata_pandas_bq_testing_dddqwbgrjz/tables/hrsctwrmva (Caused by NewConnectionError('\u003curllib3.connection.VerifiedHTTPSConnection object at 0x7ff87946e898\u003e: Failed to establish a new connection: [Errno 110] Connection timed out',))\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/urllib3/util/retry.py:436: MaxRetryError\r\nDuring handling of the above exception, another exception occurred:\r\nself = \u003cpandas.tests.io.test_gbq.TestToGBQIntegrationWithServiceAccountKeyPath object at 0x7ff879725160\u003e\r\ngbq_dataset = 'pydata_pandas_bq_testing_dddqwbgrjz.hrsctwrmva'\r\n    def test_roundtrip(self, gbq_dataset):\r\n        destination_table = gbq_dataset\r\n    \r\n        test_size = 20001\r\n        df = make_mixed_dataframe_v2(test_size)\r\n    \r\n        df.to_gbq(\r\n            destination_table,\r\n            _get_project_id(),\r\n            chunksize=None,\r\n\u003e           credentials=_get_credentials(),\r\n        )\r\npandas/tests/io/test_gbq.py:185: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\npandas/core/frame.py:1551: in to_gbq\r\n    credentials=credentials,\r\npandas/io/gbq.py:219: in to_gbq\r\n    private_key=private_key,\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas_gbq/gbq.py:1202: in to_gbq\r\n    if table.exists(table_id):\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas_gbq/gbq.py:1312: in exists\r\n    self.client.get_table(table_ref)\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/cloud/bigquery/client.py:679: in get_table\r\n    retry, method=\"GET\", path=table_ref.path, timeout=timeout\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/cloud/bigquery/client.py:556: in _call_api\r\n    return call()\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/api_core/retry.py:286: in retry_wrapped_func\r\n    on_error=on_error,\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/api_core/retry.py:184: in retry_target\r\n    return target()\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/cloud/_http.py:419: in api_request\r\n    timeout=timeout,\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/cloud/_http.py:277: in _make_request\r\n    method, url, headers, data, target_object, timeout=timeout\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/cloud/_http.py:315: in _do_request\r\n    url=url, method=method, headers=headers, data=data, timeout=timeout\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/google/auth/transport/requests.py:317: in request\r\n    **kwargs\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/requests/sessions.py:533: in request\r\n    resp = self.send(prep, **send_kwargs)\r\n../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/requests/sessions.py:646: in send\r\n    r = adapter.send(request, **kwargs)\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n```\r\n\r\n","author":{"url":"https://github.com/tswast","@type":"Person","name":"tswast"},"datePublished":"2020-02-07T19:18:49.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":12},"url":"https://github.com/34/python-bigquery/issues/34"}