René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"python-api-core 2.16.0 introduced timeout on pooling","articleBody":"Hello,\r\n\r\nWe are using python-api-core via [python-bigquery](https://github.com/googleapis/python-bigquery) and since we upgraded to 2.16+ version of python-api-core, we experience timeout on `to_dataframe()` method (in python-bigquery) for very large queries. After investigation this seems to be a regression in python-api-core with the introduction of a 900s (15min) timeout (in pooling.py).\r\n\r\n#### Environment details\r\n\r\n  - OS type and version: macOS / linux\r\n  - Python version: 3.10.10\r\n  - pip version: 24.0\r\n  - `google-cloud-bigquery` version: 3.6.0\r\n\r\n#### Steps to reproduce\r\n\r\n- Install google api core \u003e 2.15\r\n- Use bigquery client to get in a dataframe data from a query that takes more than 15 minutes to run\r\n\r\n#### Code example\r\n\r\n1. \r\n```bash\r\npip install google-api-core==2.16.2\r\npip install google-cloud-bigquery[bqstorage,pandas]==3.6.0\r\n```\r\n2. \r\n```python\r\nfrom google.cloud import bigquery\r\nclient = bigquery.Client()\r\n\r\nsql = \"SELECT ...\" # Long running that takes more than 15 minutes to process in bq\r\n\r\nclient.query(sql).to_dataframe()\r\n```\r\n#### Stack trace\r\n```\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/job/query.py\", line 1687, in to_dataframe\r\n    query_result = wait_for_query(self, progress_bar_type, max_results=max_results)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/_tqdm_helpers.py\", line 104, in wait_for_query\r\n    return query_job.result(max_results=max_results)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/job/query.py\", line 1499, in result\r\n    do_get_result()\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py\", line 293, in retry_wrapped_func\r\n    return retry_target(\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py\", line 153, in retry_target\r\n    _retry_error_helper(\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/retry/retry_base.py\", line 212, in _retry_error_helper\r\n    raise final_exc from source_exc\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/retry/retry_unary.py\", line 144, in retry_target\r\n    result = target()\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/job/query.py\", line 1489, in do_get_result\r\n    super(QueryJob, self).result(retry=retry, timeout=timeout)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/job/base.py\", line 728, in result\r\n    return super(_AsyncJob, self).result(timeout=timeout, **kwargs)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/future/polling.py\", line 256, in result\r\n    self._blocking_poll(timeout=timeout, retry=retry, polling=polling)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/cloud/bigquery/job/query.py\", line 1245, in _blocking_poll\r\n    super(QueryJob, self)._blocking_poll(timeout=timeout, **kwargs)\r\n  File \"/usr/local/lib/python3.10/site-packages/google/api_core/future/polling.py\", line 139, in _blocking_poll\r\n    raise concurrent.futures.TimeoutError(\r\nconcurrent.futures._base.TimeoutError: Operation did not complete within the designated timeout of 900 seconds.\r\n```","author":{"url":"https://github.com/mautini","@type":"Person","name":"mautini"},"datePublished":"2024-02-08T12:50:37.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/602/python-api-core/issues/602"}