Title: Big Query RowIterator offset error · Issue #18601 · googleapis/google-cloud-python · GitHub
Open Graph Title: Big Query RowIterator offset error · Issue #18601 · googleapis/google-cloud-python
X Title: Big Query RowIterator offset error · Issue #18601 · googleapis/google-cloud-python
Description: Determine this is the right repository I determined this is the correct repository in which to report this bug. Summary of the issue Context After a recent update fetching Big Query data in pages raises an Exception depending on page siz...
Open Graph Description: Determine this is the right repository I determined this is the correct repository in which to report this bug. Summary of the issue Context After a recent update fetching Big Query data in pages r...
X Description: Determine this is the right repository I determined this is the correct repository in which to report this bug. Summary of the issue Context After a recent update fetching Big Query data in pages r...
Opengraph URL: https://github.com/googleapis/google-cloud-python/issues/18601
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Big Query RowIterator offset error","articleBody":"### Determine this is the right repository\n\n- [x] I determined this is the correct repository in which to report this bug.\n\n### Summary of the issue\n\n**Context**\nAfter a recent update fetching Big Query data in pages raises an Exception depending on page size.\n\n**Expected Behavior:**\nNo Exceptions\n\n**Actual Behavior:**\nBadRequest: When using a page token, you cannot specify an arbitrary startIndex.\n\n\n### API client name and version\n\ngoogle-api-core 2.41.0\ngoogle-auth 2.60.0\ngoogle-cloud-aiplatform 2.4.0\ngoogle-cloud-api-keys 0.9.1\ngoogle-cloud-bigquery 3.46.1\ngoogle-cloud-bigquery-storage 2.42.0\ngoogle-cloud-core 2.8.0\ngoogle-cloud-modelarmor 0.7.2\ngoogle-cloud-resource-manager 1.19.0\ngoogle-cloud-run 0.16.2\ngoogle-cloud-secret-manager 2.31.0\ngoogle-cloud-storage 3.16.0\ngoogle-crc32c 1.9.0\ngoogle-genai 2.28.0\ngoogle-resumable-media 2.11.0\ngoogleapis-common-protos 1.75.5\ngrpc-google-iam-v1 0.14.5\nkeyrings.google-artifactregistry-auth 1.1.2\n\n### Reproduction steps: code\n\n```python\nfrom google.cloud import bigquery\n\nTABLE_NAME = ... # mine has 3406 rows\nquery = f\"\"\"SELECT * FROM `{TABLE_NAME}`\"\"\"\n\nclient = bigquery.Client()\njob = client.query(query)\nwhile not job.done():\n await asyncio.sleep(1)\n\npage_size = 285 # \u003c= 280 works for me, >= 285 does not\noffset = 0\n\nrows = []\nwhile True:\n page = job.result(start_index=offset, max_results=page_size) \n total_rows = page.total_rows\n rows.extend(list(page))\n print(len(rows), total_rows)\n offset += page.num_results\n if page.num_results \u003c page_size:\n break\n```\n\n### Reproduction steps: supporting files\n\n_No response_\n\n### Reproduction steps: actual results\n\n```\n285 3406\n570 3406\n855 3406\n1140 3406\n---------------------------------------------------------------------------\nBadRequest Traceback (most recent call last)\nCell In[54], line 18\n 14 rows = []\n 15 while True:\n 16 page = job.result(start_index=offset, max_results=page_size)\n 17 total_rows = page.total_rows\n---> 18 rows.extend(list(page))\n 19 print(len(rows), total_rows)\n 20 offset += page.num_results\n 21 if page.num_results \u003c page_size:\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\page_iterator.py:208, in Iterator._items_iter(self)\n 206 def _items_iter(self):\n 207 \"\"\"Iterator for each item returned.\"\"\"\n--> 208 for page in self._page_iter(increment=False):\n 209 for item in page:\n 210 self.num_results += 1\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\page_iterator.py:250, in Iterator._page_iter(self, increment)\n 248 self.num_results += page.num_items\n 249 yield page\n--> 250 page = self._next_page()\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\page_iterator.py:373, in HTTPIterator._next_page(self)\n 366 \"\"\"Get the next page in the iterator.\n 367 \n 368 Returns:\n 369 Optional[Page]: The next page in the iterator or :data:`None` if\n 370 there are no pages left.\n 371 \"\"\"\n 372 if self._has_next_page():\n--> 373 response = self._get_next_page_response()\n 374 items = response.get(self._items_key, ())\n 375 page = Page(self, items, self.item_to_value, raw_page=response)\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\cloud\\bigquery\\table.py:2171, in RowIterator._get_next_page_response(self)\n 2168 if self.page_number and \"startIndex\" in params:\n 2169 del params_copy[\"startIndex\"]\n-> 2171 return self.api_request(\n 2172 method=self._HTTP_METHOD, path=self.path, query_params=params_copy\n 2173 )\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\cloud\\bigquery\\client.py:917, in Client._call_api(self, retry, span_name, span_attributes, job_ref, headers, **kwargs)\n 912 with create_span(\n 913 name=span_name, attributes=span_attributes, client=self, job_ref=job_ref\n 914 ):\n 915 return call()\n--> 917 return call()\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\retry\\retry_unary.py:295, in Retry.__call__.\u003clocals>.retry_wrapped_func(*args, **kwargs)\n 291 target = functools.partial(func, *args, **kwargs)\n 292 sleep_generator = exponential_sleep_generator(\n 293 self._initial, self._maximum, multiplier=self._multiplier\n 294 )\n--> 295 return retry_target(\n 296 target,\n 297 self._predicate,\n 298 sleep_generator,\n 299 timeout=self._timeout,\n 300 on_error=on_error,\n 301 )\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\retry\\retry_unary.py:157, in retry_target(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\n 153 # pylint: disable=broad-except\n 154 # This function explicitly must deal with broad exceptions.\n 155 except Exception as exc:\n 156 # defer to shared logic for handling errors\n--> 157 next_sleep = _retry_error_helper(\n 158 exc,\n 159 deadline,\n 160 sleep_iter,\n 161 error_list,\n 162 predicate,\n 163 on_error,\n 164 exception_factory,\n 165 timeout,\n 166 )\n 167 # if exception not raised, sleep before next attempt\n 168 time.sleep(next_sleep)\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\retry\\retry_base.py:215, in _retry_error_helper(exc, deadline, sleep_iterator, error_list, predicate_fn, on_error_fn, exc_factory_fn, original_timeout)\n 209 if not predicate_fn(exc):\n 210 final_exc, source_exc = exc_factory_fn(\n 211 error_list,\n 212 RetryFailureReason.NON_RETRYABLE_ERROR,\n 213 original_timeout,\n 214 )\n--> 215 raise final_exc from source_exc\n 216 if on_error_fn is not None:\n 217 on_error_fn(exc)\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\api_core\\retry\\retry_unary.py:148, in retry_target(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\n 146 while True:\n 147 try:\n--> 148 result = target()\n 149 if inspect.isawaitable(result):\n 150 warnings.warn(_ASYNC_RETRY_WARNING)\n\nFile ~\\AppData\\Local\\miniforge3\\envs\\test-big-query\\Lib\\site-packages\\google\\cloud\\_http\\__init__.py:509, in JSONConnection.api_request(self, method, path, query_params, data, content_type, headers, api_base_url, api_version, expect_json, _target_object, timeout, extra_api_info)\n 497 response = self._make_request(\n 498 method=method,\n 499 url=url,\n (...) 505 extra_api_info=extra_api_info,\n 506 )\n 508 if not 200 \u003c= response.status_code \u003c 300:\n--> 509 raise exceptions.from_http_response(response)\n 511 if expect_json and response.content:\n 512 return response.json()\n\nBadRequest: 400 GET https://bigquery.googleapis.com/bigquery/v2/projects/gcp-wow-people-de-lab-dev/queries/d3576736-a9dc-4958-b31c-05abd63aff38?pageToken=BFNORWQYUEAQAAASA4EAAEEAQCAAKGQGBD7AUEE5AIQLBLQVJKTAECTXBISAUGLHMNYC253PO4WXAZLPOBWGKLLEMUWWYYLCFVSGK5QRYJQGXWCDAAAAAERJL42GEYJVGA4GGZLGGFRWIMJWG4YTEODBMQ2TCNJQG43WINDGHFQTSZJVGE3GGMZSGQNCIY3CMQ2DQYRQMMWTCMLEMQWTIODFHAWTSOLGMYWWKYRTGM4DMMJZG43DOZASIRQW433OGFQWEMJTGU2DMYJYHEYDIMBXME2WIMJUMI2TAOLCGBRDAMBTMY4TMYTBGY4WCN3DHA4TANTBGM2WGZTBMVQWKZLGG4YDIYZZGNRTENA2MUYWCYRRGM2TINTBHA4TANBQG5QTKZBRGRRDKMBZMIYGEMBQGNTDSNTCME3DSYJXMM4DSMBWMEZTKY3GMFSWCZLFMY3TANDDHEZWGMRUENTDEZDEGE3TQZJNGYZTOYJNGRRWMNBNMIYDKMRNMMZTGZDEGJTDMMJZGY2A%3D%3D%3D%3D&maxResults=19&fields=jobReference%2CtotalRows%2CpageToken%2Crows&location=US&startIndex=1140&formatOptions.useInt64Timestamp=True&prettyPrint=false: When using a page token, you cannot specify an arbitrary startIndex.\n```\n### Python environment\n\n3.13.14\n","author":{"url":"https://github.com/asmith-wowcorp","@type":"Person","name":"asmith-wowcorp"},"datePublished":"2026-10-08T00:22:57Z","url":"https://github.com/googleapis/google-cloud-python/issues/18601"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:ba4bdade-9f39-1588-051b-7829899c8e43 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | EE46:203319:28DE3F:378A73:6AFE1A06 |
| html-safe-nonce | 1199a996347c360f97c1f929cb38047eec8ba7f882cde4a024f6e0e59612276d |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFRTQ2OjIwMzMxOToyOERFM0Y6Mzc4QTczOjZBRkUxQTA2IiwidmlzaXRvcl9pZCI6IjIxODgzMzg5NzY0NDI0MjM4MTQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | bc70bdbc5cc864144394660c825511df47b4c8a0ef92092d5277a465d1220be0 |
| hovercard-subject-tag | issue:5753806727 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/google-cloud-python/18601/issue_layout |
| twitter:image | https://opengraph.githubassets.com/c4337a05f30a989a6af8e507d11cfae92db001a82f08f63da6663f1f4f9488e4/googleapis/google-cloud-python/issues/18601 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/c4337a05f30a989a6af8e507d11cfae92db001a82f08f63da6663f1f4f9488e4/googleapis/google-cloud-python/issues/18601 |
| og:image:alt | Determine this is the right repository I determined this is the correct repository in which to report this bug. Summary of the issue Context After a recent update fetching Big Query data in pages r... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | asmith-wowcorp |
| hostname | github.com |
| expected-hostname | github.com |
| None | d0525ab1464526bf4532a5551aa348cf100dff75ef808a96fc67a08b1d946aa9 |
| turbo-cache-control | no-preview |
| go-import | github.com/googleapis/google-cloud-python git https://github.com/googleapis/google-cloud-python.git |
| octolytics-dimension-user_id | 16785467 |
| octolytics-dimension-user_login | googleapis |
| octolytics-dimension-repository_id | 16316451 |
| octolytics-dimension-repository_nwo | googleapis/google-cloud-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 16316451 |
| octolytics-dimension-repository_network_root_nwo | googleapis/google-cloud-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | fef3993a71fa20f7be8e167132fbf9619bc66e12 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width