René's URL Explorer Experiment


Title: `Client.query()` sometimes raises 404, despite the query job being successfully started · Issue #2134 · googleapis/python-bigquery · GitHub

Open Graph Title: `Client.query()` sometimes raises 404, despite the query job being successfully started · Issue #2134 · googleapis/python-bigquery

X Title: `Client.query()` sometimes raises 404, despite the query job being successfully started · Issue #2134 · googleapis/python-bigquery

Description: Environment details OS type and version: Google Colab Python version: python --version 3.11 pip version: pip --version N/A google-cloud-bigquery version: pip show google-cloud-bigquery 3.25.0 Steps to reproduce It's a bit difficult to re...

Open Graph Description: Environment details OS type and version: Google Colab Python version: python --version 3.11 pip version: pip --version N/A google-cloud-bigquery version: pip show google-cloud-bigquery 3.25.0 Steps...

X Description: Environment details OS type and version: Google Colab Python version: python --version 3.11 pip version: pip --version N/A google-cloud-bigquery version: pip show google-cloud-bigquery 3.25.0 Steps...

Opengraph URL: https://github.com/googleapis/python-bigquery/issues/2134

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`Client.query()` sometimes raises 404, despite the query job being successfully started","articleBody":"#### Environment details\n\n  - OS type and version: Google Colab\n  - Python version: `python --version` 3.11\n  - pip version: `pip --version` N/A\n  - `google-cloud-bigquery` version: `pip show google-cloud-bigquery` 3.25.0\n\n#### Steps to reproduce\n\nIt's a bit difficult to reproduce, but it seems to happen most often with creating a vector index via [CREATE VECTOR INDEX DDL](https://cloud.google.com/bigquery/docs/vector-index#create_a_vector_index) as generated by [bigframes.bigquery.create_vector_index](https://cloud.google.com/python/docs/reference/bigframes/latest/bigframes.bigquery#bigframes_bigquery_create_vector_index). \n\n#### Code example\n\n\u003cdetails>\u003csummary>Long Python sample, sorry\u003c/summary>\n\n```python\nPROJECT_ID = \"my-project\"  # @param {type:\"string\"}\nDATASET_ID = \"my_dataset\"  # @param {type:\"string\"}\nCONNECTION_ID = \"bqml_llm_conn\"  # @param {type:\"string\"}\n\n# Set the project id\n! gcloud config set project {PROJECT_ID}\n\nREGION = \"US\"  # @param {type: \"string\"}\n\n## Text embedding variables\n\nTEXT_TABLE_ID = \"PATENT_TABLE_BF\"\nTEXT_MODEL_ID = \"llm_embedding_model\"\nTEXT_EMBEDDING_TABLE_ID = \"patent_embedding_BF_test\"\nTEXT_EMBEDDING_TABLE_ID2 = \"patent_embedding\"\nTEXT_MODEL_ENDPOINT = \"text-embedding-004\"\nTEXT_VECTOR_INDEX_ID = \"patent_index_BF\"\n\nimport bigframes.pandas as bf\nimport bigframes.ml as bf_ml\nimport bigframes.bigquery as bf_bq\nimport bigframes.ml.llm as bf_llm\n\n\nfrom google.cloud import bigquery\nfrom google.cloud import storage\n\n# Construct a BigQuery client object.\nclient = bigquery.Client()\n\nimport pandas as pd\nfrom IPython.display import Image, display\nfrom PIL import Image as PILImage\nimport io\n\nbf.options.bigquery.project = PROJECT_ID\nbf.options.bigquery.location = REGION\nbf.options.bigquery.ordering_mode = \"partial\"\n\ntext_model = bf_llm.TextEmbeddingGenerator() ## which endpoint this uses??\n\npublications = bf.read_gbq('patents-public-data.google_patents_research.publications')\nkeep = (publications.embedding_v1.str.len() > 0) & (publications.title.str.len() > 0) & (publications.abstract.str.len() > 0)\npublications = publications[[\"publication_number\", \"title\", \"abstract\"]].rename(columns={'abstract': 'content'})\n\n## creating embeddings for 10000 rows only for testing, change in the line below for embedding higher number of rows\npublications_subset = publications[publications[\"content\"].str.len() > 30].peek(10000)\n## publications_subset = publications_subset[[\"publication_number\", \"title\", \"abstract\"]].rename(columns={'abstract': 'content'})\n\n\nembedding = text_model.predict(publications_subset)[[\"publication_number\", \"title\", \"content\", \"ml_generate_embedding_result\",\"ml_generate_embedding_status\"]]\n\n## filter out rows where the embedding generation failed. the embedding status value is empty if the embedding generation was successful\nembedding = embedding[~embedding[\"ml_generate_embedding_status\"].isnull()]\n\n# store embeddings in a BQ table\nembedding.to_gbq(f\"{DATASET_ID}.{TEXT_EMBEDDING_TABLE_ID}\", if_exists='replace')\n\n## python code to create index\nbf_bq.create_vector_index(\n    table_id = f\"{DATASET_ID}.{TEXT_EMBEDDING_TABLE_ID}\",\n    column_name = \"ml_generate_embedding_result\",\n    replace= True,\n    index_name = \"bf_python_index\",\n    distance_type=\"cosine\",\n    index_type= \"ivf\"\n)\n```\n\n\u003c/details>\n\nUnfortunately, even this is not a guarantee, but it does seem pretty reproducible on one of the projects my colleague is using.\n\n#### Stack trace\n\n```\n/usr/local/lib/python3.10/dist-packages/bigframes/session/_io/bigquery/__init__.py in start_query_with_client(bq_client, sql, job_config, max_results, timeout, api_name, metrics)\n    229 \n    230     try:\n--> 231         query_job = bq_client.query(sql, job_config=job_config, timeout=timeout)\n    232     except google.api_core.exceptions.Forbidden as ex:\n    233         if \"Drive credentials\" in ex.message:\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/client.py in query(self, query, job_config, job_id, job_id_prefix, location, project, retry, timeout, job_retry, api_method)\n   3490             )\n   3491         elif api_method == enums.QueryApiMethod.INSERT:\n-> 3492             return _job_helpers.query_jobs_insert(\n   3493                 self,\n   3494                 query,\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/_job_helpers.py in query_jobs_insert(client, query, job_config, job_id, job_id_prefix, location, project, retry, timeout, job_retry)\n    157             return query_job\n    158 \n--> 159     future = do_query()\n    160     # The future might be in a failed state now, but if it's\n    161     # unrecoverable, we'll find out when we ask for it's result, at which\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/_job_helpers.py in do_query()\n    143 \n    144             try:\n--> 145                 query_job = client.get_job(\n    146                     job_id,\n    147                     project=project,\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/client.py in get_job(self, job_id, project, location, retry, timeout)\n   2199         span_attributes = {\"path\": path, \"job_id\": job_id, \"location\": location}\n   2200 \n-> 2201         resource = self._call_api(\n   2202             retry,\n   2203             span_name=\"BigQuery.getJob\",\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/client.py in _call_api(self, retry, span_name, span_attributes, job_ref, headers, **kwargs)\n    831                 name=span_name, attributes=span_attributes, client=self, job_ref=job_ref\n    832             ):\n--> 833                 return call()\n    834 \n    835         return call()\n\n/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py in retry_wrapped_func(*args, **kwargs)\n    291                 self._initial, self._maximum, multiplier=self._multiplier\n    292             )\n--> 293             return retry_target(\n    294                 target,\n    295                 self._predicate,\n\n/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py in retry_target(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\n    151         except Exception as exc:\n    152             # defer to shared logic for handling errors\n--> 153             _retry_error_helper(\n    154                 exc,\n    155                 deadline,\n\n/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_base.py in _retry_error_helper(exc, deadline, next_sleep, error_list, predicate_fn, on_error_fn, exc_factory_fn, original_timeout)\n    210             original_timeout,\n    211         )\n--> 212         raise final_exc from source_exc\n    213     if on_error_fn is not None:\n    214         on_error_fn(exc)\n\n/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py in retry_target(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\n    142     for sleep in sleep_generator:\n    143         try:\n--> 144             result = target()\n    145             if inspect.isawaitable(result):\n    146                 warnings.warn(_ASYNC_RETRY_WARNING)\n\n/usr/local/lib/python3.10/dist-packages/google/cloud/_http/__init__.py in api_request(self, method, path, query_params, data, content_type, headers, api_base_url, api_version, expect_json, _target_object, timeout, extra_api_info)\n    492 \n    493         if not 200 \u003c= response.status_code \u003c 300:\n--> 494             raise exceptions.from_http_response(response)\n    495 \n    496         if expect_json and response.content:\n\nNotFound: 404 GET https://bigquery.googleapis.com/bigquery/v2/projects/my-project/jobs/c540a00e-05ed-4caf-b76c-b23d16b9617f?projection=full&location=US&prettyPrint=false: Not found: Job my-project:US.c540a00e-05ed-4caf-b76c-b23d16b9617f\n```\n\nNote that it is getting a 404 here: https://github.com/googleapis/python-bigquery/blob/b03a2afabde7f42be45f62fabd3dc0e6a9a493e1/google/cloud/bigquery/_job_helpers.py#L145\n\nThis is after a Conflict exception, meaning the job was created successfully, the retry failed because it already exists, but then getting that already existing job failed with 404. Something funky is going on with the networking I think for this case to happen, but it should be possible to simulate in unit tests, at least.\n","author":{"url":"https://github.com/tswast","@type":"Person","name":"tswast"},"datePublished":"2025-02-18T20:05:43Z","url":"https://github.com/googleapis/python-bigquery/issues/2134"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:b5b1416b-bc9f-83eb-72df-2ac28441aafb
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id894C:245A7:10D11C4:1661287:6AFEA47D
html-safe-nonce792e3dda61fa60d184d2d43a71cc65d070d14c508d800ca72ad952a5c8468920
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4OTRDOjI0NUE3OjEwRDExQzQ6MTY2MTI4Nzo2QUZFQTQ3RCIsInZpc2l0b3JfaWQiOiI4OTY0MDgwMzM2MDk0Nzk4OTczIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmace06f465e5f82236009875049fa267ed14602c31dd08e459930cd5290fb0ba9ee
hovercard-subject-tagissue:2861530571
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/python-bigquery/2134/issue_layout
twitter:imagehttps://opengraph.githubassets.com/6c9569024b937ddf40321af990429c987a6198696c50364da033193254cdaab2/googleapis/python-bigquery/issues/2134
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/6c9569024b937ddf40321af990429c987a6198696c50364da033193254cdaab2/googleapis/python-bigquery/issues/2134
og:image:altEnvironment details OS type and version: Google Colab Python version: python --version 3.11 pip version: pip --version N/A google-cloud-bigquery version: pip show google-cloud-bigquery 3.25.0 Steps...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernametswast
hostnamegithub.com
expected-hostnamegithub.com
None95b2753c978240785d6cca00783c00a371b2d7d9879bb32e305170a1b19bd28e
turbo-cache-controlno-preview
go-importgithub.com/googleapis/python-bigquery git https://github.com/googleapis/python-bigquery.git
octolytics-dimension-user_id16785467
octolytics-dimension-user_logingoogleapis
octolytics-dimension-repository_id226992475
octolytics-dimension-repository_nwogoogleapis/python-bigquery
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id226992475
octolytics-dimension-repository_network_root_nwogoogleapis/python-bigquery
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release3f25273206cd841171bca0002261ae4c9757d934
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/googleapis/python-bigquery/issues/2134#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-bigquery%2Fissues%2F2134
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-bigquery%2Fissues%2F2134
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=googleapis%2Fpython-bigquery
Reloadhttps://github.com/googleapis/python-bigquery/issues/2134
Reloadhttps://github.com/googleapis/python-bigquery/issues/2134
Reloadhttps://github.com/googleapis/python-bigquery/issues/2134
googleapishttps://github.com/googleapis
python-bigqueryhttps://github.com/googleapis/python-bigquery
Codehttps://github.com/googleapis/python-bigquery
Issueshttps://github.com/googleapis/python-bigquery/issues
Pull requestshttps://github.com/googleapis/python-bigquery/pulls
Actionshttps://github.com/googleapis/python-bigquery/actions
Projectshttps://github.com/googleapis/python-bigquery/projects
Security and qualityhttps://github.com/googleapis/python-bigquery/security
Insightshttps://github.com/googleapis/python-bigquery/insights
https://github.com/googleapis/python-bigquery
#2135https://github.com/googleapis/python-bigquery/pull/2135
Client.query() sometimes raises 404, despite the query job being successfully startedhttps://github.com/googleapis/python-bigquery/issues/2134#top
#2135https://github.com/googleapis/python-bigquery/pull/2135
https://github.com/tswast
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22priority%3A%20p1%22
type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20bug%22
https://github.com/tswast
tswasthttps://github.com/tswast
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#issue-2861530571
CREATE VECTOR INDEX DDLhttps://cloud.google.com/bigquery/docs/vector-index#create_a_vector_index
bigframes.bigquery.create_vector_indexhttps://cloud.google.com/python/docs/reference/bigframes/latest/bigframes.bigquery#bigframes_bigquery_create_vector_index
python-bigquery/google/cloud/bigquery/_job_helpers.pyhttps://github.com/googleapis/python-bigquery/blob/b03a2afabde7f42be45f62fabd3dc0e6a9a493e1/google/cloud/bigquery/_job_helpers.py#L145
b03a2afhttps://github.com/googleapis/python-bigquery/commit/b03a2afabde7f42be45f62fabd3dc0e6a9a493e1
tswasthttps://github.com/tswast
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#event-16354983305
product-auto-labelhttps://github.com/apps/product-auto-label
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#event-16354984454
tswasthttps://github.com/tswast
type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20bug%22
priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22priority%3A%20p1%22
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#event-16354987410
https://github.com/tswast
tswasthttps://github.com/tswast
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#issuecomment-2666825915
https://github.com/tswast
tswasthttps://github.com/tswast
on Feb 18, 2025https://github.com/googleapis/python-bigquery/issues/2134#issuecomment-2666963676
tswasthttps://github.com/tswast
completedhttps://github.com/googleapis/python-bigquery/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#2135https://github.com/googleapis/python-bigquery/pull/2135
on Feb 26, 2025https://github.com/googleapis/python-bigquery/issues/2134#event-16471523381
Sign up for freehttps://github.com/signup?return_to=https://github.com/googleapis/python-bigquery/issues/2134
Sign in to commenthttps://github.com/login?return_to=https://github.com/googleapis/python-bigquery/issues/2134
tswasthttps://github.com/tswast
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22priority%3A%20p1%22
type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20bug%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.