René's URL Explorer Experiment


Title: Pagination should allow fetching pages in batch of N pages. · Issue #112 · googleapis/python-bigquery · GitHub

Open Graph Title: Pagination should allow fetching pages in batch of N pages. · Issue #112 · googleapis/python-bigquery

X Title: Pagination should allow fetching pages in batch of N pages. · Issue #112 · googleapis/python-bigquery

Description: Currently I am facing an issue where I have to process big number of records coming as part of query result. The results are available in multiple pages. When I am trying to process the row from result I notice that it takes around 10 se...

Open Graph Description: Currently I am facing an issue where I have to process big number of records coming as part of query result. The results are available in multiple pages. When I am trying to process the row from re...

X Description: Currently I am facing an issue where I have to process big number of records coming as part of query result. The results are available in multiple pages. When I am trying to process the row from re...

Opengraph URL: https://github.com/googleapis/python-bigquery/issues/112

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Pagination should allow fetching pages in batch of N pages.","articleBody":"Currently I am facing an issue where I have to process big number of records coming as part of query result. The results are available in multiple pages.\r\n\r\nWhen I am trying to process the row from result I notice that it takes around 10 seconds to load next page. I am able to process one page in a sec or 2 and then I have to wait for 10 seconds for next page. It is time consuming.\r\n\r\nI cannot load entire result in memory as the response can be more than 10 GB and we start hitting various QUOTA limits.\r\n\r\nIt there a way i can load batch of 100 pages in memory at once and start my process and in the background next batch of 100 pages are loaded during processing on first batch of pages.\r\n\r\n","author":{"url":"https://github.com/yantrikskaran","@type":"Person","name":"yantrikskaran"},"datePublished":"2020-05-17T13:15:01Z","url":"https://github.com/googleapis/python-bigquery/issues/112"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:02ccd56d-3962-4533-c4ba-cef47bd7dadc
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id88EC:24F846:2DDF0B:3B8D36:6AFE3C30
html-safe-nonce45feddb69532fa8accf98ea3e09ae68a4b98d531f8efe21bdb53a8b1fe2d6650
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4OEVDOjI0Rjg0NjoyRERGMEI6M0I4RDM2OjZBRkUzQzMwIiwidmlzaXRvcl9pZCI6IjQ0MTM4MjY4MzY5MjQ4NzM3NzciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac1279e6e2a4c63143a84a15dd889d5ae7f93883dcb97c4078747185041575f1ea
hovercard-subject-tagissue:620728299
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/python-bigquery/112/issue_layout
twitter:imagehttps://opengraph.githubassets.com/cc3fe6c31ea820b23910545fc0555b85afe35e8d79a3e45f164301ee19a872f8/googleapis/python-bigquery/issues/112
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/cc3fe6c31ea820b23910545fc0555b85afe35e8d79a3e45f164301ee19a872f8/googleapis/python-bigquery/issues/112
og:image:altCurrently I am facing an issue where I have to process big number of records coming as part of query result. The results are available in multiple pages. When I am trying to process the row from re...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameyantrikskaran
hostnamegithub.com
expected-hostnamegithub.com
Nonef1a5901e2377b0af2a0b303f9528678f1475d04b9219dca1e993344196eaa07c
turbo-cache-controlno-preview
go-importgithub.com/googleapis/python-bigquery git https://github.com/googleapis/python-bigquery.git
octolytics-dimension-user_id16785467
octolytics-dimension-user_logingoogleapis
octolytics-dimension-repository_id226992475
octolytics-dimension-repository_nwogoogleapis/python-bigquery
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id226992475
octolytics-dimension-repository_network_root_nwogoogleapis/python-bigquery
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasec4447fdce9ed0f33af959d2b6680d0bc18212a6a
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/googleapis/python-bigquery/issues/112#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-bigquery%2Fissues%2F112
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgoogleapis%2Fpython-bigquery%2Fissues%2F112
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=googleapis%2Fpython-bigquery
Reloadhttps://github.com/googleapis/python-bigquery/issues/112
Reloadhttps://github.com/googleapis/python-bigquery/issues/112
Reloadhttps://github.com/googleapis/python-bigquery/issues/112
googleapishttps://github.com/googleapis
python-bigqueryhttps://github.com/googleapis/python-bigquery
Codehttps://github.com/googleapis/python-bigquery
Issueshttps://github.com/googleapis/python-bigquery/issues
Pull requestshttps://github.com/googleapis/python-bigquery/pulls
Actionshttps://github.com/googleapis/python-bigquery/actions
Projectshttps://github.com/googleapis/python-bigquery/projects
Security and qualityhttps://github.com/googleapis/python-bigquery/security
Insightshttps://github.com/googleapis/python-bigquery/insights
https://github.com/googleapis/python-bigquery
Pagination should allow fetching pages in batch of N pages.https://github.com/googleapis/python-bigquery/issues/112#top
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
type: questionRequest for information or clarification. Not an issue.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20question%22
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 17, 2020https://github.com/googleapis/python-bigquery/issues/112#issue-620728299
https://github.com/plamut
plamuthttps://github.com/plamut
on May 18, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-629968273
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 18, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-629994963
plamuthttps://github.com/plamut
googleapis/google-cloud-pythonhttps://github.com/googleapis/google-cloud-python
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#event-3350363572
product-auto-labelhttps://github.com/apps/product-auto-label
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#event-3350363858
plamuthttps://github.com/plamut
type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20feature%20request%22
type: questionRequest for information or clarification. Not an issue.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20question%22
type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20feature%20request%22
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#event-3350364678
https://github.com/plamut
plamuthttps://github.com/plamut
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-630632655
@yantrikskaranhttps://github.com/yantrikskaran
result()https://github.com/googleapis/python-bigquery/blob/38a5c01ca830daf165592357c45f2fb4016aad23/google/cloud/bigquery/job.py#L3162-L3164
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-630648364
@plamuthttps://github.com/plamut
https://github.com/plamut
plamuthttps://github.com/plamut
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-630861619
@yantrikskaranhttps://github.com/yantrikskaran
become a defaulthttps://github.com/googleapis/python-bigquery/pull/55
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-630879638
@plamuthttps://github.com/plamut
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 19, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-630986084
@plamuthttps://github.com/plamut
https://github.com/plamut
plamuthttps://github.com/plamut
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631281632
@yantrikskaranhttps://github.com/yantrikskaran
public datasetshttps://console.cloud.google.com/marketplace/browse?filter=solution-type:dataset&_ga=2.86286808.170615762.1589870841-1482347578.1554420678&pli=1
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631284726
@yantrikskaranhttps://github.com/yantrikskaran
public datasetshttps://console.cloud.google.com/marketplace/browse?filter=solution-type:dataset&_ga=2.86286808.170615762.1589870841-1482347578.1554420678&pli=1
@plamuthttps://github.com/plamut
https://github.com/plamut
plamuthttps://github.com/plamut
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631381426
@yantrikskaranhttps://github.com/yantrikskaran
iteration logichttps://github.com/googleapis/python-api-core/blob/335c1097b4d697f93ed1088c581981850a66239e/google/api_core/page_iterator.py#L210-L249
https://github.com/plamut
plamuthttps://github.com/plamut
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631395378
@yantrikskaranhttps://github.com/yantrikskaran
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631455079
@plamuthttps://github.com/plamut
https://github.com/emkornfield
emkornfieldhttps://github.com/emkornfield
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631564685
@yantrikskaranhttps://github.com/yantrikskaran
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 20, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-631612579
@yantrikskaranhttps://github.com/yantrikskaran
https://github.com/plamut
plamuthttps://github.com/plamut
on May 22, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-632675580
@yantrikskaranhttps://github.com/yantrikskaran
BQ Storage APIhttps://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/list
https://github.com/yantrikskaran
yantrikskaranhttps://github.com/yantrikskaran
on May 22, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-632901519
@yantrikskaranhttps://github.com/yantrikskaran
BQ Storage APIhttps://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/list
@plamuthttps://github.com/plamut
https://github.com/plamut
plamuthttps://github.com/plamut
on Jun 2, 2020https://github.com/googleapis/python-bigquery/issues/112#issuecomment-637443872
@yantrikskaranhttps://github.com/yantrikskaran
plamuthttps://github.com/plamut
completedhttps://github.com/googleapis/python-bigquery/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Jun 2, 2020https://github.com/googleapis/python-bigquery/issues/112#event-3396962328
JustinBeckwithhttps://github.com/JustinBeckwith
plamuthttps://github.com/plamut
on Feb 2, 2021https://github.com/googleapis/python-bigquery/issues/112#event-4276803972
https://github.com/mrn-aglic
mrn-aglichttps://github.com/mrn-aglic
on Apr 17, 2023https://github.com/googleapis/python-bigquery/issues/112#issuecomment-1511770657
https://github.com/Tunneller
Tunnellerhttps://github.com/Tunneller
on Apr 8, 2024https://github.com/googleapis/python-bigquery/issues/112#issuecomment-2041810934
@plamuthttps://github.com/plamut
https://github.com/plamut
plamuthttps://github.com/plamut
on Apr 8, 2024https://github.com/googleapis/python-bigquery/issues/112#issuecomment-2042291244
@Tunnellerhttps://github.com/Tunneller
plamuthttps://github.com/plamut
on Apr 8, 2024https://github.com/googleapis/python-bigquery/issues/112#event-12389834272
Sign up for freehttps://github.com/signup?return_to=https://github.com/googleapis/python-bigquery/issues/112
Sign in to commenthttps://github.com/login?return_to=https://github.com/googleapis/python-bigquery/issues/112
api: bigqueryIssues related to the googleapis/python-bigquery API.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22api%3A%20bigquery%22
type: questionRequest for information or clarification. Not an issue.https://github.com/googleapis/python-bigquery/issues?q=state%3Aopen%20label%3A%22type%3A%20question%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.