René's URL Explorer Experiment


Title: Stream pickle collection batches like CPython by youknowdot · Pull Request #9691 · RustPython/RustPython · GitHub

Open Graph Title: Stream pickle collection batches like CPython by youknowdot · Pull Request #9691 · RustPython/RustPython

X Title: Stream pickle collection batches like CPython by youknowdot · Pull Request #9691 · RustPython/RustPython

Description: Summary Serialize live list, dict, and set entries so reducers observe later replacements and mutations. Share incremental list/dict iterator batching, including protocol-0 ordering, two-item binary lookahead, tuple validation, and cleanup order. Use reduced iterators directly without another __iter__ call, and select batching by protocol rather than the writable bin flag. Preserve singleton fast paths and detect set size changes. Stop exhausted native cursors instead of emitting empty batches indefinitely after same-size compaction. Reference: CPython 3.15.0 collection batching. Validation Independent source review passed, including callback ordering, object lifetimes, lock release, and native cursor exhaustion. Bounded CPython 3.15 reference probes passed for protocol/batch boundaries, writable bin, mutation visibility, iterator errors, and finalizer ordering. Final focused cargo check --offline --locked -j1 -p rustpython-host_env -p rustpython-vm -p rustpython-stdlib --features rustpython-stdlib/threading passed, as did normal commit hooks and formatting checks. Focused all-targets Clippy completed with four warnings reproduced on unchanged main: needless_collect in posix.rs and two use_self/one uninlined_format_args in elementtree.rs. The changed code adds no remaining warning; strict -D warnings is blocked by this baseline. No warnings were suppressed and unrelated code was not changed. RustPython runtime, full workspace/C-API suites, and cross-platform execution were not run. No tests, Lib files, markers, or workflows changed. Same-size structural mutations remain dependent on the container's native iteration order. This change does not promise CPython's hash-table layout or redesign pickle memo/configuration state. AI assistance: OpenAI Codex performed source comparison, implementation, reference probes, and independent source review; model version was not exposed. Summary by CodeRabbit Bug Fixes Pickle serialization processes lists, dictionaries, and sets from their live contents while retaining checks for collection size changes. Dictionary entries provided through iterators are checked for the expected key-value structure. Serialization errors also include context about the item that caused the error. Collection serialization uses protocol-appropriate output, including incremental handling for larger collections.

Open Graph Description: Summary Serialize live list, dict, and set entries so reducers observe later replacements and mutations. Share incremental list/dict iterator batching, including protocol-0 ordering, two-item bina...

X Description: Summary Serialize live list, dict, and set entries so reducers observe later replacements and mutations. Share incremental list/dict iterator batching, including protocol-0 ordering, two-item bina...

Opengraph URL: https://github.com/RustPython/RustPython/pull/9691

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:3d634fda-afbf-df9e-5ad5-c9894b9721e8
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idBD18:1FE434:155C0C6:1C30A10:6B01FA73
html-safe-nonce839e234de016e1871f1857e3478f62cfe8126a5aba480e1b6566f0186b8f36fb
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCRDE4OjFGRTQzNDoxNTVDMEM2OjFDMzBBMTA6NkIwMUZBNzMiLCJ2aXNpdG9yX2lkIjoiNzQ3OTM1NjI4Njk5NjQ0NTgxMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac0d2230b0b15b8fc5683ca4861440e19e4ac3a9b648ef6efe4e599cee7416da1a
hovercard-subject-tagpull_request:5150889698
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9691/files
twitter:imagehttps://avatars.githubusercontent.com/u/336763669?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/336763669?s=400&v=4
og:image:altSummary Serialize live list, dict, and set entries so reducers observe later replacements and mutations. Share incremental list/dict iterator batching, including protocol-0 ordering, two-item bina...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git
octolytics-dimension-user_id39710557
octolytics-dimension-user_loginRustPython
octolytics-dimension-repository_id135201145
octolytics-dimension-repository_nwoRustPython/RustPython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id135201145
octolytics-dimension-repository_network_root_nwoRustPython/RustPython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/RustPython/RustPython/pull/9691/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F9691%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F9691%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=RustPython%2FRustPython
Reloadhttps://github.com/RustPython/RustPython/pull/9691/files
Reloadhttps://github.com/RustPython/RustPython/pull/9691/files
Reloadhttps://github.com/RustPython/RustPython/pull/9691/files
RustPythonhttps://github.com/RustPython
RustPythonhttps://github.com/RustPython/RustPython
Codehttps://github.com/RustPython/RustPython
Issues195 (195)https://github.com/RustPython/RustPython/issues
Pull requests133 (133)https://github.com/RustPython/RustPython/pulls
Discussionshttps://github.com/RustPython/RustPython/discussions
Actionshttps://github.com/RustPython/RustPython/actions
Projectshttps://github.com/RustPython/RustPython/projects
Wikihttps://github.com/RustPython/RustPython/wiki
Security and qualityhttps://github.com/RustPython/RustPython/security
Insightshttps://github.com/RustPython/RustPython/insights
Sign up for GitHub https://github.com/signup?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
youknowonehttps://github.com/youknowone
RustPython:mainhttps://github.com/RustPython/RustPython/tree/main
youknowdot:fix/pickle-streaming-batches-20261104https://github.com/youknowdot/RustPython/tree/fix/pickle-streaming-batches-20261104
Conversation 7 https://github.com/RustPython/RustPython/pull/9691
Commits 4 https://github.com/RustPython/RustPython/pull/9691/commits
Checks 19 https://github.com/RustPython/RustPython/pull/9691/checks
Files changed https://github.com/RustPython/RustPython/pull/9691/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
Stream pickle collection batches like CPython https://github.com/RustPython/RustPython/pull/9691/files#top
Show all changes 4 commits https://github.com/RustPython/RustPython/pull/9691/files
513c501 Stream pickle collection batches like CPython youknowdot Nov 4, 2026 https://github.com/RustPython/RustPython/pull/9691/commits/513c501b37b999e0dd4ec0493b03516b7e7cd3e9
4d85e32 Mark native set cursor results as must-use youknowdot Nov 4, 2026 https://github.com/RustPython/RustPython/pull/9691/commits/4d85e32cc68e59eb54d29f5b6f9c14265fc47b09
7fb9ed3 Match protocol-zero iterator results directly youknowdot Nov 4, 2026 https://github.com/RustPython/RustPython/pull/9691/commits/7fb9ed3d05ba559140e270d856b6756090dfcd75
bcfa386 Keep the pickle cursor separate from set operation changes youknowdot Nov 4, 2026 https://github.com/RustPython/RustPython/pull/9691/commits/bcfa386ed40a18cfa313940c6b708dabaf1bffb8
Clear filters https://github.com/RustPython/RustPython/pull/9691/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
pickle.rs https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
set.rs https://github.com/RustPython/RustPython/pull/9691/files#diff-ca3250b43726cb4327f8b4f6d413151036abc2f00f7c3197f55432013ca2c07f
crates/stdlib/src/pickle.rshttps://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
View file https://github.com/youknowdot/RustPython/blob/bcfa386ed40a18cfa313940c6b708dabaf1bffb8/crates/stdlib/src/pickle.rs
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/9691/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
coderabbitaihttps://github.com/apps/coderabbitai
Nov 5, 2026https://github.com/RustPython/RustPython/pull/9691/files#r4475989807
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
coderabbitaihttps://github.com/apps/coderabbitai
Nov 5, 2026https://github.com/RustPython/RustPython/pull/9691/files#r4475989829
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
coderabbitaihttps://github.com/apps/coderabbitai
Nov 5, 2026https://github.com/RustPython/RustPython/pull/9691/files#r4475989838
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
https://github.com/RustPython/RustPython/pull/9691/files#diff-3029d63dd60cbab4abbf02b7ddea49000477dfb2399b94ec7d5c021428fbe641
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9691/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.