René's URL Explorer Experiment


Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython · GitHub

Open Graph Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython

X Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython

Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including independent from_buffer acquisitions and aggregate writes that overlap a smaller view. Use weak links to the roots retaining written values, including writes through temporary pointer fields, without extending their lifetime. Amortize cleanup across new registrations and keep the destination owner alive through conversion, writing and ownership publication. Publish actual write ranges after user conversion so nested writes cannot replace the outer operation's range. Reuse validated weak ownership paths for direct pointers, casts and aggregate pointer fields. Cache each field on its retention root, including pointer roots, matching byref casts and private byref arguments copied into fields. Validate the exact private slot on reuse so a surviving sibling owner cannot validate a replaced field's path. Avoid repeated ownership-graph searches for writes into nested storage. Follow buffer-exporter links for borrowed views, including views of pointer contents and writes through pointers targeting those views. Keep both current contents and retained cast storage able to resolve later writes, including differently sized views of the same buffer and retargeting through intermediate pointers or aggregate fields. Lazily index direct retention entries for reads through registered writers, avoiding repeated scans of all retained values after pointer-field writes. The index stores O(N) Rust metadata and validates cached keys against current values without retaining extra Python references. Retain byref arguments privately when casting them, retaining the originating ctypes object for provenance lookup without changing the public _objects value. Link borrowed sources weakly to their writers even when the write falls outside the source view, so independent buffer views created before or after the write can resolve the value. Preserve existing owners when assigning None. Keep from_buffer acquisitions alive through scalar _objects replacement, use one range-index implementation for independent aliases and actual pointer writes, with weak per-interpreter registrations removed when their owners are cleared or destroyed. Actual-write entries belong to the retention root, cover copied pointers after the original source view dies, and retain only the latest range per ownership slot. Prefer recorded writes and overlapping aliases on reads. On a miss, search current registered owners so earlier addresses remain readable after pointer retargeting, without accumulating address history. Follow native and Python-defined buffer exporters in both read and write provenance. The private byref retention is stronger than CPython's cast lifetime guarantee. from_buffer retains its original memoryview after scalar assignment; unlike CPython, scalar assignment alone no longer permits resizing a bytearray still borrowed by the view. Reads never reconstruct a Rust reference from an arbitrary native address. Performance Write benchmarks compare with 073983299; read and mixed benchmarks compare with 1d9469382. The cast-rooted and byref-rooted field rows compare with a3a072dc3; shared-buffer read rows compare with 842238ce6. The private-byref-field and out-of-view writer rows compare with c65a11a7f. Median of five runs: Windows 11 x64 Workload Before After 4,000 writes through a pointer field 259.1 ms 5.28 ms Warm writes through 1,000 separate pointer fields 211.1 ms 1.66 ms Read 4,000 backing-array elements after pointer-field writes 406.4 ms 2.75 ms 4,000 pointer-field writes followed by backing-array reads 439.6 ms 10.01 ms Warm writes through 800 cast-rooted pointer fields 318.71 ms 1.58 ms Warm writes through 800 byref-rooted pointer fields 237.68 ms 1.78 ms Read 800 independent views of one shared buffer 203.86 ms 1.27 ms Warm writes through 800 fields containing separate byref casts 224.02 ms 1.60 ms Read 800 slots written outside one byref source view 178.68 ms 0.58 ms Debian 12 amd64 Workload Before After 4,000 writes through a pointer field 264.1 ms 4.92 ms Warm writes through 1,000 separate pointer fields 194.1 ms 1.65 ms Read 4,000 backing-array elements after pointer-field writes 369.0 ms 2.62 ms 4,000 pointer-field writes followed by backing-array reads 378.3 ms 9.73 ms Warm writes through 800 cast-rooted pointer fields 298.40 ms 1.60 ms Warm writes through 800 byref-rooted pointer fields 226.40 ms 1.98 ms Read 800 independent views of one shared buffer 201.25 ms 1.21 ms Warm writes through 800 fields containing separate byref casts 218.89 ms 1.53 ms Read 800 slots written outside one byref source view 183.18 ms 0.56 ms Compared with a3a072dc3, registering a from_buffer view adds about 0.5–0.6 µs to construction on these systems. AI assistance This PR follows our AI policy. Written with Codex (GPT-6), reviewed by a human before submission. Summary by CodeRabbit Bug Fixes Improved ctypes.py_object assignments through arrays, pointers, and shared views so updates are reflected consistently. Improved object lifetime handling when pointer contents are replaced, retaining the replacement and releasing the previous value appropriately. Fixed assigning None so it does not prematurely release a buffer export. Improved casts from byref() arguments so the source object remains available while the cast pointer exists. Tests Added regression coverage for shared views, pointer updates, object replacement, buffer exports, and restoring an in-range index after pointer contents change.

Open Graph Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including...

X Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including...

Opengraph URL: https://github.com/RustPython/RustPython/pull/9391

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:d1bebf71-d858-22ca-09fb-57df0bc9cded
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idE4B8:F8C5C:1628F4:1BDD13:6B01E6D5
html-safe-nonce24109dc1ec33304b9cff54b01e7e777fd5dd1c77082a9a56fd3c1ca0d85cecbb
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNEI4OkY4QzVDOjE2MjhGNDoxQkREMTM6NkIwMUU2RDUiLCJ2aXNpdG9yX2lkIjoiNzY1ODA4ODk3MTYwNzU5ODgwNSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacddb78435e2fd59c1574ecf88c55c773d42ceb65e9ecfade326634366bab398c9
hovercard-subject-tagpull_request:5014794468
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9391/files
twitter:imagehttps://avatars.githubusercontent.com/u/9205230?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/9205230?s=400&v=4
og:image:altExtends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Nonecc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git
octolytics-dimension-user_id39710557
octolytics-dimension-user_loginRustPython
octolytics-dimension-repository_id135201145
octolytics-dimension-repository_nwoRustPython/RustPython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id135201145
octolytics-dimension-repository_network_root_nwoRustPython/RustPython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8c77c9e8439031ad54c75a250694842892f693e1
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/RustPython/RustPython/pull/9391/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F9391%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F9391%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=RustPython%2FRustPython
Reloadhttps://github.com/RustPython/RustPython/pull/9391/files
Reloadhttps://github.com/RustPython/RustPython/pull/9391/files
Reloadhttps://github.com/RustPython/RustPython/pull/9391/files
RustPythonhttps://github.com/RustPython
RustPythonhttps://github.com/RustPython/RustPython
Codehttps://github.com/RustPython/RustPython
Issues195 (195)https://github.com/RustPython/RustPython/issues
Pull requests133 (133)https://github.com/RustPython/RustPython/pulls
Discussionshttps://github.com/RustPython/RustPython/discussions
Actionshttps://github.com/RustPython/RustPython/actions
Projectshttps://github.com/RustPython/RustPython/projects
Wikihttps://github.com/RustPython/RustPython/wiki
Security and qualityhttps://github.com/RustPython/RustPython/security
Insightshttps://github.com/RustPython/RustPython/insights
Sign up for GitHub https://github.com/signup?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
youknowonehttps://github.com/youknowone
RustPython:mainhttps://github.com/RustPython/RustPython/tree/main
1ndahous3:ctypes_pyobject_ownershttps://github.com/1ndahous3/RustPython/tree/ctypes_pyobject_owners
Conversation 57 https://github.com/RustPython/RustPython/pull/9391
Commits 11 https://github.com/RustPython/RustPython/pull/9391/commits
Checks 27 https://github.com/RustPython/RustPython/pull/9391/checks
Files changed https://github.com/RustPython/RustPython/pull/9391/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9391/files
Preserve ctypes py_object provenance across casts and pointer writes https://github.com/RustPython/RustPython/pull/9391/files#top
Show all changes 11 commits https://github.com/RustPython/RustPython/pull/9391/files
154d1c5 Preserve ctypes py_object provenance across casts and pointer writes 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/154d1c590a73b949e5e2a8b91384defa2955c4ff
2078378 Validate ctypes writer paths across casts and retargeting 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/2078378cfb1e480f569310871f360f545f17634d
0739832 Preserve ctypes writer roots and avoid nested storage scans 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/073983299287c5898e288d6a930887d76fe3b7b6
1d94693 Register ctypes exporters and cache pointer-field writers 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/1d9469382e507aa9897042dc9eebff5c94160968
a3a072d Register overlapping ctypes views and index retained objects 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/a3a072dc398af8a7b7124a65aa94a2716f94b789
842238c Track shared ctypes buffer owners and cache cast fields 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/842238ce60c0f6b6f4dc18778363adc469ff27ee
1e11d82 Index ctypes buffer aliases and register pointer exporters 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/1e11d8279a376033f325205259bdee562b511f15
c65a11a Publish disjoint ctypes byref writes to borrowed sources 1ndahous3 Oct 26, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/c65a11a7f256e5a0799de80455aa9054a7ed677c
9ad3d90 Index ctypes write provenance and validate private field paths 1ndahous3 Oct 27, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/9ad3d9085b17044841e9429bf9606ce18125d437
1b6337a Resolve retained ctypes values after pointer retargeting 1ndahous3 Oct 27, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/1b6337ae24c7a5d5ca6f887294d1a395342282be
37c84fb Merge main into ctypes_pyobject_owners 1ndahous3 Oct 27, 2026 https://github.com/RustPython/RustPython/pull/9391/commits/37c84fbb5541684ab1e32bbd8078ae3cd165a036
Clear filters https://github.com/RustPython/RustPython/pull/9391/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9391/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9391/files
memory.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-83fe7bb7e7b6dd3b601eae76f14c31d467ad70f40cb4183de0e82f5af619c721
_ctypes.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-d2bf8c03d89a90728436dbecb8b7f929c67d08ff5d6e18b2a0fc279e8fe77a8d
array.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
base.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-16c64cd355bd8034d61677ad8e407239d548e0e35c48ee05e951e03e93aa1781
function.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-d4cf72f95963a6d3e08f9e7e53594d6b5e2cc7711afbc0b490894d97dc8d8648
pointer.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-31a7efabe21a7be0f4dcee8c644af456ac58989d1c551faf515841b41e652a66
simple.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-640c11bed665f401309f30eaa8f59ef7ca4c930c57bc97dbf036e19030e1e64a
mod.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-5ef3107a41920f6fa3ac6749b8701b4a278dba5ad37e12488de9cfd22688dc49
interpreter.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-2361f366cc46771907dbd3b09eb8b9c839e7f059710345f2265c1753862a1b20
mod.rs https://github.com/RustPython/RustPython/pull/9391/files#diff-fe30ecf7c6b462e46e0874a55a05ea0edba83cefad5ac7a86e7137fa8f86ba74
stdlib_ctypes.py https://github.com/RustPython/RustPython/pull/9391/files#diff-7cfa080a4c0c0eac19bf2bd0eea152938488bab4c8a8d1b1d7bbc14bce41a61c
crates/vm/src/builtins/memory.rshttps://github.com/RustPython/RustPython/pull/9391/files#diff-83fe7bb7e7b6dd3b601eae76f14c31d467ad70f40cb4183de0e82f5af619c721
View file https://github.com/1ndahous3/RustPython/blob/37c84fbb5541684ab1e32bbd8078ae3cd165a036/crates/vm/src/builtins/memory.rs
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/9391/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/9391/files#diff-83fe7bb7e7b6dd3b601eae76f14c31d467ad70f40cb4183de0e82f5af619c721
https://github.com/RustPython/RustPython/pull/9391/files#diff-83fe7bb7e7b6dd3b601eae76f14c31d467ad70f40cb4183de0e82f5af619c721
crates/vm/src/stdlib/_ctypes.rshttps://github.com/RustPython/RustPython/pull/9391/files#diff-d2bf8c03d89a90728436dbecb8b7f929c67d08ff5d6e18b2a0fc279e8fe77a8d
View file https://github.com/1ndahous3/RustPython/blob/37c84fbb5541684ab1e32bbd8078ae3cd165a036/crates/vm/src/stdlib/_ctypes.rs
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/9391/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/9391/files#diff-d2bf8c03d89a90728436dbecb8b7f929c67d08ff5d6e18b2a0fc279e8fe77a8d
https://github.com/RustPython/RustPython/pull/9391/files#diff-d2bf8c03d89a90728436dbecb8b7f929c67d08ff5d6e18b2a0fc279e8fe77a8d
crates/vm/src/stdlib/_ctypes/array.rshttps://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
View file https://github.com/1ndahous3/RustPython/blob/37c84fbb5541684ab1e32bbd8078ae3cd165a036/crates/vm/src/stdlib/_ctypes/array.rs
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/9391/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
https://github.com/RustPython/RustPython/pull/9391/files#diff-1e6ef8a4a7706885a4b2f08152c3a0298bf0db75e092d52708b08caa6726d321
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9391/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/9391/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.