Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython · GitHub
Open Graph Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython
X Title: Preserve ctypes py_object provenance across casts and pointer writes by 1ndahous3 · Pull Request #9391 · RustPython/RustPython
Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including independent from_buffer acquisitions and aggregate writes that overlap a smaller view. Use weak links to the roots retaining written values, including writes through temporary pointer fields, without extending their lifetime. Amortize cleanup across new registrations and keep the destination owner alive through conversion, writing and ownership publication. Publish actual write ranges after user conversion so nested writes cannot replace the outer operation's range. Reuse validated weak ownership paths for direct pointers, casts and aggregate pointer fields. Cache each field on its retention root, including pointer roots, matching byref casts and private byref arguments copied into fields. Validate the exact private slot on reuse so a surviving sibling owner cannot validate a replaced field's path. Avoid repeated ownership-graph searches for writes into nested storage. Follow buffer-exporter links for borrowed views, including views of pointer contents and writes through pointers targeting those views. Keep both current contents and retained cast storage able to resolve later writes, including differently sized views of the same buffer and retargeting through intermediate pointers or aggregate fields. Lazily index direct retention entries for reads through registered writers, avoiding repeated scans of all retained values after pointer-field writes. The index stores O(N) Rust metadata and validates cached keys against current values without retaining extra Python references. Retain byref arguments privately when casting them, retaining the originating ctypes object for provenance lookup without changing the public _objects value. Link borrowed sources weakly to their writers even when the write falls outside the source view, so independent buffer views created before or after the write can resolve the value. Preserve existing owners when assigning None. Keep from_buffer acquisitions alive through scalar _objects replacement, use one range-index implementation for independent aliases and actual pointer writes, with weak per-interpreter registrations removed when their owners are cleared or destroyed. Actual-write entries belong to the retention root, cover copied pointers after the original source view dies, and retain only the latest range per ownership slot. Prefer recorded writes and overlapping aliases on reads. On a miss, search current registered owners so earlier addresses remain readable after pointer retargeting, without accumulating address history. Follow native and Python-defined buffer exporters in both read and write provenance. The private byref retention is stronger than CPython's cast lifetime guarantee. from_buffer retains its original memoryview after scalar assignment; unlike CPython, scalar assignment alone no longer permits resizing a bytearray still borrowed by the view. Reads never reconstruct a Rust reference from an arbitrary native address. Performance Write benchmarks compare with 073983299; read and mixed benchmarks compare with 1d9469382. The cast-rooted and byref-rooted field rows compare with a3a072dc3; shared-buffer read rows compare with 842238ce6. The private-byref-field and out-of-view writer rows compare with c65a11a7f. Median of five runs: Windows 11 x64 Workload Before After 4,000 writes through a pointer field 259.1 ms 5.28 ms Warm writes through 1,000 separate pointer fields 211.1 ms 1.66 ms Read 4,000 backing-array elements after pointer-field writes 406.4 ms 2.75 ms 4,000 pointer-field writes followed by backing-array reads 439.6 ms 10.01 ms Warm writes through 800 cast-rooted pointer fields 318.71 ms 1.58 ms Warm writes through 800 byref-rooted pointer fields 237.68 ms 1.78 ms Read 800 independent views of one shared buffer 203.86 ms 1.27 ms Warm writes through 800 fields containing separate byref casts 224.02 ms 1.60 ms Read 800 slots written outside one byref source view 178.68 ms 0.58 ms Debian 12 amd64 Workload Before After 4,000 writes through a pointer field 264.1 ms 4.92 ms Warm writes through 1,000 separate pointer fields 194.1 ms 1.65 ms Read 4,000 backing-array elements after pointer-field writes 369.0 ms 2.62 ms 4,000 pointer-field writes followed by backing-array reads 378.3 ms 9.73 ms Warm writes through 800 cast-rooted pointer fields 298.40 ms 1.60 ms Warm writes through 800 byref-rooted pointer fields 226.40 ms 1.98 ms Read 800 independent views of one shared buffer 201.25 ms 1.21 ms Warm writes through 800 fields containing separate byref casts 218.89 ms 1.53 ms Read 800 slots written outside one byref source view 183.18 ms 0.56 ms Compared with a3a072dc3, registering a from_buffer view adds about 0.5–0.6 µs to construction on these systems. AI assistance This PR follows our AI policy. Written with Codex (GPT-6), reviewed by a human before submission. Summary by CodeRabbit Bug Fixes Improved ctypes.py_object assignments through arrays, pointers, and shared views so updates are reflected consistently. Improved object lifetime handling when pointer contents are replaced, retaining the replacement and releasing the previous value appropriately. Fixed assigning None so it does not prematurely release a buffer export. Improved casts from byref() arguments so the source object remains available while the cast pointer exists. Tests Added regression coverage for shared views, pointer updates, object replacement, buffer exports, and restoring an in-range index after pointer contents change.
Open Graph Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including...
X Description: Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including...
Opengraph URL: https://github.com/RustPython/RustPython/pull/9391
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:d1bebf71-d858-22ca-09fb-57df0bc9cded |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | E4B8:F8C5C:1628F4:1BDD13:6B01E6D5 |
| html-safe-nonce | 24109dc1ec33304b9cff54b01e7e777fd5dd1c77082a9a56fd3c1ca0d85cecbb |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNEI4OkY4QzVDOjE2MjhGNDoxQkREMTM6NkIwMUU2RDUiLCJ2aXNpdG9yX2lkIjoiNzY1ODA4ODk3MTYwNzU5ODgwNSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | ddb78435e2fd59c1574ecf88c55c773d42ceb65e9ecfade326634366bab398c9 |
| hovercard-subject-tag | pull_request:5014794468 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9391/files |
| twitter:image | https://avatars.githubusercontent.com/u/9205230?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/9205230?s=400&v=4 |
| og:image:alt | Extends the ctypes ownership handling from #9251 to pointer writes and byref casts. Summary Resolve py_object values written through pointers from the original storage and sibling views, including... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width