Title: gh-148286: Track JIT-allocated memory with Valgrind client requests by matthiasgoergens · Pull Request #154926 · python/cpython · GitHub
Open Graph Title: gh-148286: Track JIT-allocated memory with Valgrind client requests by matthiasgoergens · Pull Request #154926 · python/cpython
X Title: gh-148286: Track JIT-allocated memory with Valgrind client requests by matthiasgoergens · Pull Request #154926 · python/cpython
Description: Python/jit.c's jit_alloc() and jit_free() mmap and munmap the JIT's machine-code buffers directly. Valgrind's leak checker does not track raw mmap'd memory as a "block", so a forgotten jit_free() — the shape of gh-142476, a real ASan-found leak in allocate_executor — is completely invisible to it. This adds VALGRIND_MALLOCLIKE_BLOCK / VALGRIND_FREELIKE_BLOCK around those two functions, guarded by the existing --with-valgrind configure option, mirroring the pattern vendored mimalloc already uses in Include/internal/mimalloc/mimalloc/track.h. To measure the effect I injected a forgotten free into jit_free() on a machine-code JIT build (--enable-experimental-jit, LLVM 21) and ran a script that creates eight executors and drops them: build valgrind --leak-check=full without --with-valgrind definitely lost: 0 bytes in 0 blocks with --with-valgrind definitely lost: 139,264 bytes in 10 blocks and in the second case with a usable backtrace: 131,072 bytes in 8 blocks are definitely lost at jit_alloc (Python/jit.c:154) by _PyJIT_Compile by make_executor_from_uops (Python/optimizer.c:1574) by uop_optimize by _PyOptimizer_Optimize by stop_tracing_and_jit (Python/ceval.c:1090) Background noise was byte-identical between the two runs (120 bytes possibly lost, 646 bytes still reachable), so the injected leak is the only difference. The injection is not part of this patch. The include is valgrind/valgrind.h — the header configure already probes for under --with-valgrind, and the one that defines both macros used here, so no build-system change is needed. Objects/obmalloc.c includes the same header. On VALGRIND_DISCARD_TRANSLATIONS I deliberately did not add any. It seemed like the obvious thing for a JIT, but it turns out to be unnecessary here. Valgrind's munmap and mprotect syscall wrappers already discard cached translations for any range that previously held translated code, and that logic lives in architecture-independent code (coregrind/m_syswrap/syswrap-generic.c). I also checked it experimentally with a small driver that emits code, executes it enough times to guarantee a cached translation, changes it, and executes again — with a control case that genuinely does go stale so the harness is known to detect the failure. CPython's mmap → write once → mprotect → execute → munmap-whole-region pattern never executed a stale translation on either x86-64 (Valgrind 3.25.1) or aarch64 (Valgrind 3.19.0), in any --smc-check mode including none. A side effect worth recording: since that is already true, --smc-check=none is safe for CPython today with no code change. On a tight JIT-compiled integer loop that is worth about 10% over the all-non-file default (median of three runs each: 11.33 s vs 10.17 s, ranges 11.22-11.45 and 10.12-10.36, so non-overlapping). That workload is close to a best case — it is almost entirely JIT-executed — so a realistic program should expect less. Testing Builds cleanly both with and without --with-valgrind; test_capi and test_optimizer pass (1,501 tests) on the --with-valgrind build, and under valgrind memcheck. Issue: gh-148286
Open Graph Description: Python/jit.c's jit_alloc() and jit_free() mmap and munmap the JIT's machine-code buffers directly. Valgrind's leak checker does not track raw mmap'd memory as a "block", s...
X Description: Python/jit.c's jit_alloc() and jit_free() mmap and munmap the JIT's machine-code buffers directly. Valgrind's leak checker does not track raw mmap'd memory as a &...
Opengraph URL: https://github.com/python/cpython/pull/154926
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:fa5b75d3-d285-9350-95d0-2762ea1d806c |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 9FE8:1F65:867A1:AF938:6B024F3E |
| html-safe-nonce | 1ad2d19ddab912c80f1323450c4b62001d6444268041c13c0ce8b47d7b6285fb |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5RkU4OjFGNjU6ODY3QTE6QUY5Mzg6NkIwMjRGM0UiLCJ2aXNpdG9yX2lkIjoiMTAxMTI0MDQzMDkxMTc3MDQzMCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 2d0502d416f2d40beaf6369c48d04437772ddda955966c80183d387f7c43e9e3 |
| hovercard-subject-tag | pull_request:4169733508 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/154926/files |
| twitter:image | https://avatars.githubusercontent.com/u/57026?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/57026?s=400&v=4 |
| og:image:alt | Python/jit.c's jit_alloc() and jit_free() mmap and munmap the JIT's machine-code buffers directly. Valgrind's leak checker does not track raw mmap'd memory as a "block", s... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | canary-1 |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width