Title: gh-159137: `gc`: Count GC survivors in `untrack_tuples()` instead of walking the young generation again by maurycy · Pull Request #159149 · python/cpython · GitHub
Open Graph Title: gh-159137: `gc`: Count GC survivors in `untrack_tuples()` instead of walking the young generation again by maurycy · Pull Request #159149 · python/cpython
X Title: gh-159137: `gc`: Count GC survivors in `untrack_tuples()` instead of walking the young generation again by maurycy · Pull Request #159149 · python/cpython
Description: Inspired by #159137. After deduce_unreachable(), gc_collect_main() walks the surviving young generation twice. untrack_tuples() visits every object: cpython/Python/gc.c Lines 664 to 676 in 9028df3 static void untrack_tuples(PyGC_Head *head) { PyGC_Head *gc = GC_NEXT(head); while (gc != head) { PyObject *op = FROM_GC(gc); PyGC_Head *next = GC_NEXT(gc); if (PyTuple_CheckExact(op)) { _PyTuple_MaybeUntrack(op); } gc = next; } } and then gc_list_size(young) walks the same list again only to update long_lived_pending or long_lived_total: cpython/Python/gc.c Lines 1517 to 1535 in 9028df3 untrack_tuples(young); /* Move reachable objects to next generation. */ if (young != old) { if (generation == NUM_GENERATIONS - 2) { gcstate->long_lived_pending += gc_list_size(young); } gc_list_merge(young, old); } else { // In Python <= 3.13, we called untrack_dicts(young) here to untrack // atomic-only dicts (see issue #14775). Python 3.14 removed the lazy // dict tracking machinery entirely (GH-127010) -- dicts are always // tracked from creation and never untracked by GC. That way, we don't // have to restore MAINTAIN_TRACKING across every PyDict_SetItem call // site; the cost is slightly more work for full collections on dicts // with only atomic values. gcstate->long_lived_pending = 0; gcstate->long_lived_total = gc_list_size(young); } So the idea is to untrack_tuples() to return the number of objects that stay tracked, and use it instead of gc_list_size(young). Microbenchmark Full collections on a heap that stays alive for the whole run. macOS 27 (M4 Air), release build w/o PGO. Median of 6 processes per binary, interleaved A/B/B/A: heap main (9028df3) 9028df3 + patch 2M objects, dict + list + tuple per entry 77.0 ms 67.4 ms -12% 2M dicts with atomic values 66.9 ms 59.0 ms -12% 5M objects, dict + list + tuple per entry 201.7 ms 169.6 ms -16% 2M tuples of ints 11.6 ms 11.6 ms 0% Unfortunately, I don't have access to Linux baremetal at the moment. Macrobenchmark Sphinx 8.2.3 building the CPython HTML docs with one worker and fresh doctrees (sphinx-build -b html -j 1 -E), same machine and builds, 6 runs per binary interleaved A/B/B/A. GC time is the sum of duration from gc.get_stats() main (9028df3) 9028df3 + patch CPU time 46.00 s 44.37 s -3.5% GC time 10.07 s 8.63 s -14% GC time, full collections only 8.66 s 7.26 s -16% CPU time outside GC 35.93 s 35.74 s -0.5% Microbenchmark import gc, sys, time N = int(sys.argv[1]); kind = sys.argv[2] gc.disable() if kind == "mixed": keep = [({"a": i, "b": None}, [i, i], (i, i + 1)) for i in range(N // 3)] elif kind == "tuples": keep = [(i, i + 1, i + 2) for i in range(N)] elif kind == "dicts": keep = [{"a": i, "b": "x"} for i in range(N)] for _ in range(2): gc.collect() ts = [] for _ in range(7): t0 = time.perf_counter_ns(); gc.collect(); ts.append(time.perf_counter_ns() - t0) ts.sort(); print(ts[3] / 1e6) Issue: gh-159137
Open Graph Description: Inspired by #159137. After deduce_unreachable(), gc_collect_main() walks the surviving young generation twice. untrack_tuples() visits every object: cpython/Python/gc.c ...
X Description: Inspired by #159137. After deduce_unreachable(), gc_collect_main() walks the surviving young generation twice. untrack_tuples() visits every object: cpython/Python/gc.c ...
Opengraph URL: https://github.com/python/cpython/pull/159149
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:192f9846-ed52-f6e4-8ff1-8378dde3072e |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | C3E6:33444D:33E1101:44FBB1E:6B01FB88 |
| html-safe-nonce | 86cb89b53ccf664a785c3beffb8d1285e15a988dc8649b120a01b08f1ac697e3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDM0U2OjMzNDQ0RDozM0UxMTAxOjQ0RkJCMUU6NkIwMUZCODgiLCJ2aXNpdG9yX2lkIjoiNzc2MTU3MDk2NDQ0OTY1NTY4OCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 1c7e1f8920546b22c2a66ae1c688a4051c7053e10207e3b75a89e52740142f43 |
| hovercard-subject-tag | pull_request:4814498090 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/159149/files |
| twitter:image | https://avatars.githubusercontent.com/u/5383?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/5383?s=400&v=4 |
| og:image:alt | Inspired by #159137. After deduce_unreachable(), gc_collect_main() walks the surviving young generation twice. untrack_tuples() visits every object: cpython/Python/gc.c ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width