Title: Abort/Segfault: corrupted object freelist in clear_freelist · Issue #152125 · python/cpython · GitHub
Open Graph Title: Abort/Segfault: corrupted object freelist in clear_freelist · Issue #152125 · python/cpython
X Title: Abort/Segfault: corrupted object freelist in clear_freelist · Issue #152125 · python/cpython
Description: Bug report Bug description: Under OOM the free-threaded build's per-PyThreadState freelists desync size from the chain: PyList_New's Py_DECREF(op) failure path re-pushes a list with a stale dangling ob_item, tripping the clear_freelist s...
Open Graph Description: Bug report Bug description: Under OOM the free-threaded build's per-PyThreadState freelists desync size from the chain: PyList_New's Py_DECREF(op) failure path re-pushes a list with a stale danglin...
X Description: Bug report Bug description: Under OOM the free-threaded build's per-PyThreadState freelists desync size from the chain: PyList_New's Py_DECREF(op) failure path re-pushes a list with a stale...
Opengraph URL: https://github.com/python/cpython/issues/152125
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Abort/Segfault: corrupted object freelist in clear_freelist","articleBody":"# Bug report\n\n### Bug description:\n\n*Under OOM the free-threaded build's per-`PyThreadState` freelists desync `size` from the chain: `PyList_New`'s `Py_DECREF(op)` failure path re-pushes a list with a stale dangling `ob_item`, tripping the `clear_freelist` size assertion (or a `free_list_items` use-after-free SEGV on the next pop).*\n\n_AI Disclaimer: this gist was drafted by Claude Code, which also generated the reduced reproducer._\n\n## Crash report\n\nUnder sustained allocation failure the free-threaded build's per-`PyThreadState` object freelists become internally inconsistent: the `size` counter and the actual `freelist` chain disagree. This corruption surfaces two ways from the same root: (1) the GC's freelist sweep aborts on `assert(freelist->size == 0 || freelist->size == -1)` in `clear_freelist` (`Objects/object.c:909`) — the recorded crash for all three vehicles — and (2) a subsequent `PyList_New` pops a bogus \"list\" off the corrupted `lists` freelist whose stale `ob_item` points at already-freed (`0xcd...`-filled) array memory; when the immediately-following `list_allocate_array()` fails under OOM, the cleanup `Py_DECREF(op)` calls `free_list_items()` on that dangling pointer -> use-after-free SEGV in `_PyMem_DebugFree`. The assertion is debug-only (`NDEBUG` drops it), but the underlying freelist corruption is build-agnostic and crashes release, JIT and upstream builds too.\n\n## Reproducer\n\n```python\nimport gc\nfrom _testcapi import set_nomemory, remove_mem_hooks\n\ndef go():\n objs = []\n for _ in range(120):\n a = (1.0, 2.0, 3.0) # tuple + float freelists\n b = [a[0], a[1], a[2]] # list freelist (PyList_New)\n objs.append(b[0:2]) # slice -> another PyList_New\n gc.collect() # GC sweeps freelists -> clear_freelist\n return objs\n\nfor start in range(1, 900): # crashes at start=4\n set_nomemory(start, 0) # fail every allocation from #start onward\n try:\n try:\n go()\n finally:\n remove_mem_hooks()\n except BaseException:\n pass\n```\n\nDeterministic at `start=4` on the free-threaded debug+ASan build, and also crashes (SEGV) on free-threaded release, JIT, and upstream. A single `set_nomemory(0,0)` does not suffice — the budget has to let the early freelist pushes/pops succeed and then fail mid-sequence, hence the sweep.\n\nThe original fuzzing vehicles instead drained/realloc'd enough lists that the corruption surfaced one step earlier, inside the GC's `clear_freelist`, producing the `Objects/object.c:909` assertion abort (see the recorded faulthandler stack in `backtrace.txt`). Same defect, different surface.\n\n## Backtrace\n\nDeterministic SEGV surface from the reduced reproducer (gdb, `3.16_ft_debug_asan`, commit 15d7406) — byte-identical to the captured vehicle backtrace:\n\n```\n#0 _PyMem_DebugCheckAddress Objects/obmalloc.c:3344 (p = 0xcdcdcdcdcdcdcdc5 \u003c- freed-fill)\n#1 _PyMem_DebugRawFree Objects/obmalloc.c:3166\n#2 _PyMem_DebugFree Objects/obmalloc.c:3316\n#3 free_list_items Objects/listobject.c:68 (PyMem_Free(dangling ob_item))\n#4 list_dealloc Objects/listobject.c:569\n#5 _Py_Dealloc Objects/object.c:3319\n#6 Py_DECREF Include/refcount.h:359\n#7 PyList_New Objects/listobject.c:262 (Py_DECREF(op) after list_allocate_array fail)\n#8 _PyList_FromStackRefStealOnSuccess Objects/listobject.c:3298\n```\n\nRecorded abort surface (faulthandler, original `tarfile` vehicle):\n\n```\npython: Objects/object.c:909: void clear_freelist(struct _Py_freelist *, int, freefunc):\n Assertion `freelist->size == 0 || freelist->size == -1' failed.\nFatal Python error: Aborted\n ... _Py_HandlePending -> GC -> _PyObject_ClearFreeLists -> clear_freelist\n```\n\n## Root cause\n\n`Objects/object.c`, `clear_freelist` (L901):\n\n```c\nstatic void\nclear_freelist(struct _Py_freelist *freelist, int is_finalization, freefunc dofree)\n{\n void *ptr;\n while ((ptr = _PyFreeList_PopNoStats(freelist)) != NULL) { // drains chain\n dofree(ptr);\n }\n assert(freelist->size == 0 || freelist->size == -1); // L909: size desynced -> abort\n assert(freelist->freelist == NULL);\n ...\n}\n```\n\n`_PyFreeList_Push`/`_PyFreeList_PopNoStats` (`Include/internal/pycore_freelist.h`) keep `fl->size` in lockstep with the singly-linked `fl->freelist` chain (the \"next\" pointer is stored in the first word of each freed object). The `clear_freelist` loop pops until the chain is NULL and then asserts `size == 0`. The abort means `size` no longer matches the chain length: the chain terminated (or was diverted) before `size` reached zero.\n\nTwo ingredients, both triggered by the OOM injection in the free-threaded build:\n\n1. **Freelist objects are not \"next\"-cleared on the failure path.** In `PyList_New` (`Objects/listobject.c:248`) the list object is popped from the freelist *before* the size-dependent `list_allocate_array()` runs. When that allocation fails under OOM, the cleanup is `Py_DECREF(op)` (L262), so `op` goes straight back through `list_dealloc` -> `_Py_FREELIST_FREE(lists, op, ...)` (L573) and back onto the same freelist. The repeated pop/push-on-failure churn under a failing allocator is where `size` and the chain drift apart, leaving a node whose stale `ob_item` (L266 was never reached) still points at a freed array.\n\n2. **A later consumer trusts the desynced freelist.** The very next `PyList_New` pops that bad node; with `op->ob_item` dangling and `list_allocate_array` again failing, `Py_DECREF(op)` reaches `free_list_items(op->ob_item, ...)` on freed (`0xcd`) memory -> use-after-free. If a GC sweep runs first instead, `clear_freelist` trips its `size`/chain assertion.\n\n(The assertion is compiled out under `NDEBUG`, which is why release/JIT/upstream don't abort there — but they still crash on the same corruption, as a SEGV.)\n\n## Suggested fix\n\nThe robust fix is to make freelist accounting self-healing and to stop trusting `size` independently of the chain. Concretely:\n\n- In `clear_freelist`, drive the loop and the counter off the *chain* and reset `size` unconditionally instead of asserting it, so a desync can't corrupt later allocations:\n\n```c\n void *ptr;\n while ((ptr = _PyFreeList_PopNoStats(freelist)) != NULL) {\n dofree(ptr);\n }\n assert(freelist->freelist == NULL);\n freelist->size = is_finalization ? -1 : 0; /* reset, don't assert size */\n```\n\n- More importantly, fix the producer: in `PyList_New` (`listobject.c:248`), reset `op->ob_item = NULL` (and `Py_SET_SIZE(op, 0); op->allocated = 0;`) on the freelist-pop path *before* the fallible `list_allocate_array()`, so the OOM `Py_DECREF(op)` cleanup never sees a stale `ob_item`:\n\n```c\n PyListObject *op = _Py_FREELIST_POP(PyListObject, lists);\n if (op == NULL) {\n op = PyObject_GC_New(PyListObject, &PyList_Type);\n if (op == NULL) {\n return NULL;\n }\n }\n op->ob_item = NULL; /* defensive: pop'd object may carry stale state */\n Py_SET_SIZE(op, 0);\n op->allocated = 0;\n```\n\nThe deeper invariant to audit is every `_Py_FREELIST_POP` / `_Py_FREELIST_FREE` pair on a fallible allocation path: an object must be in a clean, decref-safe state before any allocation that can fail under OOM, since the failure cleanup pushes it back onto the freelist.\n\n## Notes\n\nFound by OOM-injection fuzzing (`_testcapi.set_nomemory`). Free-threading-specific in its *abort* surface (the `clear_freelist` assert and the per-`PyThreadState` freelist machinery are `Py_GIL_DISABLED`-only), but the reduced reproducer crashes all four local builds (debug+ASan abort/SEGV, release SEGV, JIT SEGV, upstream SEGV), so the underlying freelist corruption is not debug-only. Three vehicles in `python-4/` recorded the identical `Objects/object.c:909` assertion; re-running any of them now non-deterministically lands on the sibling `PyList_New`/`free_list_items` use-after-free SEGV instead — confirming a single shared root cause. Distinct from the `_co_unique_id` (`codeobject.c:2440`) assertion that the same vehicles can also hit; that is a separate FT unique-id-pool invariant, not this freelist one.\n\n## Versions\n\n- main (3.16.0a0, commit 15d7406). Reproduced (reduced repro `repro.py`) on free-threaded debug+ASan (SEGV; abort on the vehicle's `clear_freelist` surface), free-threaded release (SEGV), JIT (SEGV), and upstream (SEGV).\n\n---\n\n*Part of [python/cpython#151763](https://github.com/python/cpython/issues/151763) — an umbrella tracking 35 OOM-related crash findings.*\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\n_No response_","author":{"url":"https://github.com/sergey-miryanov","@type":"Person","name":"sergey-miryanov"},"datePublished":"2026-06-24T21:27:50Z","url":"https://github.com/python/cpython/issues/152125"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:df164df1-8dcc-f9b6-cd55-234da4a61d3e |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D242:148EE1:6255DE:7BFE6D:6B020950 |
| html-safe-nonce | 587a26fb53c9c6421f3dd188f7b71cb5b1d9785484befe1a851290726e348c80 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMjQyOjE0OEVFMTo2MjU1REU6N0JGRTZEOjZCMDIwOTUwIiwidmlzaXRvcl9pZCI6IjE2NjI0OTQxNjg5MDk0ODIzMjAiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 88a8f4483f2a289ce0d022f9ab61beac344285479a33fea0cb73f7e3280a41a7 |
| hovercard-subject-tag | issue:4738605887 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/152125/issue_layout |
| twitter:image | https://opengraph.githubassets.com/9d06e8bdc85d8a6d05d2b946a7c7b4a748e887091d87e736aa7086d8122937f3/python/cpython/issues/152125 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/9d06e8bdc85d8a6d05d2b946a7c7b4a748e887091d87e736aa7086d8122937f3/python/cpython/issues/152125 |
| og:image:alt | Bug report Bug description: Under OOM the free-threaded build's per-PyThreadState freelists desync size from the chain: PyList_New's Py_DECREF(op) failure path re-pushes a list with a stale danglin... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | sergey-miryanov |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width