René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Backport free-threading memory-safety fixes to 3.14 and 3.15","articleBody":"Note: this work was assisted by AI agents (Claude Code and Codex).\n\nThis is a tracking issue for a batch of free-threading (`Py_GIL_DISABLED`) fixes that are in `main` but missing from the 3.14 and/or 3.15 maintenance branches. Most of them are use-after-free bugs, pointer tearing, or uninitialized reads: they can crash or silently corrupt data in a free-threaded build, and 3.14 is the first release where the free-threaded build is officially supported.\n\nEvery fix listed below has been backported, built, and tested on a branch already (see [Branches](#branches)). This issue is to agree on the set and to track the resulting PRs; the individual PRs are where the code review belongs.\n\n## How the candidates were found\n\nThe audit was done entirely from local git refs, with no GitHub queries, so it is reproducible offline against the exact commits listed under [Commit ranges](#commit-ranges).\n\n1. **Screen two disjoint commit ranges** covering everything in `main` that 3.14 does not have (5,389 commits total, see below).\n2. **Filter** commit subjects, bodies, NEWS entries, and diffs for:\n   - `free-threading`, `free-threaded`, `Py_GIL_DISABLED`, `nogil`, standalone `FT`\n   - `safe`, `thread-safe`, `thread safety`\n   - `race`, `racing`, `concurrent`\n   - `UAF`, `use-after-free`, `segfault`, `crash`, `deadlock`\n   - any commit touching `Lib/test/test_free_threading/`\n   - any diff adding critical sections, atomics, mutexes, or free-threading guards\n3. **Rank by severity.** The bar for \"recommended\" is a memory-safety consequence: crash, UAF, refcount/memory corruption, or torn pointer reads. Fixes that are merely correctness or performance improvements were screened out unless they were also small and low-risk.\n4. **Check what is already backported**, using source commit hashes quoted in maintenance-branch commit messages, PR/issue IDs in subjects and bodies, `git cherry` patch-equivalence (`git cherry origin/3.15 origin/main`, likewise for 3.14), and manual inspection where the code had diverged.\n5. **Check applicability** in a throwaway worktree per candidate:\n\n   ```sh\n   git diff COMMIT^ COMMIT | git -C WORKTREE apply --check -\n   git diff --binary COMMIT^ COMMIT | git -C WORKTREE apply --3way --check -\n   ```\n\n   Note that `--3way --check` passing says nothing about semantic correctness, which is why every\n   backport was then built and run against the full test suite.\n\nA first pass only searched commits after the 3.15 fork point, which missed fixes that 3.15 inherited before the fork but that 3.14 never got (e.g. #146033, `itertools.zip_longest`). The range was widened to catch those. Anyone repeating this exercise should be careful about the same trap: the 3.14-only candidates and the \"missing from both\" candidates live in different commit ranges.\n\n## Commit ranges\n\nRefs as of the audit:\n\n| Ref | Commit |\n|---|---|\n| `origin/main` | `6d5908368dc7936802d62413344dd4596abc4159` |\n| `origin/3.15` | `dc448f3d1a2bfc6ea18e3b5a87f5bd9eb1023964` |\n| `origin/3.14` | `8e648a9c6b29c65ac6e290d6453cd69cedb3b1b4` |\n| 3.15 fork point | `f5c75351def83602b5b23c1fba361b7de8ffabc7` |\n| 3.14/`main` merge base | `b092705907c758d4f9742028652c9802f9f03dd3` (`Python 3.14.0b1`) |\n\nThe two ranges screened:\n\n```sh\n# 969 commits: post-3.15-fork work, i.e. missing from BOTH 3.15 and 3.14\ngit log f5c75351def83602b5b23c1fba361b7de8ffabc7..origin/main\n\n# 4,420 commits: between 3.14.0b1 and the 3.15 fork, i.e. inherited by 3.15, missing from 3.14\ngit log b092705907c758d4f9742028652c9802f9f03dd3..f5c75351def83602b5b23c1fba361b7de8ffabc7\n```\n\nTo pick up new candidates later, re-run the same filters over `\u003cprevious origin/main\u003e..origin/main` and re-check the 3.14 range for anything newly identified as severe.\n\nThe backport branches themselves are cut from the current branch tips, not the audit snapshot: 3.14 base `c628cd70506`, 3.15 base `9efded46646`. Each branch contains exactly one commit.\n\n## Candidates\n\n### Missing from both 3.15 and 3.14\n\n| Issue | PR | `main` commit | Summary | Severity |\n|---|---|---|---|---|\n| gh-149816 | #150024 | `d095ceb0f420` | UAF in `pickle.dumps()` when a list is mutated concurrently | Medium |\n| gh-144774 | #150578 | `f586fd9e304e` | `BaseException.__setstate__()` locked the exception, not the state dict, leaving borrowed dict refs exposed to concurrent mutation | Low |\n| gh-150858 | #150859 | `1ec6596828b0` | UAF reading a type's `__qualname__` while another thread replaces it | Low |\n| gh-150411 | #150413 | `12af26d17e43` | Atomic load for `gc.get_count()` young count | Very low, no known crash |\n| gh-151644 | #151768 | `cde31ec13590` | Atomic `sys.getdlopenflags()` / `sys.setdlopenflags()` | Very low, no known crash |\n\nThe last two are one- and two-line atomics fixes with no known crash consequence; they are included because they are trivial, not because they are urgent.\n\n### Missing from 3.14 only (3.15 already has them)\n\nConcurrent iteration safety in `itertools`, all under gh-123471. Most wrap the iterator state transition in an object critical section; `cycle` instead uses atomic index accesses plus `PyList_GetItemRef()`.\n\n| Issue | PR | `main` commit | Iterator(s) |\n|---|---|---|---|\n| gh-123471 | #131212 | `26a1cd4e8c0c` | `cycle` |\n| gh-123471 | #132814 | `847d1c2cb401` | `product`, `combinations` |\n| gh-123471 | #135689 | `0533c1faf27d` | `chain` |\n| gh-123471 | #144402 | `009c8c052f5e` | `permutations`, `combinations_with_replacement` |\n| gh-123471 | #144486 | `3a2485644700` | `accumulate` |\n| gh-123471 | #146033 | `9214e3f33eee` | `zip_longest` |\n\nOther memory-safety fixes:\n\n| Issue | PR | `main` commit | Summary | Severity |\n|---|---|---|---|---|\n| gh-129069 | #142957 | `e46f28c6afce` | Pointer tearing and uninitialized reads in memmove-style list operations | Medium |\n| gh-129069 | #143019 | `487e91c1203d` | Release stores for list reverse, insert, delete, extended-slice assign | Low |\n| gh-143100 | #143127 | `e8e044eda343` | Pointer tearing from `memcpy()` while swapping set small tables | Low |\n| gh-132070 | #143441 | `98e55d70bcb7` | Atomic object-header init in free-threaded `PyObject_Realloc()` | Medium |\n\n### Deliberately excluded\n\nNot backported here. Each needs its own decision.\n\n| Issue/PR | `main` commit | Why excluded |\n|---|---|---|\n| #142599 | `08bc03ff2a55` | Atomic generator frame-state transitions; the commit says it prevents free-threaded segfaults, but it is a large change to generated interpreter files and does not apply cleanly to 3.14. Deserves a dedicated design review. |\n| #144292 | `a01694dacd8e` | Protects `gi_yieldfrom` with a frame-state lock; should be reviewed together with #142599. |\n| #144980 | `a56532771a9e` | Code-extra C APIs vs concurrent buffer growth/replacement. Medium-sized C API/QSBR change. |\n| #143818 | `bcf9cb0217fd` | OpenSSL races during concurrent `SSLContext.load_cert_chain()`. Medium-sized SSL locking change; a current 3.14 TSan backport explicitly notes this race is still unfixed in 3.14. |\n| #151766 | `8b1dbb175404` | `gc.get_stats()` race, described by its own regression test as benign at the Python level. The fix adds a GC statistics mutex and needs adaptation for 3.14's different stats layout. Not worth it. |\n\n### Already backported upstream\n\nFound by the audit, but landed in 3.14 in the meantime (the audit snapshot `8e648a9c6b29` predates the current tip). No action needed.\n\n| Issue | PR | Now in 3.14 as |\n|---|---|---|\n| gh-145142 | #145157 | `d76c56e958c` `[3.14] gh-145142: Make str.maketrans safe under free-threading (#145320)` |\n| gh-148820 | #148852 | `e5d55416833` `[3.14] gh-148820: Fix _PyRawMutex use-after-free on spurious semaphore wakeup (#148884)` |\n\n## Clean vs. manual backports\n\n9 of the 20 branches were produced automatically by `cherry_picker`; 11 needed manual conflict resolution. In every manual case the C source change is byte-identical to the upstream commit, with one deliberate exception (gh-149816 on 3.14, below). The conflicts were in auxiliary files:\n\n- **itertools (6 PRs, 3.14)** - 3.14's `Lib/test/test_free_threading/test_itertools.py` has a `TestTeeConcurrent` class from a later `tee` backport, while upstream's copy has an `ItertoolsThreading` class that grew test by test. Each backport keeps 3.14's existing tee tests and adds only the test(s) its own PR introduced. [#131212](https://github.com/python/cpython/pull/131212) also wanted to delete `test_itertools_batched.py` (upstream folded it into `test_itertools.py`); that is unrelated to the fix, so the file was kept.\n- **[gh-129069](https://github.com/python/cpython/issues/129069) [#142957](https://github.com/python/cpython/pull/142957), [gh-143100](https://github.com/python/cpython/issues/143100), [gh-132070](https://github.com/python/cpython/issues/132070)** - conflicts only in `Tools/tsan/suppressions_free_threading.txt`, where each commit removes the suppression it retires. 3.14's copy has extra entries and, for the set fix, never had the `set_swap_bodies` suppression at all. Resolved by removing exactly the suppression each fix retires.\n- **[gh-129069](https://github.com/python/cpython/issues/129069) [#143019](https://github.com/python/cpython/pull/143019)** - both sides add tests at the same point in `test_list.py`; kept both.\n- **[gh-149816](https://github.com/python/cpython/issues/149816) on 3.14 (adapted, the one non-identical case)** - 3.14's `batch_list_exact()` has the single-item fast path *before* the batch loop; `main` moved it *inside* the loop, which changes the emitted pickle byte stream (`APPEND` vs `MARK ... APPENDS` for a trailing single item). To avoid changing 3.14's pickle output, 3.14's structure was kept and only the fix applied: the borrowed `PyList_GET_ITEM()` + `Py_INCREF()` became `PyList_GetItemRef()`, and the `list changed size during iteration` `RuntimeError` check was added after each batch. The upstream regression test (`test_pickle_dumps_with_concurrent_list_mutations`) tolerates `RuntimeError`/`IndexError` and passes unchanged.\n\n## Testing\n\nEvery one of the 20 branches builds and passes the **full test suite in both a free-threaded and a GIL-enabled `--with-pydebug` build**.\n\n- Test command: `make test` (`--fast-ci`: whole suite, parallel, `-u all,-gui`), free-threaded first, then GIL, one branch at a time.\n- Baseline on the unpatched 3.14 and 3.15 tips, both configs: 489/492 test files run, 0 failures. So any failure on a backport branch would be attributable to the backport itself.\n\n## Branches\n\nAll in [`nascheme/cpython`](https://github.com/nascheme/cpython), one commit each, cut from the current 3.14/3.15 tips. \"Method\" is how the backport was produced, not how risky it is - the manual ones are byte-identical to upstream apart from the auxiliary-file conflicts described above.\n\n### 3.15\n\n| Issue | PR | Branch | Method |\n|---|---|---|---|\n| gh-149816 | #150024 | [`ft-backport/gh-149816-3.15`](https://github.com/nascheme/cpython/tree/ft-backport/gh-149816-3.15) | cherry_picker |\n| gh-144774 | #150578 | [`ft-backport/gh-144774-3.15`](https://github.com/nascheme/cpython/tree/ft-backport/gh-144774-3.15) | cherry_picker |\n| gh-150858 | #150859 | [`ft-backport/gh-150858-3.15`](https://github.com/nascheme/cpython/tree/ft-backport/gh-150858-3.15) | cherry_picker |\n| gh-150411 | #150413 | [`ft-backport/gh-150411-3.15`](https://github.com/nascheme/cpython/tree/ft-backport/gh-150411-3.15) | cherry_picker |\n| gh-151644 | #151768 | [`ft-backport/gh-151644-3.15`](https://github.com/nascheme/cpython/tree/ft-backport/gh-151644-3.15) | cherry_picker |\n\n### 3.14\n\n| Issue | PR | Branch | Method |\n|---|---|---|---|\n| gh-149816 | #150024 | [`ft-backport/gh-149816-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-149816-3.14) | manual (adapted, see above) |\n| gh-144774 | #150578 | [`ft-backport/gh-144774-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-144774-3.14) | cherry_picker |\n| gh-150858 | #150859 | [`ft-backport/gh-150858-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-150858-3.14) | cherry_picker |\n| gh-150411 | #150413 | [`ft-backport/gh-150411-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-150411-3.14) | cherry_picker |\n| gh-151644 | #151768 | [`ft-backport/gh-151644-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-151644-3.14) | cherry_picker |\n| gh-123471 | #131212 | [`ft-backport/gh-123471-131212-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-131212-3.14) | manual |\n| gh-123471 | #132814 | [`ft-backport/gh-123471-132814-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-132814-3.14) | cherry_picker |\n| gh-123471 | #135689 | [`ft-backport/gh-123471-135689-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-135689-3.14) | manual |\n| gh-123471 | #144402 | [`ft-backport/gh-123471-144402-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-144402-3.14) | manual |\n| gh-123471 | #144486 | [`ft-backport/gh-123471-144486-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-144486-3.14) | manual |\n| gh-123471 | #146033 | [`ft-backport/gh-123471-146033-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-123471-146033-3.14) | manual |\n| gh-129069 | #142957 | [`ft-backport/gh-129069-142957-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-129069-142957-3.14) | manual |\n| gh-129069 | #143019 | [`ft-backport/gh-129069-143019-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-129069-143019-3.14) | manual |\n| gh-143100 | #143127 | [`ft-backport/gh-143100-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-143100-3.14) | manual |\n| gh-132070 | #143441 | [`ft-backport/gh-132070-3.14`](https://github.com/nascheme/cpython/tree/ft-backport/gh-132070-3.14) | manual |\n\n## Checklist\n\nBackport PRs to open, once the set above is agreed:\n\n3.15:\n\n- [ ] [gh-149816](https://github.com/python/cpython/issues/149816) / [#150024](https://github.com/python/cpython/pull/150024) - `pickle.dumps()` UAF\n- [ ] [gh-144774](https://github.com/python/cpython/issues/144774) / [#150578](https://github.com/python/cpython/pull/150578) - `BaseException.__setstate__()`\n- [ ] [gh-150858](https://github.com/python/cpython/issues/150858) / [#150859](https://github.com/python/cpython/pull/150859) - type `__qualname__`\n- [ ] [gh-150411](https://github.com/python/cpython/issues/150411) / [#150413](https://github.com/python/cpython/pull/150413) - `gc.get_count()`\n- [ ] [gh-151644](https://github.com/python/cpython/issues/151644) / [#151768](https://github.com/python/cpython/pull/151768) - `sys.get/setdlopenflags()`\n\n3.14:\n\n- [ ] [gh-149816](https://github.com/python/cpython/issues/149816) / [#150024](https://github.com/python/cpython/pull/150024) - `pickle.dumps()` UAF (adapted)\n- [ ] [gh-144774](https://github.com/python/cpython/issues/144774) / [#150578](https://github.com/python/cpython/pull/150578) - `BaseException.__setstate__()`\n- [ ] [gh-150858](https://github.com/python/cpython/issues/150858) / [#150859](https://github.com/python/cpython/pull/150859) - type `__qualname__`\n- [ ] [gh-150411](https://github.com/python/cpython/issues/150411) / [#150413](https://github.com/python/cpython/pull/150413) - `gc.get_count()`\n- [ ] [gh-151644](https://github.com/python/cpython/issues/151644) / [#151768](https://github.com/python/cpython/pull/151768) - `sys.get/setdlopenflags()`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#131212](https://github.com/python/cpython/pull/131212) - `itertools.cycle`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#132814](https://github.com/python/cpython/pull/132814) - `itertools.product`, `combinations`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#135689](https://github.com/python/cpython/pull/135689) - `itertools.chain`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#144402](https://github.com/python/cpython/pull/144402) - `itertools.permutations`, `combinations_with_replacement`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#144486](https://github.com/python/cpython/pull/144486) - `itertools.accumulate`\n- [ ] [gh-123471](https://github.com/python/cpython/issues/123471) / [#146033](https://github.com/python/cpython/pull/146033) - `itertools.zip_longest`\n- [ ] [gh-129069](https://github.com/python/cpython/issues/129069) / [#142957](https://github.com/python/cpython/pull/142957) - list memmove tearing\n- [ ] [gh-129069](https://github.com/python/cpython/issues/129069) / [#143019](https://github.com/python/cpython/pull/143019) - list release stores\n- [ ] [gh-143100](https://github.com/python/cpython/issues/143100) / [#143127](https://github.com/python/cpython/pull/143127) - set `memcpy` tearing\n- [ ] [gh-132070](https://github.com/python/cpython/issues/132070) / [#143441](https://github.com/python/cpython/pull/143441) - `PyObject_Realloc()` header init\n\nOpen questions for the release managers:\n\n- Are the two no-known-crash atomics fixes ([#150413](https://github.com/python/cpython/pull/150413), [#151768](https://github.com/python/cpython/pull/151768)) worth backporting at all, or is the churn not justified?\n- Should the six itertools fixes go in as one coordinated 3.14 batch rather than six PRs?\n- Is the generator frame-state crash ([#142599](https://github.com/python/cpython/pull/142599) / [#144292](https://github.com/python/cpython/pull/144292)) severe enough to justify the dedicated backport work it needs?\n","author":{"url":"https://github.com/nascheme","@type":"Person","name":"nascheme"},"datePublished":"2026-07-14T19:00:45.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/153714/cpython/issues/153714"}