Title: Free-threading: PyType_Freeze() can deadlock with concurrent class updates · Issue #158194 · python/cpython · GitHub
Open Graph Title: Free-threading: PyType_Freeze() can deadlock with concurrent class updates · Issue #158194 · python/cpython
X Title: Free-threading: PyType_Freeze() can deadlock with concurrent class updates · Issue #158194 · python/cpython
Description: Bug description: On free-threaded builds of main and 3.15, PyType_Freeze() can deadlock with a concurrent class update such as K.__eq__ = f (which @dataclass(eq=True, order=True) does). The pattern: one thread freezes a type while a seco...
Open Graph Description: Bug description: On free-threaded builds of main and 3.15, PyType_Freeze() can deadlock with a concurrent class update such as K.__eq__ = f (which @dataclass(eq=True, order=True) does). The pattern...
X Description: Bug description: On free-threaded builds of main and 3.15, PyType_Freeze() can deadlock with a concurrent class update such as K.__eq__ = f (which @dataclass(eq=True, order=True) does). The pattern...
Opengraph URL: https://github.com/python/cpython/issues/158194
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Free-threading: PyType_Freeze() can deadlock with concurrent class updates","articleBody":"### Bug description:\n\nOn free-threaded builds of main and 3.15, `PyType_Freeze()` can deadlock with a concurrent class update such as `K.__eq__ = f` (which `@dataclass(eq=True, order=True)` does). The pattern: one thread freezes a type while a second thread assigns a special method on a class and a stop-the-world is pending (another thread, or the GC). `_zstd` calls `PyType_Freeze()` from its module exec, and `import shutil`, `tempfile` and `zipfile` all import `_zstd`.\n\nObserved on fresh free-threaded builds of main 8c056c33 and 3.15 13b1659d (macOS arm64, Clang, 2026-09-25), and earlier on main 6757482 and 3.15 2dafe2a. 3.14 (whose `PyType_Freeze()` does not stop the world) and the default build are not affected.\n\nThis is a follow-up to gh-156910 (closed) and its fix, PR #156948, which pinned `TYPE_LOCK` in `type_set_abstractmethods()` but did not touch `PyType_Freeze()`. `PyType_Freeze()` is now the only place in `Objects/typeobject.c` that stops the world while holding `TYPE_LOCK` unpinned.\n\n#### Reproducer (high-hit-rate stress reproducer)\n\nOne thread re-imports `_zstd` while others assign a special method and stop the world:\n\n```python\nimport faulthandler, importlib, sys, threading, time\nprint(\"sys._is_gil_enabled() =\", sys._is_gil_enabled())\nif sys._is_gil_enabled() is not False:\n sys.exit(\"needs a free-threaded build with the GIL disabled\")\nfaulthandler.dump_traceback_later(60, exit=True) # dump stacks and exit if hung\nimportlib.import_module(\"_zstd\")\nN, BURST, done, errors = 500, 50, False, []\n\ndef importer(): # _zstd_exec -> PyType_Freeze(): TYPE_LOCK, then stop-the-world\n global done\n try:\n for _ in range(N):\n sys.modules.pop(\"_zstd\", None)\n importlib.import_module(\"_zstd\")\n except BaseException as e:\n errors.append(e); raise\n finally:\n done = True\n\ndef slot_setter(): # type_setattro -> apply_type_slot_updates: pinned TYPE_LOCK, then stop-the-world\n class K: pass\n def eq(self, other): return NotImplemented\n while not done:\n for _ in range(BURST):\n K.__eq__ = eq; del K.__eq__\n time.sleep(0.001)\n\ndef stw_user(): # func_set_qualname: stop-the-world without TYPE_LOCK (as the GC does)\n def f(): pass\n while not done:\n for _ in range(BURST):\n f.__qualname__ = \"f\"\n time.sleep(0.001)\n\nts = [threading.Thread(target=t) for t in (importer, slot_setter, stw_user)]\nfor t in ts: t.start()\nfor t in ts: t.join()\nif errors: sys.exit(f\"FAIL: importer raised {errors[0]!r}\")\nprint(\"ok\")\n```\n\n- **Actual:** the process hangs. faulthandler dumps the stacks after 60 s and exits with status 1. The attached `repro.py` hung 5/5 on each fresh stock build (main debug and release, 3.15 debug).\n- **Expected:** the script terminates, printing `ok`. On the patched builds, `repro.py` has taken 6.1–21.6 s (main debug) and 7.6–20.6 s (3.15 debug); under contention the patched freezer waits while holding `TYPE_LOCK`.\n\n#### Native stacks of a hang\n\nStock main 8c056c33 (free-threaded debug, no patch), `repro.py`, captured with macOS `sample` (trimmed; every hang sampled had this shape):\n\n```\nimporter _zstd_exec _zstdmodule.c:758 -> PyType_Freeze typeobject.c:12467\n -> stop_the_world pystate.c:2475 -> _PyMutex_LockTimed -> _PyParkingLot_Park\n -> _PyThreadState_Attach pystate.c:2305 -> _PyCriticalSection_Resume critical_section.c:152\n -> _PyMutex_LockTimed\nslot_setter type_setattro typeobject.c:6720 -> update_slot_after_setattr typeobject.c:6618\n -> apply_type_slot_updates typeobject.c:3904 -> stop_the_world pystate.c:2475\n -> _PyMutex_LockTimed\nstw_user func_set_qualname funcobject.c:755 -> stop_the_world pystate.c:2475 -> _PyMutex_LockTimed\n```\n\nThese stacks are consistent with the lock-cycle diagnosis below: the woken importer is re-acquiring `TYPE_LOCK` while `slot_setter` waits for `stw->mutex` on a path that pins `TYPE_LOCK`. Lock ownership is inferred from the code paths, not read from the samples.\n\n#### Mechanism\n\n1. `PyType_Freeze()` takes `TYPE_LOCK`, then calls `types_stop_world()` without `type_lock_prevent_release()`. If `stw->mutex` is busy, it parks. Parking detaches the thread, which suspends its critical sections and releases `TYPE_LOCK`.\n2. A second thread takes `TYPE_LOCK` on a pinned path, such as `apply_type_slot_updates()`, and parks on `stw->mutex` while still holding it.\n3. `stw->mutex` is handed to the freezing thread. On re-attach, `_PyCriticalSection_Resume()` waits for `TYPE_LOCK` while that thread owns `stw->mutex`: ABBA.\n\n#### Proposed fix\n\nPin `TYPE_LOCK` around the stop-the-world in `PyType_Freeze()`, following the pattern of merged PR #156948 for `type_set_abstractmethods()` (`fix-main.diff`, `fix-3.15.diff`). The C change is 8 lines; the diffs also add a regression test in `test_free_threading/test_type.py` and a NEWS entry.\n\nNo failure was observed on the patched builds tested (debug only): `repro.py` terminated 5/5 on patched main and 5/5 on patched 3.15, and the regression test passed 3/3 in-tree on each (its standalone copy hung 3/3 on each stock build).\n\n**Release timing:** 3.15.0 final is expected on 2026-10-01 ([PEP 790](https://peps.python.org/pep-0790/)). Could the release manager assess whether this should block 3.15.0? I have not applied any priority label.\n\n\u003cdetails>\n\u003csummary>Validation on fresh builds and environment\u003c/summary>\n\nFresh out-of-tree free-threaded builds of main 8c056c33 and 3.15 13b1659d, stock and with exactly the attached `fix-main.diff` (sha256 `42e37353…`) or `fix-3.15.diff` (`fa33c860…`), run one at a time on 2026-09-25. A run counts as hung only when the process stopped using CPU and a native `sample` showed stacks consistent with the lock cycle above.\n\n| | stock | patched |\n|---|---|---|\n| `repro.py`, main debug | 5/5 hung | 0/5 hung (5/5 terminated, 6.1–8.1 s) |\n| `repro.py`, 3.15 debug | 5/5 hung | 0/5 hung (5/5 terminated, 7.6–20.6 s) |\n| `repro.py`, main release | 5/5 hung | (not built) |\n| regression test, main / 3.15 | 3/3 / 3/3 hung (standalone) | 3/3 / 3/3 pass (in-tree) |\n\n`--control` completed 2/2 on stock and patched. `test_free_threading.test_type`, `test_types`, `test_descr`, `test_capi.test_type` and `test_zstd` pass on both patched builds. Not run on fresh builds: `repro_natural.py`, `repro_ctypes.py`, `single_shot_tempfile.py`, patched release builds, 3.14.\n\n`python -VV` of the fresh stock builds:\n\n```\nPython 3.16.0a0 free-threading build (heads/val/main-base-dirty:8c056c3, Sep 25 2026, 08:58:37) [Clang 21.0.0 (clang-2100.3.34.2)]\nPython 3.15.0rc2+dev free-threading build (heads/val/315-base-dirty:13b1659, Sep 25 2026, 09:07:27) [Clang 21.0.0 (clang-2100.3.34.2)]\n```\n\nEarlier stock builds (used for the older results below):\n\n```\nPython 3.16.0a0 free-threading build (heads/main-dirty:6757482, Sep 22 2026, 20:29:11) [Clang 21.0.0 (clang-2100.3.34.2)]\nPython 3.15.0rc2+dev free-threading build (heads/3.15-dirty:2dafe2a, Sep 22 2026, 21:35:07) [Clang 21.0.0 (clang-2100.3.34.2)]\nPython 3.14.7+ free-threading build (heads/3.14-dirty:1d8c9f8, Sep 22 2026, 20:25:42) [Clang 21.0.0 (clang-2100.3.34.2)]\n```\n\nThe `-dirty` suffix comes from CPython's Makefile running `git describe --dirty` from the out-of-tree build directory, where git sees every tracked file as missing. All of these builds were made out-of-tree, and their source trees show no tracked changes; the fresh trees were verified clean after each build.\n\n\u003c/details>\n\n\u003cdetails>\n\u003csummary>Older results, variants, root cause and patch discussion\u003c/summary>\n\n**Attached reproducers.** Each one prints `sys._is_gil_enabled()` and exits with status 2 unless it is `False`. A worker exception, or an importer that did not finish, exits with status 3.\n\n- `repro.py`: the script above, plus `--control` (re-imports `_json`, which does not freeze types) and a timeout argument.\n- `repro_natural.py`: no explicit stop-the-world setter. `_zstd` re-import, `@dataclass(eq=True, order=True)` classes, and cyclic garbage (automatic GC).\n- `single_shot_tempfile.py`: a fresh process doing **one** first `import tempfile` (or a module name given as an argument, e.g. `compression.zstd`) while other threads define dataclasses and allocate garbage. It hangs only occasionally, so run it in a loop.\n- `repro_ctypes.py`: calls the public `PyType_Freeze` through `ctypes.pythonapi`.\n\n**Older stock-build results.** These are stock builds (no patch) of main @ 6757482, 3.15 @ 2dafe2a and 3.14 @ 1d8c9f8, run on 2026-09-25 with the reproducers as they were before the GIL check and the exception propagation were added (the stress logic is the same). CPU-time inactivity (less than 0.02 s over 5 s) was used only to screen for a hang.\n\n| Build | `repro.py` | `repro_ctypes.py` | `repro_natural.py` | 1× `import tempfile` | 1× `import compression.zstd` |\n|---|---|---|---|---|---|\n| main FT debug | 3/3 hung | – | – | – | – |\n| main FT release | 3/3 hung | 2/2 hung | 2/2 hung | 1/60 hung | 2/60 hung |\n| 3.15 FT debug | 3/3 hung | – | 1/2 hung | 1/40 hung | – |\n| 3.14 FT debug | 0/3 | – | 0/2 | 0/30 | – |\n\nControls on main release: `repro.py --control` 0/2, and the single-shot script importing `json` 0/40. The standalone regression test hung 1/1 on stock main release. The single-shot rates are observations of this stress configuration (tight dataclass and garbage loops). They do not estimate how often this happens in production.\n\nA native `sample` of one first `import tempfile` in `single_shot_tempfile.py` on stock main 6757482 (release) had the same shape as the stacks above (`_zstd_exec _zstdmodule.c:758 -> PyType_Freeze typeobject.c:12467 -> stop_the_world pystate.c:2454 -> _PyThreadState_Attach pystate.c:2305 -> _PyCriticalSection_Resume critical_section.c:152`, with a dataclass-creating thread in `type_setattro typeobject.c:6720 -> update_slot_after_setattr -> stop_the_world` and the GC in `gc_collect_main -> stop_the_world`). The 3.15 2dafe2a debug stack has the same shape (`PyType_Freeze` `typeobject.c:12616` → `stop_the_world` `pystate.c:2423` → `_PyCriticalSection_Resume`).\n\n**Earlier investigation (2026-09-23; stock main 6757482 / 3.15 2dafe2a; earlier revision of the patch).** The `Objects/typeobject.c` change is identical to the attached diffs. The NEWS text differs, and the regression test may differ (one earlier revision lacked a `try/finally`).\n\n- Stock: `repro.py` and `repro_ctypes.py` each hung 10/10 on main debug, main release and 3.15 debug, and 0/10 on 3.14 debug. With a plain 4 s timeout (no CPU check), one `import tempfile` failed to finish in 4/150 runs and one `import compression.zstd` in 11/150 on main release.\n- Patched: both stress reproducers completed 10/10 on main debug, main release and 3.15 debug, and `repro_natural.py` 4/4 on main release. Single-shot `tempfile` and `compression.zstd` had 0/150 timeouts each. The regression test passed 10/10.\n- On the patched builds, `test_free_threading`, `test_capi.test_type`, `test_abc`, `test_descr` and `test_zstd` passed. Under contention the patched freezer took longer to complete, because it holds `TYPE_LOCK` while it waits (the fresh 3.15 debug runs above took up to 20.6 s). No benchmark was run.\n\n**Root cause.** main [`Objects/typeobject.c#L12466-L12472`](https://github.com/python/cpython/blob/6757482c25d4fa308d3f3baaa57b6db1a8c720f5/Objects/typeobject.c#L12466-L12472) (3.15 [`#L12615-L12621`](https://github.com/python/cpython/blob/2dafe2a2ccdfb943940892f50a7efe6fa54ee118/Objects/typeobject.c#L12615-L12621)):\n\n```c\n BEGIN_TYPE_LOCK();\n types_stop_world(); // may park on stw->mutex -> TYPE_LOCK released\n type_add_flags(type, Py_TPFLAGS_IMMUTABLETYPE);\n types_start_world();\n ASSERT_TYPE_LOCK_HELD();\n _PyType_Modified_Unlocked(type);\n END_TYPE_LOCK();\n```\n\nEvery other `types_stop_world()` under `TYPE_LOCK` is pinned (main lines 673, 1713, 1909, 1975, 3904, 6402). Those pinned sites are only correct if no thread ever waits for `TYPE_LOCK` while it owns `stw->mutex`. The stop-the-world in `PyType_Freeze()` came from gh-127266 / GH-133177 (fbbbc10055, 3.15). `_zstd` started calling it in gh-132983 / GH-133784. The unpinned code is still present at the tips of main and 3.15 (checked 2026-09-25).\n\n**The patch** (explanatory comment omitted):\n\n```diff\n+ pinned_mutexes_t pinned;\n BEGIN_TYPE_LOCK();\n+ type_lock_prevent_release(&pinned);\n types_stop_world();\n type_add_flags(type, Py_TPFLAGS_IMMUTABLETYPE);\n types_start_world();\n+ type_lock_allow_release(&pinned);\n```\n\nAlternatives are to stop the world outside `TYPE_LOCK`, as `_PyType_SetFlags()` does, or to drop the stop-the-world, since the documentation says a type must not be used before it is frozen. The pin was chosen to match PR #156948 and to keep the 3.15.0 change small.\n\n**Precedent:** gh-155400, the same defect class, was marked release blocker.\n\n\u003c/details>\n\n#### How it was found\n\nAn audit of `types_stop_world()` calls made under `TYPE_LOCK` found this as the one unpinned site left after gh-156910.\n\n### CPython versions tested on:\n\n3.14, 3.15, CPython main branch\n\n### Operating systems tested on:\n\nmacOS\n\n\n#### Attachments\n\n[repro.py.txt](https://github.com/user-attachments/files/32672638/repro.py.txt)\n[repro_natural.py.txt](https://github.com/user-attachments/files/32672637/repro_natural.py.txt)\n[single_shot_tempfile.py.txt](https://github.com/user-attachments/files/32672636/single_shot_tempfile.py.txt)\n[repro_ctypes.py.txt](https://github.com/user-attachments/files/32672639/repro_ctypes.py.txt)\n\n[fix-main.diff.txt](https://github.com/user-attachments/files/32672664/fix-main.diff.txt)\n[fix-3.15.diff.txt](https://github.com/user-attachments/files/32672663/fix-3.15.diff.txt)\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-158674\n* gh-161655\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/scasella","@type":"Person","name":"scasella"},"datePublished":"2026-09-25T23:24:43Z","url":"https://github.com/python/cpython/issues/158194"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:19fb6984-5668-c5f2-c76a-88c466ff2a8f |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D5E8:380435:34E574:42CED8:6B01FA2D |
| html-safe-nonce | a36a7cb6c26571b379224cd5d0ba8af76557694c398ee894a0035e60665e147a |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENUU4OjM4MDQzNTozNEU1NzQ6NDJDRUQ4OjZCMDFGQTJEIiwidmlzaXRvcl9pZCI6Ijg1NDg1NjUwNTY0ODMzNTkyNzciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 67bcfd4df12eaacc09baba66d2ff9ecc002a603220f4203e41ba3c70a73c748e |
| hovercard-subject-tag | issue:5589609987 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/158194/issue_layout |
| twitter:image | https://opengraph.githubassets.com/a2338a9ad5d83f493a5edfc77b77e7ab2d21e459f23015051b74a12717ac0810/python/cpython/issues/158194 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/a2338a9ad5d83f493a5edfc77b77e7ab2d21e459f23015051b74a12717ac0810/python/cpython/issues/158194 |
| og:image:alt | Bug description: On free-threaded builds of main and 3.15, PyType_Freeze() can deadlock with a concurrent class update such as K.__eq__ = f (which @dataclass(eq=True, order=True) does). The pattern... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | scasella |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width