Title: Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading · Issue #154535 · python/cpython · GitHub
Open Graph Title: Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading · Issue #154535 · python/cpython
X Title: Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading · Issue #154535 · python/cpython
Description: Crash report What happened? On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and dereferences a garbage node pointer → SIGSEGV. The H...
Open Graph Description: Crash report What happened? On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and dereferences...
X Description: Crash report What happened? On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and derefere...
Opengraph URL: https://github.com/python/cpython/issues/154535
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading","articleBody":"# Crash report\n\n### What happened?\n\nOn the free-threaded build, advancing a single shared iterator over a `contextvars.Context` from multiple threads corrupts the iterator's traversal cursor and dereferences a garbage node pointer → **SIGSEGV**.\n\nThe HAMT iterator keeps its whole cursor — `i_nodes[]` (borrowed node pointers), `i_pos[]`, `i_level` — inside the iterator object and advances it with plain reads/writes and **no critical section** (`Python/hamt.c`, main@`a1d580430c8`):\n\n```c\nstatic hamt_iter_t\nhamt_iterator_next(PyHamtIteratorState *iter, PyObject **key, PyObject **val) {\n if (iter->i_level \u003c 0) /* :2188 plain read of the cursor level */\n return I_END;\n PyHamtNode *current = iter->i_nodes[iter->i_level]; /* :2194 read node at current level */\n if (IS_BITMAP_NODE(current)) { /* :2196 Py_TYPE(current) -> SEGV on a wild node */\n return hamt_iterator_bitmap_next(iter, key, val);\n }\n ...\n}\n\nstatic hamt_iter_t\nhamt_iterator_bitmap_next(PyHamtIteratorState *iter, ...) {\n int8_t level = iter->i_level;\n PyHamtNode_Bitmap *node = (PyHamtNode_Bitmap *)(iter->i_nodes[level]);\n if (pos + 1 >= Py_SIZE(node)) { /* :2092 Py_SIZE(node) -> SEGV on a wild node */\n iter->i_level--; /* :2097 plain write of the cursor level */\n return hamt_iterator_next(iter, key, val);\n }\n ...\n}\n```\n\nTwo threads calling `next()` on the same iterator desync `i_level` against `i_nodes[]`, so a thread reads `current = iter->i_nodes[iter->i_level]` as a stale / `NULL` / wild pointer and immediately evaluates `IS_BITMAP_NODE(current)` = `Py_TYPE(current)` (or `Py_SIZE(node)` in `hamt_iterator_bitmap_next`) — dereferencing garbage. Because the nodes in `i_nodes` are held **borrowed** (`hamt_iterator_init`: \"we don't incref/decref nodes in i_nodes\"), a concurrent structural change is not even required — cursor desync alone produces the wild dereference. This is safe under the GIL (one thread runs the advance at a time).\n\nThe HAMT iterator backs `contextvars.Context` iteration — `iter(ctx)`, `ctx.keys()`, `ctx.values()`, `ctx.items()` — via `hamt_baseiter_tp_iternext` (`hamt.c:2479`).\n\n### Reproducer\n\nFree-threaded build, `PYTHON_GIL=0`:\n\n```python\nimport contextvars, threading\nNT = 8; ITERS = 40000\nvs = [contextvars.ContextVar(f\"v{i}\") for i in range(16)]\nctx = contextvars.copy_context()\ndef populate():\n for i, v in enumerate(vs):\n v.set(i)\nctx.run(populate)\ncell = [iter(ctx)]\ndef worker():\n for _ in range(ITERS):\n it = cell[0]\n try: next(it)\n except StopIteration: cell[0] = iter(ctx)\n except Exception: pass\nts = [threading.Thread(target=worker) for _ in range(NT)]\nfor t in ts: t.start()\nfor t in ts: t.join()\n```\n\n- **6/6 SIGSEGV** on a plain free-threaded debug build (no sanitizer).\n- On a free-threaded ASan build:\n\n```\nAddressSanitizer: SEGV in _Py_TYPE_impl Include/object.h:234\n hamt_iterator_bitmap_next Python/hamt.c:2092\n hamt_iterator_next Python/hamt.c:2196\n hamt_baseiter_tp_iternext Python/hamt.c:2479\n context_run Python/context.c:731\n```\n\n- Under TSan: `WARNING: ThreadSanitizer: data race … Python/hamt.c:2188 in hamt_iterator_next`.\n\nThe crash is not `Py_DEBUG`-only — it reproduces as a raw SIGSEGV from the wild-pointer dereference on the plain free-threaded build.\n\n### Relationship to gh-124397\n\nPer gh-124397 (\"Strategy for Iterators in Free Threading\") point 3, C iterators should get \"the minimal changes necessary to cause them to **not crash** … Concurrent access is allowed to return duplicate values, skip values, or raise an exception.\" The HAMT / `Context` iterator was never brought under that hardening, so it crashes rather than returning dup/skip. It is the `contextvars.Context` sibling of the already-tracked shared-iterator crashes: dict (#154130), set (#144357), and memoryview.\n\n### Suggested fix\n\nBring the HAMT iterator advance under a per-iterator critical section (or make the `i_level`/`i_pos`/`i_nodes` cursor accesses atomic with a NULL/bounds check on the borrowed `i_nodes[i_level]` before dereferencing), matching the `dictiter`/`setiter` hardening. The guard must span the whole `hamt_iterator_next` → `*_bitmap_next` / `*_array_next` descent because `i_nodes` holds borrowed pointers.\n\n*Found with [fusil](https://github.com/devdanzin/fusil)'s `--tsan` mode; reproducer reduced and this report drafted with AI assistance (Claude Code), then verified by hand on free-threaded debug / ASan / TSan builds of `main`.*\n\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n### Output from running 'python -VV' on the command line:\n\nPython 3.16.0a0 free-threading build (heads/main:a1d580430c8, Jul 18 2026, 20:23:36) [Clang 21.1.8 (6ubuntu1)]\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-154586\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/devdanzin","@type":"Person","name":"devdanzin"},"datePublished":"2026-07-23T11:12:54Z","url":"https://github.com/python/cpython/issues/154535"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:ae64d7cc-32e3-a94f-bed2-326ad8f31203 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | EBEA:1087D4:34CCA1F:4688103:6B020978 |
| html-safe-nonce | 2a0ee99531d0c8faa0dcaf4ad71bb5fdb872764498ac5d406a108bbecd531758 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQkVBOjEwODdENDozNENDQTFGOjQ2ODgxMDM6NkIwMjA5NzgiLCJ2aXNpdG9yX2lkIjoiMjc4MDIzODIxNjMyNDg0Mzg5NiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | de578d9966be570470c6b8f26c9c18e188da9925c081ba3ac6625bf1c8bc881c |
| hovercard-subject-tag | issue:4958244703 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/154535/issue_layout |
| twitter:image | https://opengraph.githubassets.com/50b70786673b17314ef539746acdb04c0e99f37e07e21900789763ab38b5c78a/python/cpython/issues/154535 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/50b70786673b17314ef539746acdb04c0e99f37e07e21900789763ab38b5c78a/python/cpython/issues/154535 |
| og:image:alt | Crash report What happened? On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and dereferences... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | devdanzin |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width