René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"_decimal: `mpd_context_t::status`/`traps` mutated non-atomically leading to data race","articleBody":"# Bug report\n\n### Bug description:\n\n## Summary\n\n`Modules/_decimal/_decimal.c` performs unsynchronized read-modify-write and plain stores on the `status` and `traps` fields of `mpd_context_t` embedded in a Python `decimal.Context`. Whenever a `Context` instance is reachable from more than one free-threaded thread — explicitly via `context=` arguments, `Context.\u003cmethod\u003e(…)`, `ctx.flags[…] = …`, etc. — those accesses race.\n\n#141148 + #146482 fix the *implicit* sharing case (inherited context via `contextvars`). This issue is for the underlying primitive, which is independent of how the `Context` ended up shared and is still racy after #146482.\n\n## Affected sites\n\nAll in `Modules/_decimal/_decimal.c`:\n\n| Line | Code | Reached from Python by |\n|------|------|------------------------|\n| 616  | `ctx-\u003estatus \\|= status;` (in `dec_addstatus`) | any arithmetic on the context |\n| 617, 625, 629 | reads of `ctx-\u003etraps` (in `dec_addstatus` trap path) | same |\n| 715  | `SdFlags(self) \u0026 flag` (in `signaldict_getitem`) | `ctx.flags[X]`, `ctx.traps[X]` |\n| 744  | `SdFlags(self) \\|= flag;` (in `signaldict_setitem`) | `ctx.flags[X] = True`, `ctx.traps[X] = True` |\n| 747  | `SdFlags(self) \u0026= ~flag;` (in `signaldict_setitem`) | `ctx.flags[X] = False`, `ctx.traps[X] = False` |\n| 1407 | `CTX(self)-\u003etraps = 0;` (in `_decimal_Context_clear_traps_impl`) | `ctx.clear_traps()` |\n| 1421 | `CTX(self)-\u003estatus = 0;` (in `_decimal_Context_clear_flags_impl`) | `ctx.clear_flags()` |\n\n`SdFlags(v)` is `*v-\u003eflags` where `v-\u003eflags` is bound to either `\u0026CTX(ctx)-\u003estatus` or `\u0026CTX(ctx)-\u003etraps` (`_decimal.c:1474–1475`), so the signaldict paths are the same memory as the context-level paths via a different surface.\n\n## Triggering pattern\n\nAny pure-Python code that shares one `Context` instance across free-threaded threads. Five minimal reproducers, one per site, are below. They use a barrier so the racing windows align on the first iteration; under TSan on a free-threaded debug build (`./configure --disable-gil --with-thread-sanitizer`) each one should reliably produce a data race report attributable to the matching site.\n\n```python\n# common.py\nimport threading\nN_THREADS = 8\nITERATIONS = 100_000\ndef run_concurrently(workers):\n    barrier = threading.Barrier(len(workers))\n    threads = [threading.Thread(target=w, args=(barrier,)) for w in workers]\n    for t in threads: t.start()\n    for t in threads: t.join()\n```\n\n### 1. `dec_addstatus` (`:616`)\n\n```python\n# repro_status_or.py\nimport decimal\nfrom common import N_THREADS, ITERATIONS, run_concurrently\nSHARED = decimal.Context(prec=4)  # prec=4 makes \"1.23456\" Inexact|Rounded\ndef worker(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.create_decimal(\"1.23456\")  # -\u003e dec_addstatus(SHARED, ...)\nrun_concurrently([worker] * N_THREADS)\n```\n\n### 2. `clear_flags` race vs. `dec_addstatus` (`:1421` ↔ `:616`)\n\n```python\n# repro_clear_flags.py\nimport decimal\nfrom common import ITERATIONS, run_concurrently\nSHARED = decimal.Context(prec=4)\ndef producer(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.create_decimal(\"1.23456\")\ndef clearer(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.clear_flags()             # ctx-\u003estatus = 0;\nrun_concurrently([producer]*4 + [clearer]*4)\n```\n\n### 3. `clear_traps` race vs. trap-detection read (`:1407` ↔ `:617`)\n\n```python\n# repro_clear_traps.py\nimport decimal\nfrom common import ITERATIONS, run_concurrently\nSHARED = decimal.Context(prec=4, traps=[decimal.Inexact])\ndef producer(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        try:\n            SHARED.create_decimal(\"1.23456\")  # reads ctx-\u003etraps\n        except decimal.Inexact:\n            pass\ndef clearer(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.clear_traps()             # ctx-\u003etraps = 0;\nrun_concurrently([producer]*4 + [clearer]*4)\n```\n\n### 4. `signaldict_setitem` self-race (`:744` / `:747`)\n\n```python\n# repro_signaldict_set.py\nimport decimal\nfrom common import ITERATIONS, run_concurrently\nSHARED = decimal.Context(prec=28)\nA, B = decimal.Inexact, decimal.Rounded  # different bits, same word\ndef setter_a(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.flags[A] = True     # |= bit_a\n        SHARED.flags[A] = False    # \u0026= ~bit_a\ndef setter_b(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.flags[B] = True\n        SHARED.flags[B] = False\nrun_concurrently([setter_a]*4 + [setter_b]*4)\nprint(\"final flags:\", dict(SHARED.flags))   # observable lost-update on FT\n```\n\n(Substituting `SHARED.traps` for `SHARED.flags` produces the same race on `ctx-\u003etraps`.)\n\n### 5. `signaldict_getitem` read vs. `dec_addstatus` write (`:715` ↔ `:616`)\n\n```python\n# repro_signaldict_get.py\nimport decimal\nfrom common import ITERATIONS, run_concurrently\nSHARED = decimal.Context(prec=4)\nINEXACT = decimal.Inexact\ndef producer(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        SHARED.create_decimal(\"1.23456\")\ndef reader(barrier):\n    barrier.wait()\n    for _ in range(ITERATIONS):\n        _ = SHARED.flags[INEXACT]   # reads ctx-\u003estatus non-atomically\nrun_concurrently([producer]*4 + [reader]*4)\n```\n\n## Suggested fix\n\nThe cleanest free-threading-safe option is a per-`Context` `PyMutex` covering all reads and writes of `status` and `traps`. The fields are tiny (one `uint32_t` each) and accessed at very high frequency, so an alternative is to switch to `_Py_atomic_or_uint32` / `_Py_atomic_and_uint32` / `_Py_atomic_load_uint32` / `_Py_atomic_store_uint32` directly on the fields. Given that the trap-detection path needs to read `traps` and `status` together, atomics-only is a little awkward (the OR-then-test sequence wants both observations to be from the same logical state), so `PyMutex` is probably the better fit; the lock-free path can be reserved for the hot read in `signaldict_getitem` if profiling shows the mutex matters.\n\nEither way, the fix should also cover the four `CTX(...)-\u003estatus = 0;` resets at `:1825`, `:1901`, `:1924`, `:1985` (in `current_context_from_dict`, `PyDec_SetCurrentContext`, `init_current_context`, and `PyDec_SetCurrentContext` for the contextvar variant) — those are stores into a `Context` that has just been created or just been swapped in, so they're not strictly racy in the current code, but if any future change exposes them earlier the same atomicity argument applies.\n\n## Related\n\n- #141148 / #146482 — fixes the implicit `getcontext()`/`current_context()` inheritance path. After that PR lands, the primitive sites above are still racy whenever a `Context` is shared explicitly (e.g. `Decimal(value, context=shared_ctx)` or `shared_ctx.create_decimal(s)`); see the issue's own MRE for an explicit-share case that #146482 doesn't cover.\n\n_Drafted by Claude Code, reviewed by a human._\n\n### CPython versions tested on:\n\nCPython main branch, 3.15\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-150598\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/devdanzin","@type":"Person","name":"devdanzin"},"datePublished":"2026-04-29T09:21:55.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/149142/cpython/issues/149142"}