René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Data race in `intern_common` when interning str objects in the free threading build","articleBody":"# Bug report\n\nWhen running `./python -m test test_exceptions --parallel-threads=10` in a TSAN build:\n\n```\nWARNING: ThreadSanitizer: data race (pid=763025)\n  Atomic read of size 4 at 0x7fffbe0718cc by thread T190:\n    #0 _Py_atomic_load_uint32_relaxed Include/cpython/pyatomic_gcc.h:367 (python+0x1d386e) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #1 Py_INCREF Include/refcount.h:261 (python+0x1d386e)\n    #2 _Py_NewRef Include/refcount.h:518 (python+0x1d386e)\n    #3 dict_setdefault_ref_lock_held Objects/dictobject.c:4386 (python+0x1e6817) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #4 PyDict_SetDefaultRef Objects/dictobject.c:4403 (python+0x1e6a37) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #5 intern_common Objects/unicodeobject.c:15820 (python+0x2a7a8d) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #6 _PyUnicode_InternImmortal Objects/unicodeobject.c:15874 (python+0x2e92d1) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #7 _PyPegen_new_identifier Parser/pegen.c:549 (python+0xad61f) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n....\n\n  Previous write of size 4 at 0x7fffbe0718cc by thread T182:\n    #0 Py_SET_REFCNT Include/refcount.h:176 (python+0x2a7c9a) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #1 Py_SET_REFCNT Include/refcount.h:145 (python+0x2a7c9a)\n    #2 intern_common Objects/unicodeobject.c:15848 (python+0x2a7c9a)\n    #3 _PyUnicode_InternImmortal Objects/unicodeobject.c:15874 (python+0x2e92d1) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n    #4 _PyPegen_new_identifier Parser/pegen.c:549 (python+0xad61f) (BuildId: 5612db6eff0f51c7fd99ee4409b2ceafceea484c)\n...\n```\n\nThere are a few thread-safety issues with `intern_common`:\n\n* It can return a string that's not marked as interned because we insert into `interned` before we mark the string as interned. This can be fixed with some additional locking.\n* The `Py_SET_REFCNT(s, Py_REFCNT(s) - 2)` modification is not thread-safe with respect to other reference count modifications in the free threading build\n* `_Py_SetImmortal` is not thread-safe in some circumstances (see https://github.com/python/cpython/issues/113956)\n\nThe `_Py_SetImmortal()` issue is tricky and I think it's unlikely to cause problems in practice, so I think we can defer dealing with that for now.\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-130089\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/colesbury","@type":"Person","name":"colesbury"},"datePublished":"2025-02-05T20:42:11.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/129701/cpython/issues/129701"}