René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"TSAN warnings in dictobject.c","articleBody":"A [LazyImports test](https://github.com/LazyImportsCabal/cpython/blob/31b7fe99cac75ca887b38192663918e76f1733d4/Lib/test/test_import/test_lazy_imports.py#L1439-L1472) is generating TSAN warnings that look something like:\n\n```\nWARNING: ThreadSanitizer: data race (pid=453778)\n  Read of size 8 at 0x567fc2060568 by thread T3:\n    #0 _Py_TYPE_impl /home/sgross/cpython/./Include/object.h:313:16 (python+0x3265a1) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n    #1 _Py_IS_TYPE_impl /home/sgross/cpython/./Include/object.h:328:12 (python+0x3265a1)\n    #2 compare_unicode_unicode_threadsafe /home/sgross/cpython/Objects/dictobject.c:1424:13 (python+0x3265a1)\n    #3 do_lookup /home/sgross/cpython/Objects/dictobject.c:1009:23 (python+0x325b48) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n    #4 unicodekeys_lookup_unicode_threadsafe /home/sgross/cpython/Objects/dictobject.c:1445:12 (python+0x30d85f) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n\n...\n\n   Previous write of size 8 at 0x567fc2060568 by thread T2:\n    #0 __tsan_memset \u003cnull\u003e (python+0xf5c91) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n    #1 fill_mem_debug /home/sgross/cpython/Objects/obmalloc.c (python+0x37b9f9) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n    #2 _PyMem_DebugRawAlloc /home/sgross/cpython/Objects/obmalloc.c:2904:9 (python+0x37b9f9)\n    #3 _PyMem_DebugRawMalloc /home/sgross/cpython/Objects/obmalloc.c:2920:12 (python+0x37b9f9)\n    #4 _PyMem_DebugMalloc /home/sgross/cpython/Objects/obmalloc.c:3085:12 (python+0x37b9f9)\n    #5 PyObject_Malloc /home/sgross/cpython/Objects/obmalloc.c:1493:12 (python+0x37a41d) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n    #6 PyUnicode_New /home/sgross/cpython/Objects/unicodeobject.c:1320:24 (python+0x3fd681) (BuildId: 149c3950b350c299f7b543875dac3ce12f85640f)\n...\n```\n\nThe problem is that the reader is using \"relaxed\" memory ordering. (The writer is using \"release\", which is good).\n\nhttps://github.com/python/cpython/blob/46295677a13f141b8d52cc36202384554adba65d/Objects/dictobject.c#L1416-L1417\n\nThis is a case where the C11 memory model isn't a good fit for actual hardware. The C11 memory model requires at least \"consume\", but compilers treat \"consume\" like the stronger \"acquire\", which emits stronger than necessary fences on aarch64 [^1]. \"consume\" should be \"free\" on aarch64, but \"acquire\" requires a load with a fence like `LDAR/LDAPR`.\n\nWe have a few options:\n\n1) Keep using \"relaxed\" and sometimes trigger TSAN warnings -- not great\n2) Use \"acquire\" and suffer a potential performance hit on aarch64\n3) Add fake \"consume\" bindings in pyatomic.h that are \"acquire\" under TSAN and \"relaxed\" otherwise\n\nI think (3) is the best option, but I'll benchmark (2). \n\n### FAQ\n\n\u003e Would #3 mean that we're not doing the right thing on aarch64 though?\n\nNo, we'd be doing the right thing. A plain load is sufficient on aarch64 (and x86-64, POWER, armv7, SPARC, etc.) and pretty much every mainstream CPU in existence, except (famously) the DEC Alpha, which hasn't been manufactured in two decades. CPUs (except the DEC Alpha) respect address dependencies -- they won't reorder dependent loads.\n\n### References\n\n* [P2055R0: A Relaxed Guide to memory_order_relaxed](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2055r0.pdf) - see 2.6.7\n* https://docs.kernel.org/core-api/wrappers/memory-barriers.html - see `READ_ONCE`, which is the Linux kernel counterpart to `memory_order_relaxed`\n* [Is Parallel Programming Hard, And, If So, What Can You Do About It?](https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html) - see 15.3.4 Address Dependencies\n* [P0190R4: Proposal for New memory order consume Definition](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0190r4.pdf)\n\n[^1]: There's no performance issue when using \"acquire\" on x86-64 because plain loads have \"acquire\" semantics so \"acquire\" is free.\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-142544\n* gh-142603\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/colesbury","@type":"Person","name":"colesbury"},"datePublished":"2025-12-10T20:05:24.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/142534/cpython/issues/142534"}