René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`intern_common` should use `_Py_LOCK_DONT_DETACH`","articleBody":"# Bug report\n\n`PyUnicode_InternFromString` and various other interning functions call `intern_common`, which locks an interpreter-wide lock in the free threading build. We should use `_Py_LOCK_DONT_DETACH` when locking that lock to avoid potential deadlocks in certain extensions.\n\nPyTorch, TensorFlow, and possibly other extensions use C++11 one time initialization via function local statics to intern string. For example:\n\n```c++\nvoid PyNode::compiled_args(CompiledNodeArgs& args) const {\n  static PyObject* method_name = PyUnicode_InternFromString(\"_compiled_autograd_key\");\n  ...\n}\n```\n\nhttps://github.com/pytorch/pytorch/blob/8804b122355d1aa1cc6bfffd1516f3df39f96c5f/torch/csrc/autograd/python_function.cpp#L369-L370\n\nC++ static locals implement thread-safe one time initialization, similar to `std::call_once` or `pthread_once`. That means that a (hidden) C++ lock is held during the call to `PyUnicode_InternFromString`. If we detach, during `PyUnicode_InternFromString`, we risk a lock ordering deadlock.\n\nHere's a complicated scenario that could deadlock:\n\n* Thread 1: requests a stop the world pause\n* Thread 2: holds the interned mutex (causes lock contention)\n* Thread 3: calls  `static PyObject* method_name = PyUnicode_InternFromString(...)`. Blocks on interned mutex and **detaches** for stop the world pause\n* Thread 4: calls `static PyObject* method_name = PyUnicode_InternFromString(...)` and blocks on C++ one time initialization lock **without detaching**.\n\nSo there's a deadlock where Thread 4 - (waiting on) -> Thread 3 - (waiting on) -> Thread 1 - (waiting on) -> Thread 4.\n\nThis isn't a problem in GIL-enabled builds because thread 3 won't block during `PyUnicode_InternFromString`.\n\n`_Py_LOCK_DONT_DETACH` prevents this problem because `PyUnicode_InternFromString()` will no longer block on a stop the world if there's contention on the interned mutex.\n\nNote that if callers used something like pybind11's `gil_safe_call_once_and_store` that would also avoid the deadlock.\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-149164\n* gh-153814\n* gh-153815\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/colesbury","@type":"Person","name":"colesbury"},"datePublished":"2026-04-29T18:37:07Z","url":"https://github.com/python/cpython/issues/149162"}