René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"3.15 regression: concurrent first imports can fail or hang without any import cycle","articleBody":"... because parents' module locks stay held while a submodule's body runs\n\n### Bug report\n\n_(bear with me this is verbose, yes mostly model written, i tried to make sure the important info was up top - and i have a proposed fix coming) -gps_\n\n### Description\n\nSince gh-83065 (GH-137196, in 3.15.0b1 and later), `_find_and_load(\"a.leaf\")` also takes the module lock of `a` if `a` isn't fully imported yet, and releases it together with the lock of `a.leaf`, after the body of `a/leaf.py` has run. So a thread whose first import of `a.leaf` also imports `a` keeps the lock of `a` long after `a` is fully initialised, for as long as the body of `a.leaf` runs, including everything that body imports. A second thread that found `a` missing or still initializing waits for that lock... Oops! 😬 \n\nThis causes three failures that 3.14 and earlier don't have. None needs an import cycle, and in one thread the same imports work in either order:\n\n1. `_DeadlockError` reaches user code.\n2. `ImportError: cannot import name 'X' from partially initialized module 'b' (most likely due to a circular import)`, or an `AttributeError` on a partially initialised module. Real packages fail this way (cryptography, protobuf, pyasn1; see below).\n3. The second thread blocks for as long as the submodule's body runs. If that body waits for the second thread by anything other than an import, both hang, and no `_DeadlockError` is raised.\n\n### Short reproducer (failure 1)\n\n```python\nimport os, shutil, sys, tempfile, threading, time\n\nroot = tempfile.mkdtemp()\nsys.path.insert(0, root)\nfor path, text in {\n    \"a/__init__.py\": \"import time; time.sleep(0.2)\",   # any package whose __init__ takes a while\n    \"a/leaf.py\":     \"from b.leaf import X\",\n    \"b/__init__.py\": \"import a\",\n    \"b/leaf.py\":     \"X = 1\",\n}.items():\n    os.makedirs(os.path.dirname(os.path.join(root, path)), exist_ok=True)\n    with open(os.path.join(root, path), \"w\") as f:\n        f.write(text + \"\\n\")\n\ndef thread_1():\n    import a.leaf\n\ndef thread_2():\n    time.sleep(0.1)        # start while a/__init__.py is running\n    import b.leaf\n\nthreads = [threading.Thread(target=thread_1), threading.Thread(target=thread_2)]\nfor t in threads: t.start()\nfor t in threads: t.join()\nshutil.rmtree(root)\nprint(\"a.leaf imported:\", \"a.leaf\" in sys.modules, \"- b.leaf imported:\", \"b.leaf\" in sys.modules)\nsys.exit(\"a.leaf\" not in sys.modules or \"b.leaf\" not in sys.modules)\n```\n\nThe dependencies are `a.leaf` -> `b.leaf` -> `b` -> `a`, and `a` imports nothing, so there is no cycle. On 3.15.0:\n\n```\nException in thread Thread-1 (thread_1):\nTraceback (most recent call last):\n  File \"threading.py\", line 1218, in _bootstrap_inner\n    self._context.run(self.run)\n    ~~~~~~~~~~~~~~~~~^^^^^^^^^^\n  File \"threading.py\", line 1160, in run\n    self._target(*self._args, **self._kwargs)\n    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"repro_mini.py\", line 16, in thread_1\n    import a.leaf\n  File \"a/leaf.py\", line 1, in \u003cmodule>\n    from b.leaf import X\n  File \"\u003cfrozen importlib._bootstrap>\", line 1347, in _find_and_load\n  File \"\u003cfrozen importlib._bootstrap>\", line 458, in __enter__\n  File \"\u003cfrozen importlib._bootstrap>\", line 346, in acquire\n_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('b.leaf') at 140633457717456\na.leaf imported: False - b.leaf imported: True\n```\n\n- **Expected** (as on 3.11 to 3.14): both imports succeed; the script prints `a.leaf imported: True - b.leaf imported: True` and exits with status 0. Thread 2 needs only `a`, and `a` is complete.\n- **Actual** (3.15.0): thread 1 raises `_DeadlockError`; exit status 1.\n\nThe cause shows without threads. With an empty `w/__init__.py` and this `w/main.py`:\n\n```python\nimport importlib._bootstrap as b\nref = b._module_locks.get(\"w\"); lock = ref() if ref else None\nprint(\"lock of w held while w.main's body runs:\", bool(lock and lock.locked()))\n```\n\n`python -c \"import w.main\"` prints `True` on 3.15.0 and `False` on 3.14.7.\n\n#### directional thoughts...\n\n- The change adding `_HierarchicalLockManager` still fixed more than it added so just rolling it back to match 3.14.x and 3.13.16+ behavior doesn't feel right but would be an option.\n- **Workaround:** import the parent in its own statement first (`import a`, then `import a.leaf`; 0 of 20 runs of reproducer 1 fail then), or make the imports on the main thread before threads start.\n\n### What happens, step by step\n\nLine numbers are those of `Lib/importlib/_bootstrap.py` at tag `v3.15.0`.\n\n- `_find_and_load()` chooses `_HierarchicalLockManager(name)` for a dotted name (1342-1343). Its `with` block (1347) spans all of `_find_and_load_unlocked()` (1350), which imports the parent (1277) and then runs the module's own body (1309, `_load_unlocked()`, `exec_module` at 915).\n- `__enter__` (448-477) takes the lock of every ancestor that is missing from `sys.modules` or still initialising, then the leaf's. `__exit__` (479-482) releases them all, only at the end of that `with` block.\n- The parent's own import (1277; for a top-level parent through `_ModuleLockManager`, 1345) takes the parent's lock a second time. When `a/__init__.py` is done, its release only brings the count from 2 to 1, so `release()` wakes nobody (378-383).\n- Other threads wait for that lock because they found `a` missing from `sys.modules` or still initialising: in `_lock_unlock_module()` (524-538, called by `import_ensure_initialized()`, `Python/import.c` 310-337, \"Wait until module is done importing\"), in `_ModuleLockManager.__enter__` (419-421; a plain `import a` that doesn't find `a` in `sys.modules`), in `_HierarchicalLockManager.__enter__` (452-458), or by way of 1291-1292. All they need is that `a` is complete.\n\n\u003cdetails>\n\u003csummary>\u003cb>Lock trace of reproducer 1\u003c/b>\u003c/summary>\n\nEvery `acquire()` and `release()` of the module locks by the two threads of reproducer 1 on 3.15.0, in order (the figure is how many times the thread holds the lock afterwards):\n\n```\nthread 1  wants       a\nthread 1  has (1)     a\nthread 1  wants       a.leaf\nthread 1  has (1)     a.leaf\nthread 1  wants       a\nthread 1  has (2)     a\nthread 2  wants       b\nthread 2  has (1)     b\nthread 2  wants       b.leaf\nthread 2  has (1)     b.leaf\nthread 2  wants       b\nthread 2  has (2)     b\nthread 2  wants       a\nthread 1  gives back (1) a\nthread 1  wants       b\nthread 1  DEADLOCK    b\nthread 1  wants       b.leaf\nthread 1  DEADLOCK    b.leaf\nthread 1  gives back (0) a.leaf\nthread 1  gives back (0) a\nthread 2  has (1)     a\nthread 2  gives back (0) a\nthread 2  gives back (1) b\nthread 2  gives back (0) b.leaf\nthread 2  gives back (0) b\n```\n\nOn 3.14.x the line after `thread 2  wants  a` is `thread 1  gives back (0) a`. Thread 2 goes on and finishes `b` and `b.leaf`; thread 1 waits for `b.leaf` if it isn't finished yet.\n\n\u003c/details>\n\nHow the three failures arise:\n\n1. Thread 1 asks for `b.leaf`, which is not in `sys.modules` yet. `__enter__` gets `_DeadlockError` on `b` and skips it (459-467), gets it again on `b.leaf`, and raises, because that is the leaf (460-461).\n2. If the wanted module is already in `sys.modules`, because thread 2 is inside its body, `import.c` goes through `_lock_unlock_module()`, which swallows the `_DeadlockError` (533-536) and hands out the partially initialised module. `importlib.import_module('b')` in the same place raises `_DeadlockError` instead (20 of 20 runs).\n3. In the hang, thread 1 isn't waiting for a module lock at all, so the deadlock detector has nothing to find.\n\n\"Parent before child\" orders the locks along one dotted name, but gives no order between two chains, (`a`, `a.leaf`) and (`b`, `b.leaf`). Since a chain stays held while module bodies run, a thread can ask for a second chain while holding the first. Before 3.15, a thread inside the body of `a.leaf` held only the locks of `a.leaf` and of the modules further out on its own import stack, all really unfinished, so it could only block a thread that needed one of those. Now it also blocks every thread that needs only the finished `a` and happened to look before `a` was complete.\n\nThis trade-off (ancestors' locks held through the leaf's body) isn't discussed in the class docstring (437-441), the comment at 462-466, the two new tests, the NEWS entry (3.15.0 beta 1: \"so the two threads serialise instead of raising ``_DeadlockError``\") or What's New, so it looks unintended. `test_cross_package_circular_import` (`Lib/test/test_importlib/test_threaded_import.py`, 410-414) says even of two packages that do import each other that concurrent imports of their submodules \"must not raise _DeadlockError\". CPython's own tests don't catch these failures: `python -m test -j2 test_importlib test_import test_threading test_runpy test_pkgutil test_zipimport test_lazy_import` passes on the 3.15.0 build.\n\n\u003cdetails>\n\u003csummary>\u003cb>Reproducer 1: both errors, without sleeps\u003c/b>\u003c/summary>\n\nStandard library only. It writes three or four small files to a temporary folder and first checks that both orders work in one thread. Exit code 1: an import failed; 0: none did.\n\n```python\n\"\"\"Two threads make first imports in two packages.  No import cycle, and either\norder works in one thread, yet thread 1 fails on 3.15 (passes on 3.11 to 3.14).\n\n    python repro_error.py           -> _DeadlockError\n    python repro_error.py partial   -> ImportError: cannot import name 'X' from\n                                       partially initialized module 'b'\n\"\"\"\nimport importlib._bootstrap as bootstrap\nimport os, shutil, sys, tempfile, threading, time, traceback, types\n\nCASES = {\n    # Needs: a.leaf -> b.leaf -> b -> a.  \"a\" imports nothing, so there is no cycle.\n    \"deadlock\": (\"import a.leaf\", \"import b.leaf\", {\n        \"a/__init__.py\": \"import sync; sync.slow_init()\",\n        \"a/leaf.py\":     \"from b.leaf import X\",\n        \"b/__init__.py\": \"import a\",\n        \"b/leaf.py\":     \"X = 1\"}),\n    # Needs: a.leaf -> b -> a.  No cycle either.\n    \"partial\": (\"import a.leaf\", \"import b\", {\n        \"a/__init__.py\": \"import sync; sync.slow_init()\",\n        \"a/leaf.py\":     \"from b import X\",\n        \"b.py\":          \"import a; X = 1\"}),\n}\nstmt1, stmt2, files = CASES[sys.argv[1] if len(sys.argv) > 1 else \"deadlock\"]\n\nsys.dont_write_bytecode = True\nroot = tempfile.mkdtemp()\nsys.path.insert(0, root)\nfor path, text in files.items():\n    os.makedirs(os.path.dirname(os.path.join(root, path)), exist_ok=True)\n    with open(os.path.join(root, path), \"w\") as f:\n        f.write(text + \"\\n\")\n\n# \"sync\" is not a file: it lets a/__init__.py call back into this script.\nsync = sys.modules[\"sync\"] = types.ModuleType(\"sync\")\n\n# Control: one thread, both orders.\nsync.slow_init = lambda: None\nfor order in ((stmt1, stmt2), (stmt2, stmt1)):\n    for stmt in order:\n        exec(stmt)\n    for name in [n for n in sys.modules if n.split(\".\")[0] in (\"a\", \"b\")]:\n        del sys.modules[name]\nprint(\"one thread, both orders: ok\")\n\n# Two threads.  a/__init__.py stands for any package whose __init__ takes a while:\n# it returns once thread 2 has got as far as waiting for the module lock of \"a\".\n# No sleep decides the order; the private _blocking_on is only read, to see that moment.\nin_a = threading.Event()\nfailed = []\n\ndef slow_init():\n    in_a.set()\n    while \"_ModuleLock('a')\" not in repr(bootstrap._blocking_on.get(t2.ident)):\n        time.sleep(0.001)\nsync.slow_init = slow_init\n\ndef run(stmt, start_after=None):\n    if start_after:\n        start_after.wait()\n    try:\n        exec(stmt)\n    except BaseException:\n        failed.append(stmt)\n        print(f\"{threading.current_thread().name}, {stmt!r}:\")\n        traceback.print_exc(file=sys.stdout)\n\nt1 = threading.Thread(target=run, args=(stmt1,), name=\"thread 1\", daemon=True)\nt2 = threading.Thread(target=run, args=(stmt2, in_a), name=\"thread 2\", daemon=True)\nt2.start(); t1.start()\nt1.join(30); t2.join(30)\nif t1.is_alive() or t2.is_alive():\n    failed.append(\"still running after 30 s\")\nshutil.rmtree(root, ignore_errors=True)\nprint(\"two threads:\", f\"FAILED {failed}\" if failed else \"ok\", \"on\", sys.version.split()[0])\nsys.exit(1 if failed else 0)\n```\n\nOn 3.15.0 (on 3.14.7 both print `two threads: ok`):\n\n```\n$ python repro_error.py\none thread, both orders: ok\nthread 1, 'import a.leaf':\nTraceback (most recent call last):\n  File \"repro_error.py\", line 62, in run\n    exec(stmt)\n    ~~~~^^^^^^\n  File \"\u003cstring>\", line 1, in \u003cmodule>\n  File \"a/leaf.py\", line 1, in \u003cmodule>\n    from b.leaf import X\n  File \"\u003cfrozen importlib._bootstrap>\", line 1347, in _find_and_load\n  File \"\u003cfrozen importlib._bootstrap>\", line 458, in __enter__\n  File \"\u003cfrozen importlib._bootstrap>\", line 346, in acquire\n_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('b.leaf') at 139688739099088\ntwo threads: FAILED ['import a.leaf'] on 3.15.0\n$ python repro_error.py partial\none thread, both orders: ok\nthread 1, 'import a.leaf':\nTraceback (most recent call last):\n  File \"repro_error.py\", line 62, in run\n    exec(stmt)\n    ~~~~^^^^^^\n  File \"\u003cstring>\", line 1, in \u003cmodule>\n  File \"a/leaf.py\", line 1, in \u003cmodule>\n    from b import X\nImportError: cannot import name 'X' from partially initialized module 'b' (most likely due to a circular import) (b.py)\ntwo threads: FAILED ['import a.leaf'] on 3.15.0\n```\n\nThe second message blames a circular import that doesn't exist.\n\n\u003c/details>\n\n\u003cdetails>\n\u003csummary>\u003cb>Reproducer 2: the wait and the hang\u003c/b>\u003c/summary>\n\n```python\n\"\"\"Thread 1 makes the first import of w.main.  Thread 2 imports w.other, and starts\nwhile w/__init__.py is still running.  On 3.15 thread 2 then has to wait until the\nBODY of w.main has ended, though \"w\" was complete long before.  Here that body waits\nfor thread 2, so both would hang forever; no _DeadlockError is raised.  The wait is\ncut off after 5 s so that the script ends.  Passes on 3.11 to 3.14.\n\"\"\"\nimport importlib._bootstrap as bootstrap\nimport os, shutil, sys, tempfile, threading, time, traceback, types\n\nfiles = {\n    \"w/__init__.py\": \"import sync; sync.slow_init()\",\n    \"w/main.py\":     \"import sync; sync.wait_for_thread_2()\",\n    \"w/other.py\":    \"\",\n}\nsys.dont_write_bytecode = True\nroot = tempfile.mkdtemp()\nsys.path.insert(0, root)\nos.mkdir(os.path.join(root, \"w\"))\nfor path, text in files.items():\n    with open(os.path.join(root, path), \"w\") as f:\n        f.write(text + \"\\n\")\n\nin_w = threading.Event()\nthread_2_done = threading.Event()\nhung = []\n\ndef slow_init():\n    # Stands for any __init__ that takes a while: returns once thread 2 waits for\n    # the module lock of \"w\" (or is done: before gh-130094 it doesn't wait at all).\n    in_w.set()\n    while not (thread_2_done.is_set() or\n               \"_ModuleLock('w')\" in repr(bootstrap._blocking_on.get(t2.ident))):\n        time.sleep(0.001)\n\ndef wait_for_thread_2():\n    w = sys.modules[\"w\"]\n    print(\"w.main's body runs; w.__spec__._initializing =\", w.__spec__._initializing)\n    if not thread_2_done.wait(5):\n        hung.append(True)\n        print(\"after 5 s thread 2 is still here:\")\n        traceback.print_stack(sys._current_frames()[t2.ident], limit=4, file=sys.stdout)\n\n# \"sync\" is not a file: it lets the modules call back into this script.\nsync = sys.modules[\"sync\"] = types.ModuleType(\"sync\")\nsync.slow_init, sync.wait_for_thread_2 = slow_init, wait_for_thread_2\n\ndef thread_1():\n    import w.main\n\ndef thread_2():\n    in_w.wait()\n    t0 = time.monotonic()\n    import w.other\n    thread_2_done.set()\n    print(f\"thread 2: 'import w.other' took {time.monotonic() - t0:.1f} s\")\n\nt1 = threading.Thread(target=thread_1, daemon=True)\nt2 = threading.Thread(target=thread_2, daemon=True)\nt2.start(); t1.start()\nt1.join(30); t2.join(30)\nshutil.rmtree(root, ignore_errors=True)\nprint(\"HANG (cut off by the script)\" if hung else \"ok\", \"on\", sys.version.split()[0])\nsys.exit(1 if hung else 0)\n```\n\nOn 3.15.0:\n\n```\n$ python repro_hang.py\nw.main's body runs; w.__spec__._initializing = False\nafter 5 s thread 2 is still here:\n  File \"repro_hang.py\", line 53, in thread_2\n    import w.other\n  File \"\u003cfrozen importlib._bootstrap>\", line 1347, in _find_and_load\n  File \"\u003cfrozen importlib._bootstrap>\", line 458, in __enter__\n  File \"\u003cfrozen importlib._bootstrap>\", line 365, in acquire\nthread 2: 'import w.other' took 5.1 s\nHANG (cut off by the script) on 3.15.0\n```\n\nOn 3.14.7 (the first two lines can come in either order):\n\n```\nw.main's body runs; w.__spec__._initializing = False\nthread 2: 'import w.other' took 0.0 s\nok on 3.14.7\n```\n\nThe same happens if thread 2 says `import w` or `from w.other import X` (40 of 40 runs on 3.15.0, 0 of 40 on 3.14.7).\n\nThe plain form of this failure is the wait: 5 s here against 0.0 s, for a module whose parent is complete and which has nothing to do with `w.main`. The hang is the extreme case. A module body that waits for another thread is unusual, but per-module locks were introduced for just that: What's New in 3.3 names \"deadlocks when importing a module would trigger code execution in a different thread as a side-effect\" as the annoyance they remove, and `test_side_effect_import` in `test_threaded_import.py` imports a module whose body starts a thread that imports, and joins it. A realistic shape: `app/__init__.py` starts a background thread that imports `app.something` on first use, and the main thread's first import is `import app.main`, whose body takes a long time. `python -m app.main` is not affected, because runpy imports the parent by itself first.\n\n\u003c/details>\n\n### CPython versions tested on:\n\nCPython main branch, 3.15\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-159965\n* gh-160630\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/gpshead","@type":"Person","name":"gpshead"},"datePublished":"2026-10-20T22:00:37Z","url":"https://github.com/python/cpython/issues/159963"}