René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Concurrent imports of a package and its submodule raise _DeadlockError in 3.14 and since 3.13.16","articleBody":"\n### Bug description:\n\nSince the fix for gh-130094 (GH-130101, backported to 3.13 in GH-155915 and first released in 3.13.16), `_find_and_load_unlocked()` re-imports a parent package whose spec is still `_initializing`, and it does so while holding the child module's lock. That creates a lock-order inversion between two common imports that worked before:\n\n- Thread A runs `import pkg`. It holds `pkg`'s module lock while `pkg/__init__.py` runs, and that file does `from .sub import X`, so A then needs the lock for `pkg.sub`.\n- Thread B runs `import pkg.sub` while `pkg` is still initializing. It takes the `pkg.sub` lock, sees `pkg.__spec__._initializing`, and now waits for `pkg`'s lock.\n\nimportlib detects the cycle and raises `_DeadlockError` in thread A. Up to 3.13.15, thread B loaded `pkg.sub` without waiting for the parent, so both imports succeeded. There is no import cycle in user code, and the same imports work in either order on a single thread.\n\nReproducer (fails every time on 3.13.16 and 3.14.4; passes on 3.13.13):\n\n```python\nimport os, sys, tempfile, threading, time\n\nroot = tempfile.mkdtemp()\nos.makedirs(os.path.join(root, \"pkg\"))\nwith open(os.path.join(root, \"pkg\", \"__init__.py\"), \"w\") as f:\n    f.write(\"import time\\ntime.sleep(0.5)\\nfrom .sub import X\\n\")\nwith open(os.path.join(root, \"pkg\", \"sub.py\"), \"w\") as f:\n    f.write(\"X = 1\\n\")\nsys.path.insert(0, root)\n\nerrors = []\n\ndef run(name):\n    try:\n        __import__(name)\n    except BaseException as e:\n        errors.append(f\"{name}: {type(e).__name__}: {e}\")\n\na = threading.Thread(target=run, args=(\"pkg\",))\nb = threading.Thread(target=run, args=(\"pkg.sub\",))\na.start()\ntime.sleep(0.1)  # pkg is now initializing; thread a holds its module lock\nb.start()\na.join(); b.join()\nprint(sys.version.split()[0], \"->\", errors or \"OK\")\n```\n\n```\n3.13.13 -> OK\n3.13.16 -> [\"pkg: _DeadlockError: deadlock detected by _ModuleLock('pkg.sub') at 0x...\"]\n3.14.4  -> [\"pkg: _DeadlockError: deadlock detected by _ModuleLock('pkg.sub') at 0x...\"]\n```\n\nRemoving only the new block below from 3.13.16's `_find_and_load_unlocked()` makes the reproducer pass. The `meta_path = list(meta_path)` part of the same change has no effect on it:\n\n```python\n        if getattr(parent_spec, '_initializing', False):\n            _call_with_frames_removed(import_, parent)\n        # Crazy side-effects (again)!\n        module = sys.modules.get(name)\n        if module is not None:\n            return module\n```\n\nReal-world impact: any package whose `__init__` imports its own submodules is affected (`requests`, `prometheus_client`, and so on) as soon as two threads import it and one of its submodules for the first time at once. The Datadog Agent loads Python checks from several threads and hit this right after moving to 3.13.16. Checks failed at startup with `_DeadlockError: deadlock detected by _ModuleLock('requests.exceptions')`, so we went back to 3.13.15 (DataDog/datadog-agent#57902). Other projects report the same error after upgrading to 3.13.16 or 3.14, for example gke-labs/kube-agents#2446, dgunning/edgartools#1325 and marimo-team/marimo#11095.\n\nBecause this is a regression in a bugfix release, could the 3.13 backport be reverted, or this part changed so the thread doesn't wait for the parent's lock while holding the child's? The same code is on 3.14 and `main`.\n\ngh-159963 is related but separate: it covers a 3.15-only change from gh-83065. This report is about the gh-130094 change that is already in 3.13.16 and 3.14.\n\n### CPython versions tested on:\n\n3.13, 3.14\n\n### Operating systems tested on:\n\nmacOS\n","author":{"url":"https://github.com/Kyle-Neale","@type":"Person","name":"Kyle-Neale"},"datePublished":"2026-10-21T15:31:11Z","url":"https://github.com/python/cpython/issues/160045"}