René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Parallel import of modules causes a ModuleNotFoundError","articleBody":"# Bug report\n\n### Bug description:\n\nUsing the free-threading build parallel import of modules causes a `ModuleNotFoundError`. The module that cannot be found can differ. A minimal reproducer:\n```python\nimport sys\nfrom threading import Thread, Barrier\nfrom importlib import import_module, reload\nimport time\n\n\n# Setup threads\nt0 = time.perf_counter()\nnumber_of_threads = 8\nbarrier = Barrier(number_of_threads)\n\npmods = []\n\n\ndef work(ii):\n    # Method doing the actual import\n    barrier.wait()\n    while True:\n        try:\n            m = pmods.pop()\n            mod = import_module(m)\n            # print(f'  {ii}: {m} done')\n        except IndexError:\n            return\n\n\nworker_threads = []\nfor ii in range(number_of_threads):\n    worker_threads.append(Thread(target=work, args=[ii]))\n\ndt = time.perf_counter() - t0\nprint(f\"setup threads {dt*1e3:.2f}\")\n\n\ndef parallel_import(modules: list[str]):\n    global pmods\n    pmods += modules\n    for t in worker_threads:\n        t.start()\n    for t in worker_threads:\n        t.join()\n\n\ndef seq_import(modules: list[str]):\n    for m in modules:\n        mod = import_module(m)\n\n\nimport_method = seq_import\nimport_method = parallel_import\n\nmods = [\n    \"abc\",\n    \"functools\",\n    \"weakref\",\n    \"linecache\",\n    \"glob\",\n    \"annotationlib\",\n    \"argparse\",\n    \"fnmatch\",\n    \"itertools\",\n    \"operator\",\n    \"string\",\n    \"re\",\n    \"collections\",\n    \"sqlite3\",\n    \"pathlib\",\n    \"urllib\",\n    \"typing\",\n    \"csv\",\n    \"uuid\",\n]\n\nmods += [\n    \"setuptools\",\n    \"sympy\",\n    \"django\",\n    \"boto3\",\n]  # these are some packages that need to be installed. without these the reproduction is the issue is much harder\n\nt0 = time.perf_counter()\nimport_method(mods)\ndt = time.perf_counter() - t0\nprint(f\"{import_method.__name__} {dt*1e3:.2f}\")\n```\nNote: this script fails on my system about 1 in 4 times. Changing the modules imported (or the order, or the number of threads) can effect this.\n\nModules that have given issues: `collections.abs`, `glob`, `sympy`, `django.utils.regex_helper`. This suggests it is a general issue, and not a particular package.\n\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nWindows\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-130101\n* gh-155915\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/eendebakpt","@type":"Person","name":"eendebakpt"},"datePublished":"2025-02-13T19:49:05Z","url":"https://github.com/python/cpython/issues/130094"}