René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`multiprocessing.Process.is_alive()` can incorrectly return True after `join()`","articleBody":"# Bug report\n\n### Bug description:\n\nThis came up in https://github.com/python/cpython/pull/130849#discussion_r1980137089\n\nThe problem is that `popen_fork.Popen` (and `popen_spawn.Popen` and `popen_forkserver.Popen`) are not thread-safe:\n\nhttps://github.com/python/cpython/blob/02de9cb9a8fa5b0ae3947231b8c0677834aaee45/Lib/multiprocessing/popen_fork.py#L25-L35\n\nThe first successful call to `os.waitpid()` may reap the pid so that subsequent calls raise an `OSError`. I've only seen this on macOS (not Linux). We may not yet however have set `self.returncode` -- that happens a few statements later, so `poll()` can return `None` if:\n\n1) The process has finished\n2) Another thread called poll(), but hasn't yet set `self.returncode`\n\nAnd then `is_alive()` can return True:\n\nhttps://github.com/python/cpython/blob/02de9cb9a8fa5b0ae3947231b8c0677834aaee45/Lib/multiprocessing/process.py#L153-L170\n\nNote that some classes like `concurrent.futures.ProcessPoolExecutor` use threads internally, so the user may not even know that threads are involved.\n\nRepro:\n\n\u003cdetails\u003e\n\u003csummary\u003erepro.py\u003c/summary\u003e\n\n```python\nimport os\nimport multiprocessing as mp\nimport threading\nimport time\nimport sys\n\noriginal_excepthook = threading.excepthook\n\ndef on_except(args):\n    original_excepthook(args)\n    os._exit(1)\n\nthreading.excepthook = on_except\n\ndef p1():\n    pass\n\ndef thread1(p):\n    while p.is_alive():\n        time.sleep(0.00001)\n        pass\n\ndef test():\n    for i in range(1000):\n        print(i)\n        p = mp.Process(target=p1)\n        p.start()\n\n        t = threading.Thread(target=thread1, args=(p,))\n        t.start()\n\n        p.join()\n        assert not p.is_alive()\n\n        t.join()\n\ndef main():\n    threads = [threading.Thread(target=test) for _ in range(10)]\n    for t in threads:\n        t.start()\n    for t in threads:\n        t.join()\n\nif __name__ == \"__main__\":\n    main()\n```\n\u003c/details\u003e\n\n**NOTE**:\n* This is unrelated to free threading\n* `popen_fork.Popen` (and subclasses) are distinct from `subprocess.Popen`\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nmacOS\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-131440\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/colesbury","@type":"Person","name":"colesbury"},"datePublished":"2025-03-05T21:09:49.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/130895/cpython/issues/130895"}