René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Asyncio: fork() in coroutine causes bad file descriptor","articleBody":"# Bug report\n\n### Bug description:\n\nHere is a simple repro:\n```python\nimport asyncio\nimport os\n\nasync def main():\n    pid = os.fork()\n    if pid:\n        os.waitpid(pid, 0)\n\nasyncio.run(main())\n```\n\nThe traceback looks like:\n```\nTraceback (most recent call last):\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py\", line 190, in run\n    return runner.run(main)\n           ^^^^^^^^^^^^^^^^\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py\", line 118, in run\n    return self._loop.run_until_complete(task)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 641, in run_until_complete\n    self.run_forever()\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 608, in run_forever\n    self._run_once()\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 1898, in _run_once\n    event_list = self._selector.select(timeout)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py\", line 566, in select\n    kev_list = self._selector.control(None, max_ev, timeout)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nOSError: [Errno 9] Bad file descriptor\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"~/forkasyncio.py\", line 9, in \u003cmodule\u003e\n    asyncio.run(main())\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py\", line 189, in run\n    with Runner(debug=debug) as runner:\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py\", line 63, in __exit__\n    self.close()\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py\", line 72, in close\n    loop.run_until_complete(loop.shutdown_asyncgens())\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 641, in run_until_complete\n    self.run_forever()\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 608, in run_forever\n    self._run_once()\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py\", line 1898, in _run_once\n    event_list = self._selector.select(timeout)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py\", line 566, in select\n    kev_list = self._selector.control(None, max_ev, timeout)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nOSError: [Errno 9] Bad file descriptor\nsys:1: RuntimeWarning: coroutine 'BaseEventLoop.shutdown_asyncgens' was never awaited\n```\n\nI believe the traceback is produced by the child process.\n\nI would not expect bad file descriptor in either the parent or the child because fork is supposed to copy the open file descriptors. I have even tried a custom fork module to ensure `os.fork` isn't closing the file descriptor.\n\nThe issue also happens if the child does `sys.exit()`. I find the best workaround is to have the child do `os._exit(0)`:\n\n```python\nimport asyncio\nimport os\nimport sys\n\nasync def main():\n    pid = os.fork()\n    if pid:\n        os.waitpid(pid, 0)\n    else:\n        sys.stdout.flush()\n        sys.stderr.flush()\n        os._exit(0)\n\nasyncio.run(main())\n```\n\nThis produces no traceback. I also avoid doing anything asyncio in the child.\n\nI'm mainly curious about what file descriptor is bad; it doesn't seem possible.\n\nIt sounds like this issue could be fixed by https://github.com/python/cpython/pull/99539, but I still reproduce the issue in 3.12.7 and 3.13.0, though with a slightly different exception:\n```\nTraceback (most recent call last):\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 194, in run\n    return runner.run(main)\n           ^^^^^^^^^^^^^^^^\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 118, in run\n    return self._loop.run_until_complete(task)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 674, in run_until_complete\n    self.run_forever()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 641, in run_forever\n    self._run_once()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 1948, in _run_once\n    event_list = self._selector.select(timeout)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py\", line 566, in select\n    kev_list = self._selector.control(None, max_ev, timeout)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nValueError: I/O operation on closed kqueue object\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 71, in close\n    loop.run_until_complete(loop.shutdown_asyncgens())\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 674, in run_until_complete\n    self.run_forever()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 641, in run_forever\n    self._run_once()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py\", line 1948, in _run_once\n    event_list = self._selector.select(timeout)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py\", line 566, in select\n    kev_list = self._selector.control(None, max_ev, timeout)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nValueError: I/O operation on closed kqueue object\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"~/forkasyncio.py\", line 10, in \u003cmodule\u003e\n    asyncio.run(main())\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 193, in run\n    with Runner(debug=debug, loop_factory=loop_factory) as runner:\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 62, in __exit__\n    self.close()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py\", line 77, in close\n    loop.close()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/unix_events.py\", line 68, in close\n    super().close()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py\", line 104, in close\n    self._close_self_pipe()\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py\", line 111, in _close_self_pipe\n    self._remove_reader(self._ssock.fileno())\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py\", line 305, in _remove_reader\n    self._selector.unregister(fd)\n  File \"/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py\", line 542, in unregister\n    self._selector.control([kev], 0, 0)\nValueError: I/O operation on closed kqueue object\n/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py:712: RuntimeWarning: coroutine 'BaseEventLoop.shutdown_asyncgens' was never awaited\n```\n\n\n\n### CPython versions tested on:\n\n3.11\n\n### Operating systems tested on:\n\nmacOS","author":{"url":"https://github.com/wjmelements","@type":"Person","name":"wjmelements"},"datePublished":"2025-02-22T02:10:19.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/130442/cpython/issues/130442"}