René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"shutdown_default_executor / wait_for_tstate_lock deadlock (?) after Ctrl+C","articleBody":"# Bug report\n\n### Bug description:\n\nBlocking function running in thread executor will produce deadlock on shutdown after Ctrl+C pressed.\n\nHere is minimal reproducing **example**:\n```python\nimport asyncio\nimport time\n\ndef blocking():\n    print(\"sleep start\")\n    time.sleep(1000)\n    print(\"sleep end\")\n\nasync def main():\n    await asyncio.to_thread(blocking)\n\nasyncio.run(main())\n```\n\n**Steps to reproduce**:\n1. Run script\n2. Try press Ctrl+C when blocking func sleeps | Nothing produced\n3. If press Ctrl+C again | Will print KeyboardInterrupt exception but script will still hang\n4. If press Ctrl+C again | Will print one more KeyboardInterrupt and will shutdown.\n\n(Tested on 3.9\u003c=python\u003c=3.12 on Ubuntu 22.04 all have same behaviour)\n\n\n**Output**:\n```shell\n$ python3.11 test.py \nsleep start\n^C^CTraceback (most recent call last):\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 118, in run\n    return self._loop.run_until_complete(task)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 653, in run_until_complete\n    return future.result()\n           ^^^^^^^^^^^^^^^\n  File \"/home/ilya/test.py\", line 13, in main\n    await asyncio.to_thread(blocking)\n  File \"/usr/lib/python3.11/asyncio/threads.py\", line 25, in to_thread\n    return await loop.run_in_executor(None, func_call)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nasyncio.exceptions.CancelledError\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 190, in run\n    return runner.run(main)\n           ^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 123, in run\n    raise KeyboardInterrupt()\nKeyboardInterrupt\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/home/ilya/test.py\", line 15, in \u003cmodule\u003e\n    asyncio.run(main())\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 189, in run\n    with Runner(debug=debug) as runner:\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 63, in __exit__\n    self.close()\n  File \"/usr/lib/python3.11/asyncio/runners.py\", line 73, in close\n    loop.run_until_complete(loop.shutdown_default_executor())\n  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 640, in run_until_complete\n    self.run_forever()\n  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 607, in run_forever\n    self._run_once()\n  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 1884, in _run_once\n    event_list = self._selector.select(timeout)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.11/selectors.py\", line 468, in select\n    fd_event_list = self._selector.poll(timeout, max_ev)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nKeyboardInterrupt\n^CException ignored in: \u003cmodule 'threading' from '/usr/lib/python3.11/threading.py'\u003e\nTraceback (most recent call last):\n  File \"/usr/lib/python3.11/threading.py\", line 1560, in _shutdown\n    atexit_call()\n  File \"/usr/lib/python3.11/concurrent/futures/thread.py\", line 31, in _python_exit\n    t.join()\n  File \"/usr/lib/python3.11/threading.py\", line 1119, in join\n    self._wait_for_tstate_lock()\n  File \"/usr/lib/python3.11/threading.py\", line 1139, in _wait_for_tstate_lock\n    if lock.acquire(block, timeout):\n       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nKeyboardInterrupt: \n```\n\n**Additional**:\nFound a blog post https://www.roguelynn.com/words/asyncio-sync-and-threaded/ with looks like similar behavior back in 2019 where someone is trying to workaround this.\n\n### CPython versions tested on:\n\n3.9, 3.10, 3.11, 3.12\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-115622\n* gh-115641\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/xbeastx","@type":"Person","name":"xbeastx"},"datePublished":"2023-10-26T12:37:16.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/111358/cpython/issues/111358"}