René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Crash Due to Exception in threading._shutdown()","articleBody":"One of the first thing that happens during interpreter finalization is waiting for all non-daemon threads to finish.  This is implemented by calling `threading._shutdown()`.  If an exception is raised there (e.g. in a `threading` \"atexit\" function), it gets ignored and we end up with non-daemon threads still running.  In subinterpreters that results in a fatal error almost immediately after, since we make sure there's only one thread state left at that point.\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eReproducer:\u003c/summary\u003e\r\n\r\n```\r\n$ cat \u003e /tmp/crash-interp-lingering-thread.py \u003c\u003c EOF\r\nimport threading\r\nfrom test.support import interpreters\r\n\r\ninterp = interpreters.create()\r\ninterp.exec_sync(f\"\"\"if True:\r\n    import threading\r\n    import time\r\n\r\n    done = False\r\n\r\n    def notify_fini():\r\n        global done\r\n        done = True\r\n        raise Exception  # \u003c-------\r\n        t.join()\r\n    threading._register_atexit(notify_fini)\r\n\r\n    def task():\r\n        while not done:\r\n            time.sleep(0.1)\r\n    t = threading.Thread(target=task)\r\n    t.start()\r\n\"\"\")\r\ninterp.close()\r\n\r\n\r\nEOF\r\n$ ./python /tmp/crash-interp-lingering-thread.py\r\nException ignored on threading shutdown:\r\nTraceback (most recent call last):\r\n  File \"./cpython/Lib/threading.py\", line 1623, in _shutdown\r\n    atexit_call()\r\n  File \"\u003cstring\u003e\", line 10, in notify_fini\r\nException:\r\nFatal Python error: Py_EndInterpreter: not the last thread\r\nPython runtime state: initialized\r\n\r\nCurrent thread 0x00007fc8bf76a100 (most recent call first):\r\n  \u003cno Python frame\u003e\r\n\r\nThread 0x00007fc8bd3d5700 (most recent call first):\r\n  File \"\u003cstring\u003e\", line 16 in task\r\n  File \"./cpython/Lib/threading.py\", line 1020 in run\r\n  File \"./cpython/Lib/threading.py\", line 1083 in _bootstrap_inner\r\n  File \"./cpython/Lib/threading.py\", line 1040 in _bootstrap\r\nAborted (core dumped)\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\n----\r\n\r\nThe solution?  Make sure `threading._shutdown()` finishes its fundamental job: stop all non-daemon threads and wait for them to finish.  At the very least, this means ignoring exceptions from functions registered with `threading._register_atexit()`.\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-130594\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/ericsnowcurrently","@type":"Person","name":"ericsnowcurrently"},"datePublished":"2023-12-14T21:45:55.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/113148/cpython/issues/113148"}