René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Crash at finalization after fail to start new thread","articleBody":"# Crash report\n\n### What happened?\n\nBisected to e11fc032a75d067d2167a21037722a770b9dfb51, but I guess this issue exists longer, and assertion that is added to `PyThreadState_Clear` by this commit just made it visible.\n```python\nimport resource\nimport threading\n\n# this isn't essential, but helps PyThread_start_new_thread() fail\nresource.setrlimit(resource.RLIMIT_NPROC, (150, 150))\n\nwhile True:\n    t = threading.Thread()\n    t.start()\n    t.join()\n```\n\nError message with backtrace:\n```c++\nTraceback (most recent call last):\n  File \"/home/radislav/projects/cpython/thread_repro.py\", line 10, in \u003cmodule\u003e\n    t.start()\n  File \"/home/radislav/projects/cpython/Lib/threading.py\", line 978, in start\n    _start_new_thread(self._bootstrap, ())\nRuntimeError: can't start new thread\npython: Python/pystate.c:1484: PyThreadState_Clear: Assertion `tstate-\u003e_status.initialized \u0026\u0026 !tstate-\u003e_status.cleared' failed.\n\nProgram received signal SIGABRT, Aborted.\n__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50\n50\t../sysdeps/unix/sysv/linux/raise.c: No such file or directory.\n(gdb) bt\n#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50\n#1  0x00007ffff7c87537 in __GI_abort () at abort.c:79\n#2  0x00007ffff7c8740f in __assert_fail_base (fmt=0x7ffff7dfe688 \"%s%s%s:%u: %s%sAssertion `%s' failed.\\n%n\", \n    assertion=0x5555559c7be0 \"tstate-\u003e_status.initialized \u0026\u0026 !tstate-\u003e_status.cleared\", file=0x5555559c6e2f \"Python/pystate.c\", line=1484, function=\u003coptimized out\u003e)\n    at assert.c:92\n#3  0x00007ffff7c96662 in __GI___assert_fail (assertion=assertion@entry=0x5555559c7be0 \"tstate-\u003e_status.initialized \u0026\u0026 !tstate-\u003e_status.cleared\", \n    file=file@entry=0x5555559c6e2f \"Python/pystate.c\", line=line@entry=1484, function=function@entry=0x5555559c83e0 \u003c__PRETTY_FUNCTION__.47\u003e \"PyThreadState_Clear\")\n    at assert.c:101\n#4  0x000055555587075f in PyThreadState_Clear (tstate=tstate@entry=0x555555c4be00) at Python/pystate.c:1484\n#5  0x0000555555871483 in _PyThreadState_DeleteExcept (tstate=tstate@entry=0x555555c03338 \u003c_PyRuntime+508728\u003e) at Python/pystate.c:1680\n#6  0x000055555586afa4 in Py_FinalizeEx () at Python/pylifecycle.c:1831\n#7  0x000055555589f6fc in Py_RunMain () at Modules/main.c:691\n#8  0x000055555589f74b in pymain_main (args=args@entry=0x7fffffffe160) at Modules/main.c:719\n#9  0x000055555589f7c0 in Py_BytesMain (argc=\u003coptimized out\u003e, argv=\u003coptimized out\u003e) at Modules/main.c:743\n#10 0x00005555555cf74e in main (argc=\u003coptimized out\u003e, argv=\u003coptimized out\u003e) at ./Programs/python.c:15\n```\n \nWhen trying to start a new thread, Python creates new thread state by `_PyThreadState_New` call, adding this new state to list of thread states for current interpreter:\nhttps://github.com/python/cpython/blob/c32abf1f21c4bd32abcefe4d601611b152568961/Modules/_threadmodule.c#L1191\nIf consequent call to `PyThread_start_new_thread` fails, this new state gets cleared, but remains in list:\nhttps://github.com/python/cpython/blob/c32abf1f21c4bd32abcefe4d601611b152568961/Modules/_threadmodule.c#L1203-L1209\nThen, at Python finalization, call to `_PyThreadState_DeleteExcept` attempts to clear this thread state again, which causes assertion failure:\nhttps://github.com/python/cpython/blob/3e8fcb7df74248530c4280915c77e69811f69c3f/Python/pystate.c#L1674-L1682\n\ncc @ericsnowcurrently\n\n### CPython versions tested on:\n\n3.12, CPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n### Output from running 'python -VV' on the command line:\n\nPython 3.13.0a0 (heads/main:d4cea794a7, Sep 22 2023, 18:42:05) [GCC 10.2.1 20210110]\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-109761\n* gh-127171\n* gh-127173\n* gh-127299\n* gh-127323\n* gh-127324\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/chgnrdv","@type":"Person","name":"chgnrdv"},"datePublished":"2023-09-22T15:48:49.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/109746/cpython/issues/109746"}