René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Threads started in exit handler are still running after their thread states are destroyed","articleBody":"Reproduced on main (663c049ff78a299bdf7c1a0444b9900e6d37372d), bisected to 283ab0e1c0.\r\n```python\r\nimport atexit\r\nimport threading\r\n\r\ndef t0():\r\n    pass\r\n\r\ndef t1():\r\n    threading.Thread(target=t0).start()\r\n\r\ndef f():\r\n    threading.Thread(target=t1).start()\r\n\r\natexit.register(f)\r\nexit()\r\n```\r\nOutput (can also pass without crash or segv depending on at which point `t0` tries to access its `tstate`):\r\n```\r\npython: Python/pystate.c:244: bind_tstate: Assertion `tstate_is_alive(tstate) \u0026\u0026 !tstate-\u003e_status.bound' failed.\r\nAborted (core dumped)\r\n```\r\nStack traces:\r\n```\r\n(gdb) b Python/pylifecycle.c:1810\r\nBreakpoint 1 at 0x372529: file Python/pylifecycle.c, line 1828.\r\n(gdb) r repro.py \r\nStarting program: /home/chgnrdv/cpython/python repro.py\r\n[Thread debugging using libthread_db enabled]\r\nUsing host libthread_db library \"/lib/x86_64-linux-gnu/libthread_db.so.1\".\r\n[New Thread 0x7ffff7621700 (LWP 1539803)]\r\n[New Thread 0x7ffff6e20700 (LWP 1539804)]\r\npython: Python/pystate.c:244: bind_tstate: Assertion `tstate_is_alive(tstate) \u0026\u0026 !tstate-\u003e_status.bound' failed.\r\n\r\nThread 3 \"python\" received signal SIGABRT, Aborted.\r\n[Switching to Thread 0x7ffff6e20700 (LWP 1539804)]\r\n__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50\r\n50\t../sysdeps/unix/sysv/linux/raise.c: No such file or directory.\r\n(gdb) bt\r\n#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50\r\n#1  0x00007ffff7c8f537 in __GI_abort () at abort.c:79\r\n#2  0x00007ffff7c8f40f in __assert_fail_base (fmt=0x7ffff7e076a8 \"%s%s%s:%u: %s%sAssertion `%s' failed.\\n%n\", \r\n    assertion=0x555555a1e830 \"tstate_is_alive(tstate) \u0026\u0026 !tstate-\u003e_status.bound\", file=0x555555a1df0f \"Python/pystate.c\", line=244, function=\u003coptimized out\u003e)\r\n    at assert.c:92\r\n#3  0x00007ffff7c9e662 in __GI___assert_fail (assertion=assertion@entry=0x555555a1e830 \"tstate_is_alive(tstate) \u0026\u0026 !tstate-\u003e_status.bound\", \r\n    file=file@entry=0x555555a1df0f \"Python/pystate.c\", line=line@entry=244, function=function@entry=0x555555a1f678 \u003c__PRETTY_FUNCTION__.52\u003e \"bind_tstate\")\r\n    at assert.c:101\r\n#4  0x00005555558c8213 in bind_tstate (tstate=tstate@entry=0x7ffff0000e40) at Python/pystate.c:244\r\n#5  0x00005555558c9d9e in _PyThreadState_Bind (tstate=tstate@entry=0x7ffff0000e40) at Python/pystate.c:1929\r\n#6  0x000055555596afac in thread_run (boot_raw=boot_raw@entry=0x7ffff77e80e0) at ./Modules/_threadmodule.c:1077\r\n#7  0x00005555558e5ce7 in pythread_wrapper (arg=\u003coptimized out\u003e) at Python/thread_pthread.h:233\r\n#8  0x00007ffff7f98ea7 in start_thread (arg=\u003coptimized out\u003e) at pthread_create.c:477\r\n#9  0x00007ffff7d69a2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95\r\n(gdb) frame level 4\r\n#4  0x00005555558c8213 in bind_tstate (tstate=tstate@entry=0x7ffff0000e40) at Python/pystate.c:244\r\n244\t    assert(tstate_is_alive(tstate) \u0026\u0026 !tstate-\u003e_status.bound);\r\n(gdb) print *tstate\r\n$1 = {prev = 0xdddddddddddddddd, next = 0xdddddddddddddddd, interp = 0xdddddddddddddddd, _status = {initialized = 1, bound = 0, unbound = 1, bound_gilstate = 1, \r\n    active = 1, finalizing = 0, cleared = 1, finalized = 1}, py_recursion_remaining = -572662307, py_recursion_limit = -572662307, c_recursion_remaining = -572662307, \r\n  recursion_headroom = -572662307, tracing = -572662307, what_event = -572662307, cframe = 0xdddddddddddddddd, c_profilefunc = 0xdddddddddddddddd, \r\n  c_tracefunc = 0xdddddddddddddddd, c_profileobj = 0xdddddddddddddddd, c_traceobj = 0xdddddddddddddddd, current_exception = 0xdddddddddddddddd, \r\n  exc_info = 0xdddddddddddddddd, dict = 0xdddddddddddddddd, gilstate_counter = -572662307, async_exc = 0xdddddddddddddddd, thread_id = 15987178197214944733, \r\n  native_thread_id = 15987178197214944733, trash = {delete_nesting = -572662307, delete_later = 0xdddddddddddddddd}, on_delete = 0xdddddddddddddddd, \r\n  on_delete_data = 0xdddddddddddddddd, coroutine_origin_tracking_depth = -572662307, async_gen_firstiter = 0xdddddddddddddddd, \r\n  async_gen_finalizer = 0xdddddddddddddddd, context = 0xdddddddddddddddd, context_ver = 15987178197214944733, id = 15987178197214944733, \r\n  datastack_chunk = 0xdddddddddddddddd, datastack_top = 0xdddddddddddddddd, datastack_limit = 0xdddddddddddddddd, exc_state = {exc_value = 0xdddddddddddddddd, \r\n    previous_item = 0xdddddddddddddddd}, root_cframe = {current_frame = 0xdddddddddddddddd, previous = 0xdddddddddddddddd}}\r\n(gdb) info threads\r\n  Id   Target Id                                    Frame \r\n  1    Thread 0x7ffff7c6c280 (LWP 1539799) \"python\" Py_FinalizeEx () at Python/pylifecycle.c:1828\r\n  2    Thread 0x7ffff7621700 (LWP 1539803) \"python\" futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7ffff00010b0)\r\n    at ../sysdeps/nptl/futex-internal.h:323\r\n* 3    Thread 0x7ffff6e20700 (LWP 1539804) \"python\" __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50\r\n(gdb) thread 2\r\n[Switching to thread 2 (Thread 0x7ffff7621700 (LWP 1539803))]\r\n#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7ffff00010b0) at ../sysdeps/nptl/futex-internal.h:323\r\n323\t../sysdeps/nptl/futex-internal.h: No such file or directory.\r\n(gdb) bt\r\n#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7ffff00010b0) at ../sysdeps/nptl/futex-internal.h:323\r\n#1  do_futex_wait (sem=sem@entry=0x7ffff00010b0, abstime=0x0, clockid=0) at sem_waitcommon.c:112\r\n#2  0x00007ffff7fa2278 in __new_sem_wait_slow (sem=sem@entry=0x7ffff00010b0, abstime=0x0, clockid=0) at sem_waitcommon.c:184\r\n#3  0x00007ffff7fa22f1 in __new_sem_wait (sem=sem@entry=0x7ffff00010b0) at sem_wait.c:42\r\n#4  0x00005555558e6120 in PyThread_acquire_lock_timed (lock=lock@entry=0x7ffff00010b0, microseconds=microseconds@entry=-1000000, intr_flag=intr_flag@entry=1)\r\n    at Python/thread_pthread.h:478\r\n#5  0x000055555596a3be in acquire_timed (lock=0x7ffff00010b0, timeout=-1000000000) at ./Modules/_threadmodule.c:98\r\n#6  0x000055555596a50d in lock_PyThread_acquire_lock (self=0x7ffff77d9c70, args=\u003coptimized out\u003e, kwds=\u003coptimized out\u003e) at ./Modules/_threadmodule.c:179\r\n#7  0x000055555570defe in method_vectorcall_VARARGS_KEYWORDS (func=0x7ffff792ef90, args=0x7ffff7fbd378, nargsf=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e)\r\n    at Objects/descrobject.c:365\r\n#8  0x00005555556fc9ca in _PyObject_VectorcallTstate (tstate=0x555555d24ac0, callable=0x7ffff792ef90, args=0x7ffff7fbd378, nargsf=9223372036854775809, kwnames=0x0)\r\n    at ./Include/internal/pycore_call.h:92\r\n#9  0x00005555556fcae5 in PyObject_Vectorcall (callable=callable@entry=0x7ffff792ef90, args=args@entry=0x7ffff7fbd378, nargsf=\u003coptimized out\u003e, \r\n    kwnames=kwnames@entry=0x0) at Objects/call.c:325\r\n#10 0x0000555555856917 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555d24ac0, frame=0x7ffff7fbd300, throwflag=throwflag@entry=0) at Python/bytecodes.c:2643\r\n#11 0x000055555585e296 in _PyEval_EvalFrame (throwflag=0, frame=\u003coptimized out\u003e, tstate=0x555555d24ac0) at ./Include/internal/pycore_ceval.h:87\r\n#12 _PyEval_Vector (tstate=0x555555d24ac0, func=0x7ffff7649fd0, locals=locals@entry=0x0, args=0x7ffff7620db8, argcount=1, kwnames=0x0) at Python/ceval.c:1610\r\n#13 0x00005555556fc45b in _PyFunction_Vectorcall (func=\u003coptimized out\u003e, stack=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Objects/call.c:419\r\n#14 0x0000555555700b6f in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7ffff7620db8, callable=0x7ffff7649fd0, tstate=0x555555d24ac0)\r\n    at ./Include/internal/pycore_call.h:92\r\n#15 method_vectorcall (method=\u003coptimized out\u003e, args=0x555555ca9190 \u003c_PyRuntime+91760\u003e, nargsf=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Objects/classobject.c:67\r\n#16 0x00005555556fecfe in _PyVectorcall_Call (tstate=tstate@entry=0x555555d24ac0, func=0x5555557008f9 \u003cmethod_vectorcall\u003e, callable=callable@entry=0x7ffff77b03b0, \r\n    tuple=tuple@entry=0x555555ca9178 \u003c_PyRuntime+91736\u003e, kwargs=kwargs@entry=0x0) at Objects/call.c:271\r\n#17 0x00005555556ff0aa in _PyObject_Call (tstate=0x555555d24ac0, callable=0x7ffff77b03b0, args=0x555555ca9178 \u003c_PyRuntime+91736\u003e, kwargs=0x0) at Objects/call.c:354\r\n#18 0x00005555556ff103 in PyObject_Call (callable=\u003coptimized out\u003e, args=\u003coptimized out\u003e, kwargs=\u003coptimized out\u003e) at Objects/call.c:379\r\n#19 0x000055555596afdc in thread_run (boot_raw=boot_raw@entry=0x7ffff77ada80) at ./Modules/_threadmodule.c:1081\r\n#20 0x00005555558e5ce7 in pythread_wrapper (arg=\u003coptimized out\u003e) at Python/thread_pthread.h:233\r\n#21 0x00007ffff7f98ea7 in start_thread (arg=\u003coptimized out\u003e) at pthread_create.c:477\r\n#22 0x00007ffff7d69a2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95\r\n(gdb) frame level 19\r\n#19 0x000055555596afdc in thread_run (boot_raw=boot_raw@entry=0x7ffff77ada80) at ./Modules/_threadmodule.c:1081\r\n1081\t    PyObject *res = PyObject_Call(boot-\u003efunc, boot-\u003eargs, boot-\u003ekwargs);\r\n(gdb) print *tstate\r\n$2 = {prev = 0xdddddddddddddddd, next = 0xdddddddddddddddd, interp = 0xdddddddddddddddd, _status = {initialized = 1, bound = 0, unbound = 1, bound_gilstate = 1, \r\n    active = 1, finalizing = 0, cleared = 1, finalized = 1}, py_recursion_remaining = -572662307, py_recursion_limit = -572662307, c_recursion_remaining = -572662307, \r\n  recursion_headroom = -572662307, tracing = -572662307, what_event = -572662307, cframe = 0xdddddddddddddddd, c_profilefunc = 0xdddddddddddddddd, \r\n  c_tracefunc = 0xdddddddddddddddd, c_profileobj = 0xdddddddddddddddd, c_traceobj = 0xdddddddddddddddd, current_exception = 0xdddddddddddddddd, \r\n  exc_info = 0xdddddddddddddddd, dict = 0xdddddddddddddddd, gilstate_counter = -572662307, async_exc = 0xdddddddddddddddd, thread_id = 15987178197214944733, \r\n  native_thread_id = 15987178197214944733, trash = {delete_nesting = -572662307, delete_later = 0xdddddddddddddddd}, on_delete = 0xdddddddddddddddd, \r\n  on_delete_data = 0xdddddddddddddddd, coroutine_origin_tracking_depth = -572662307, async_gen_firstiter = 0xdddddddddddddddd, \r\n  async_gen_finalizer = 0xdddddddddddddddd, context = 0xdddddddddddddddd, context_ver = 15987178197214944733, id = 15987178197214944733, \r\n  datastack_chunk = 0xdddddddddddddddd, datastack_top = 0xdddddddddddddddd, datastack_limit = 0xdddddddddddddddd, exc_state = {exc_value = 0xdddddddddddddddd, \r\n    previous_item = 0xdddddddddddddddd}, root_cframe = {current_frame = 0xdddddddddddddddd, previous = 0xdddddddddddddddd}}\r\n(gdb) thread 1\r\n[Switching to thread 1 (Thread 0x7ffff7c6c280 (LWP 1539799))]\r\n#0  Py_FinalizeEx () at Python/pylifecycle.c:1828\r\n1828\t    if (flush_std_files() \u003c 0) {\r\n(gdb) bt\r\n#0  Py_FinalizeEx () at Python/pylifecycle.c:1828\r\n#1  0x00005555558c6dd4 in Py_Exit (sts=0) at Python/pylifecycle.c:3002\r\n#2  0x00005555558cf99e in handle_system_exit () at Python/pythonrun.c:756\r\n#3  0x00005555558cfcea in _PyErr_PrintEx (tstate=0x555555d07728 \u003c_PyRuntime+478216\u003e, set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:765\r\n#4  0x00005555558d00e3 in PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:845\r\n#5  0x00005555558d00f3 in PyErr_Print () at Python/pythonrun.c:851\r\n#6  0x00005555558d06bb in _PyRun_SimpleFileObject (fp=fp@entry=0x555555d35500, filename=filename@entry=0x7ffff775ac70, closeit=closeit@entry=1, \r\n    flags=flags@entry=0x7fffffffdfa8) at Python/pythonrun.c:439\r\n#7  0x00005555558d08bd in _PyRun_AnyFileObject (fp=fp@entry=0x555555d35500, filename=filename@entry=0x7ffff775ac70, closeit=closeit@entry=1, \r\n    flags=flags@entry=0x7fffffffdfa8) at Python/pythonrun.c:78\r\n#8  0x00005555558f80d7 in pymain_run_file_obj (program_name=program_name@entry=0x7ffff77a95b0, filename=filename@entry=0x7ffff775ac70, skip_source_first_line=0)\r\n    at Modules/main.c:360\r\n#9  0x00005555558f83b9 in pymain_run_file (config=config@entry=0x555555ce9a60 \u003c_PyRuntime+356160\u003e) at Modules/main.c:379\r\n#10 0x00005555558f9480 in pymain_run_python (exitcode=exitcode@entry=0x7fffffffe11c) at Modules/main.c:610\r\n#11 0x00005555558f94d8 in Py_RunMain () at Modules/main.c:689\r\n#12 0x00005555558f952c in pymain_main (args=args@entry=0x7fffffffe160) at Modules/main.c:719\r\n#13 0x00005555558f95a1 in Py_BytesMain (argc=\u003coptimized out\u003e, argv=\u003coptimized out\u003e) at Modules/main.c:743\r\n#14 0x000055555565273e in main (argc=\u003coptimized out\u003e, argv=\u003coptimized out\u003e) at ./Programs/python.c:15\r\n```\r\n\r\nLooks like some race condition between main thread and threads started and still running at finalization, whose `tstate` main thread is trying to delete, although I'm not an expert and didn't make further debugging.\r\n\r\nPlatform: `Linux 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux`\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-104826\n* gh-105277\n* gh-109056\n* gh-109133\n* gh-109134\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/chgnrdv","@type":"Person","name":"chgnrdv"},"datePublished":"2023-05-20T14:11:27.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":9},"url":"https://github.com/104690/cpython/issues/104690"}