René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[subinterpreters] crash in _elementtree when parsing in parallel","articleBody":"\u003c!--\r\n  Use this template for hard crashes of the interpreter, segmentation faults, failed C-level assertions, and similar.\r\n  Do not submit this form if you encounter an exception being unexpectedly raised from a Python function.\r\n  Most of the time, these should be filed as bugs, rather than crashes.\r\n\r\n  The CPython interpreter is itself written in a different programming language, C.\r\n  For CPython, a \"crash\" is when Python itself fails, leading to a traceback in the C stack.\r\n--\u003e\r\n\r\n**Crash report**\r\n\r\nI have Kodi crashing when certain add-ons are enabled. I traced the error to  _elementtree and repurposed a script from https://github.com/python/cpython/issues/90228 to reproduce the bug. See [bug.py.txt](https://github.com/python/cpython/files/8599659/bug.py.txt)\r\n\r\n\r\n**Error messages**\r\n\r\n\u003cdetails\u003e\r\n\r\n```\r\n/opt/python-dbg/bin/python3 bug.py \r\nFatal Python error: Segmentation fault\r\n\r\nCurrent thread 0x00007fd85f7fe640 (most recent call first):\r\n  File \"/home/plv/tmp/bug.py\", line 18 in doIt\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 946 in run\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 1009 in _bootstrap_inner\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 966 in _bootstrap\r\n\r\nThread 0x00007fd8654d2640 (most recent call first):\r\n  File \"/home/plv/tmp/bug.py\", line 18 in doIt\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 946 in run\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 1009 in _bootstrap_inner\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 966 in _bootstrap\r\n\r\nThread 0x00007fd865f20740 (most recent call first):\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 1109 in _wait_for_tstate_lock\r\n  File \"/opt/python-dbg/lib/python3.10/threading.py\", line 1089 in join\r\n  File \"/home/plv/tmp/bug.py\", line 25 in func\r\n  File \"/home/plv/tmp/bug.py\", line 27 in \u003cmodule\u003e\r\n\r\nExtension modules: _testcapi (total: 1)\r\nSegmentation fault (core dumped)\r\n\r\n```\r\n\r\n```\r\nReading symbols from /opt/python-dbg/bin/python3.10...\r\n[New LWP 3737917]\r\n[New LWP 3737913]\r\n[New LWP 3737914]\r\n[Thread debugging using libthread_db enabled]\r\nUsing host libthread_db library \"/usr/lib/libthread_db.so.1\".\r\nCore was generated by `/opt/python-dbg/bin/python3 bug.py'.\r\nProgram terminated with signal SIGSEGV, Segmentation fault.\r\n#0  0x00007fd865da234c in __pthread_kill_implementation () from /usr/lib/libc.so.6\r\n[Current thread is 1 (Thread 0x7fd85f7fe640 (LWP 3737917))]\r\n(gdb) bt\r\n#0  0x00007fd865da234c in __pthread_kill_implementation () from /usr/lib/libc.so.6\r\n#1  0x00007fd865d554b8 in raise () from /usr/lib/libc.so.6\r\n#2  \u003csignal handler called\u003e\r\n#3  0x00007fd8640c4c8a in expat_parse (self=0x7fd85efb45f0, data=0x7fd84c0c39e0 \"\u003cdata /\u003e\", data_len=8, final=0)\r\n    at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_elementtree.c:3835\r\n#4  0x00007fd8640c5690 in _elementtree_XMLParser__parse_whole (self=0x7fd85efb45f0, file=\u003coptimized out\u003e)\r\n    at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_elementtree.c:3994\r\n#5  0x00007fd8660bfbdf in method_vectorcall_O (func=func@entry=0x7fd85eef49b0, args=args@entry=0x7fd8642061e0, nargsf=\u003coptimized out\u003e, kwnames=0x0) at Objects/descrobject.c:460\r\n#6  0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e)\r\n    at ./Include/cpython/abstract.h:114\r\n#7  PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#8  call_function (tstate=0x7fd84c01cc00, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fc940, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#9  0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd864206050, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4198\r\n#10 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864206050, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46\r\n#11 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=3, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#12 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e)\r\n    at ./Include/cpython/abstract.h:114\r\n#13 PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#14 call_function (tstate=0x7fd84c01cc00, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fcc10, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#15 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd864124240, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4198\r\n#16 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864124240, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46\r\n#17 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=1, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#18 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e)\r\n    at ./Include/cpython/abstract.h:114\r\n#19 PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#20 call_function (tstate=0x7fd84c01cc00, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fcee0, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#21 0x00007fd8660b12a6 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd864356af0, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4181\r\n#22 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd864356af0, tstate=0x7fd84c01cc00) at ./Include/internal/pycore_ceval.h:46\r\n#23 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=0, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#24 0x00007fd8661269b4 in PyEval_EvalCode (co=0x7fd864240520, globals=0x7fd86422f7d0, locals=\u003coptimized out\u003e) at Python/ceval.c:1134\r\n#25 0x00007fd866140884 in run_eval_code_obj (tstate=0x7fd84c01cc00, co=0x7fd864240520, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0) at Python/pythonrun.c:1291\r\n#26 0x00007fd8661382d6 in run_mod (mod=\u003coptimized out\u003e, filename=\u003coptimized out\u003e, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0, flags=\u003coptimized out\u003e, arena=\u003coptimized out\u003e)\r\n    at Python/pythonrun.c:1312\r\n#27 0x00007fd86612c561 in PyRun_StringFlags (str=\u003coptimized out\u003e, start=257, globals=0x7fd86422f7d0, locals=0x7fd86422f7d0, flags=0x7fd85f7fd220) at Python/pythonrun.c:1183\r\n#28 0x00007fd86612c480 in PyRun_SimpleStringFlags (command=0x7fd8656802b0 \"\\nimport xml.etree.ElementTree as ETree\\nETree.parse(\\\"data.xml\\\")\\n\", flags=flags@entry=0x7fd85f7fd220)\r\n    at Python/pythonrun.c:503\r\n#29 0x00007fd8655e0120 in run_in_subinterp (self=\u003coptimized out\u003e, args=\u003coptimized out\u003e) at /home/plv/Documents/projects/aur/python-dbg/src/Python-3.10.2/Modules/_testcapimodule.c:3639\r\n#30 0x00007fd8660b9438 in cfunction_call (func=0x7fd8656e9c10, args=\u003coptimized out\u003e, kwargs=\u003coptimized out\u003e) at Objects/methodobject.c:552\r\n#31 0x00007fd8660b6468 in _PyObject_MakeTpCall (tstate=0x556284058d10, callable=0x7fd8656e9c10, args=\u003coptimized out\u003e, nargs=\u003coptimized out\u003e, keywords=\u003coptimized out\u003e) at Objects/call.c:215\r\n#32 0x00007fd8660b55d9 in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e)\r\n    at ./Include/cpython/abstract.h:112\r\n#33 PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#34 call_function (tstate=0x556284058d10, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fd3c0, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#35 0x00007fd8660b12a6 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd865563b60, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4181\r\n#36 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd865563b60, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46\r\n#37 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=0, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#38 0x00007fd8660ae50a in PyObject_Call (kwargs=0x7fd864c33650, args=0x7fd865700250, callable=0x7fd8657d4940) at Objects/call.c:317\r\n--Type \u003cRET\u003e for more, q to quit, c to continue without paging--c\r\n#39 do_call_core (kwdict=0x7fd864c33650, callargs=0x7fd865700250, func=0x7fd8657d4940, trace_info=0x7fd85f7fd660, tstate=\u003coptimized out\u003e) at Python/ceval.c:5919\r\n#40 _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd86552abd0, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4277\r\n#41 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd86552abd0, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46\r\n#42 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=1, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#43 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:114\r\n#44 PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#45 call_function (tstate=0x556284058d10, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fd900, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#46 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd84c000ba0, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4198\r\n#47 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd84c000ba0, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46\r\n#48 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=1, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#49 0x00007fd8660b544b in _PyObject_VectorcallTstate (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:114\r\n#50 PyObject_Vectorcall (kwnames=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, args=\u003coptimized out\u003e, callable=\u003coptimized out\u003e) at ./Include/cpython/abstract.h:123\r\n#51 call_function (tstate=0x556284058d10, trace_info=\u003coptimized out\u003e, pp_stack=0x7fd85f7fdbd0, oparg=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5867\r\n#52 0x00007fd8660ab172 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, f=0x7fd86552aa00, throwflag=\u003coptimized out\u003e) at Python/ceval.c:4198\r\n#53 0x00007fd8660a980b in _PyEval_EvalFrame (throwflag=0, f=0x7fd86552aa00, tstate=0x556284058d10) at ./Include/internal/pycore_ceval.h:46\r\n#54 _PyEval_Vector (tstate=\u003coptimized out\u003e, con=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=1, kwnames=\u003coptimized out\u003e) at Python/ceval.c:5065\r\n#55 0x00007fd8660c0e2c in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7fd85f7fddb8, callable=0x7fd86555f540, tstate=0x556284058d10) at ./Include/cpython/abstract.h:114\r\n#56 method_vectorcall (method=\u003coptimized out\u003e, args=\u003coptimized out\u003e, nargsf=\u003coptimized out\u003e, kwnames=\u003coptimized out\u003e) at Objects/classobject.c:61\r\n#57 0x00007fd86618ac9a in thread_run (boot_raw=0x7fd864bf5ee0) at ./Modules/_threadmodule.c:1090\r\n#58 0x00007fd866165f18 in pythread_wrapper (arg=\u003coptimized out\u003e) at Python/thread_pthread.h:248\r\n#59 0x00007fd865da05c2 in start_thread () from /usr/lib/libc.so.6\r\n#60 0x00007fd865e25584 in clone () from /usr/lib/libc.so.6\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\n**Your environment**\r\n\r\n\u003c!-- Include as many relevant details as possible about the environment you experienced the bug in --\u003e\r\n\r\n- CPython versions tested on: `python --with-pydebug 3.10.2`\r\n- Operating system and architecture: `Linux 5.17.1-zen1-1-zen #1 ZEN SMP PREEMPT Mon, 28 Mar 2022 21:56:46 +0000 x86_64 GNU/Linux`\r\n\r\n\u003c!--\r\nYou can freely edit this text. Remove any lines you believe are unnecessary.\r\n--\u003e\r\n\r\n\r\n\u003c!-- gh-linked-prs --\u003e\r\n### Linked PRs\r\n* gh-99221\r\n* gh-101187\r\n* gh-101189\r\n* gh-101190\r\n* gh-101285\n\u003c!-- /gh-linked-prs --\u003e\r\n","author":{"url":"https://github.com/po1vo","@type":"Person","name":"po1vo"},"datePublished":"2022-05-01T20:14:17.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":11},"url":"https://github.com/92123/cpython/issues/92123"}