René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Heap-use-after-free issue in `pyexpat` related to `.ExternalEntityParserCreate`","articleBody":"# Bug report\n\n### Bug description:\n\nWhen configured with `--with-address-sanitizer`, the test below crashes all versions of CPython:\n\n```python\n# Copyright (c) 2025 Sebastian Pipping \u003csebastian@pipping.org\u003e\n# Licensed under Zero-Clause BSD (\"0BSD\")\n\nimport pyexpat as expat\nimport unittest\n\n\nclass ParentParserLifetimeTest(unittest.TestCase):\n    \"\"\"\n    Subparsers make use of their parent XML_Parser inside of Expat.\n    As a result, parent parsers need to outlive subparsers.\n    Regression test for issue 139400\n    \"\"\"\n    def test_parent_parser_outlives_its_subparsers(self):\n        parser = expat.ParserCreate()\n        subparser = parser.ExternalEntityParserCreate(None)\n\n        # Now try to cause garbage collection of the parent parser\n        # while it's still being referenced by a related subparser\n        del parser\n\n\nif __name__ == '__main__':\n    unittest.main()\n```\n\nThe finding was first documented at https://github.com/python/cpython/pull/139367#issuecomment-3343919109 .\n\n\u003cdetails\u003e\n\u003csummary\u003eFor 3.13, the AddressSanitizer crash details are: (click to expand)\u003c/summary\u003e\n\n```console\n# ./python ..test_file_with_test_class_above_added_dot_py.. -v\ntest_use_after_free__crash (__main__.UseAfterFreeCrashDemoTest.test_use_after_free__crash) ... =================================================================\n==16187==ERROR: AddressSanitizer: heap-use-after-free on address 0x7cda1ad7e038 at pc 0x7f4a1af8a94f bp 0x7ffc61f63720 sp 0x7ffc61f63710\nREAD of size 8 at 0x7cda1ad7e038 thread T0\n    #0 0x7f4a1af8a94e in getRootParserOf Modules/expat/xmlparse.c:8660\n    #1 0x7f4a1af8a94e in expat_free Modules/expat/xmlparse.c:913\n    #2 0x7f4a1af8a94e in expat_free Modules/expat/xmlparse.c:906\n    #3 0x7f4a1af8a94e in PyExpat_XML_ParserFree Modules/expat/xmlparse.c:1997\n    #4 0x7f4a1af70286 in xmlparse_dealloc Modules/pyexpat.c:1266\n    #5 0x558a55f60555 in Py_DECREF Include/object.h:949\n    #6 0x558a55f60555 in Py_XDECREF Include/object.h:1042\n    #7 0x558a55f60555 in _PyFrame_ClearLocals Python/frame.c:104\n    #8 0x558a55f60555 in _PyFrame_ClearExceptCode Python/frame.c:129\n    #9 0x558a55e9df91 in clear_thread_frame Python/ceval.c:1682\n    #10 0x558a55e9df91 in _PyEval_FrameClearAndPop Python/ceval.c:1709\n    #11 0x558a55ec6b44 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:5222\n    #12 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #13 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #14 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #15 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #16 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #17 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #18 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #19 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #20 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #21 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #22 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #23 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #24 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #25 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #26 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #27 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #28 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #29 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #30 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #31 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #32 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #33 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #34 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #35 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #36 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #37 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #38 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #39 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #40 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #41 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #42 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #43 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #44 0x558a55eaa121 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:813\n    #45 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #46 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #47 0x558a55cfb652 in slot_tp_init Objects/typeobject.c:9816\n    #48 0x558a55cd8107 in type_call Objects/typeobject.c:1997\n    #49 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #50 0x558a55eaa121 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:813\n    #51 0x558a55ed8b3e in _PyEval_EvalFrame Include/internal/pycore_ceval.h:119\n    #52 0x558a55ed8b3e in _PyEval_Vector Python/ceval.c:1820\n    #53 0x558a55ed8b3e in PyEval_EvalCode Python/ceval.c:604\n    #54 0x558a5600eb0e in run_eval_code_obj Python/pythonrun.c:1381\n    #55 0x558a5600eb0e in run_eval_code_obj Python/pythonrun.c:1348\n    #56 0x558a5600f127 in run_mod Python/pythonrun.c:1489\n    #57 0x558a560138d0 in pyrun_file Python/pythonrun.c:1295\n    #58 0x558a560138d0 in _PyRun_SimpleFileObject Python/pythonrun.c:517\n    #59 0x558a5601421c in _PyRun_AnyFileObject Python/pythonrun.c:77\n    #60 0x558a560833ec in pymain_run_file_obj Modules/main.c:410\n    #61 0x558a560833ec in pymain_run_file Modules/main.c:429\n    #62 0x558a560833ec in pymain_run_python Modules/main.c:696\n    #63 0x558a56085156 in Py_RunMain Modules/main.c:775\n    #64 0x558a56085156 in pymain_main Modules/main.c:805\n    #65 0x558a56085156 in Py_BytesMain Modules/main.c:829\n    #66 0x7f4a1b9a733f  (/lib64/libc.so.6+0x2733f)\n    #67 0x7f4a1b9a73f8 in __libc_start_main (/lib64/libc.so.6+0x273f8)\n    #68 0x558a55a23754 in _start ([..]/cpython/python+0x19c754)\n\n0x7cda1ad7e038 is located 952 bytes inside of 1096-byte region [0x7cda1ad7dc80,0x7cda1ad7e0c8)\nfreed by thread T0 here:\n    #0 0x7f4a1bd6b9eb  (/usr/lib/gcc/x86_64-pc-linux-gnu/15/libasan.so.8+0x11f9eb)\n    #1 0x7f4a1af87e12 in expat_free Modules/expat/xmlparse.c:934\n    #2 0x7f4a1af87e12 in expat_free Modules/expat/xmlparse.c:906\n    #3 0x7f4a1af87e12 in PyExpat_XML_ParserFree Modules/expat/xmlparse.c:2011\n    #4 0x7f4a1af70286 in xmlparse_dealloc Modules/pyexpat.c:1266\n    #5 0x558a55f60555 in Py_DECREF Include/object.h:949\n    #6 0x558a55f60555 in Py_XDECREF Include/object.h:1042\n    #7 0x558a55f60555 in _PyFrame_ClearLocals Python/frame.c:104\n    #8 0x558a55f60555 in _PyFrame_ClearExceptCode Python/frame.c:129\n    #9 0x558a55e9df91 in clear_thread_frame Python/ceval.c:1682\n    #10 0x558a55e9df91 in _PyEval_FrameClearAndPop Python/ceval.c:1709\n    #11 0x558a55ec6b44 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:5222\n    #12 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #13 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #14 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #15 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #16 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #17 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #18 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #19 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #20 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #21 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #22 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #23 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #24 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #25 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #26 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #27 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #28 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #29 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #30 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #31 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #32 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #33 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #34 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #35 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #36 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #37 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #38 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #39 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #40 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #41 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #42 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #43 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #44 0x558a55eaa121 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:813\n    #45 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #46 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #47 0x558a55cfb652 in slot_tp_init Objects/typeobject.c:9816\n    #48 0x558a55cd8107 in type_call Objects/typeobject.c:1997\n\npreviously allocated by thread T0 here:\n    #0 0x7f4a1bd6ceab in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/15/libasan.so.8+0x120eab)\n    #1 0x7f4a1af8b227 in parserCreate Modules/expat/xmlparse.c:1364\n    #2 0x7f4a1af6f0d1 in newxmlparseobject Modules/pyexpat.c:1211\n    #3 0x7f4a1af6f0d1 in pyexpat_ParserCreate_impl Modules/pyexpat.c:1609\n    #4 0x7f4a1af6f0d1 in pyexpat_ParserCreate Modules/clinic/pyexpat.c.h:511\n    #5 0x558a55c4ed39 in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:440\n    #6 0x558a55b48813 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #7 0x558a55b48813 in PyObject_Vectorcall Objects/call.c:327\n    #8 0x558a55eaa121 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:813\n    #9 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #10 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #11 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #12 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #13 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #14 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #15 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #16 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #17 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #18 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #19 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #20 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #21 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #22 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #23 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #24 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #25 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #26 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #27 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #28 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #29 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #30 0x558a55ebf7de in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1850\n    #31 0x558a55b523d3 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:168\n    #32 0x558a55b523d3 in method_vectorcall Objects/classobject.c:93\n    #33 0x558a55b4d5f6 in _PyVectorcall_Call Objects/call.c:273\n    #34 0x558a55b4d5f6 in _PyObject_Call Objects/call.c:348\n    #35 0x558a55b4d5f6 in PyObject_Call Objects/call.c:373\n    #36 0x558a55eb1e8a in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1362\n    #37 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #38 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #39 0x558a55ce88e6 in slot_tp_call Objects/typeobject.c:9570\n    #40 0x558a55b470d1 in _PyObject_MakeTpCall Objects/call.c:242\n    #41 0x558a55eaa121 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:813\n    #42 0x558a55b4e383 in _PyObject_VectorcallDictTstate Objects/call.c:135\n    #43 0x558a55b4e383 in _PyObject_Call_Prepend Objects/call.c:504\n    #44 0x558a55cfb652 in slot_tp_init Objects/typeobject.c:9816\n    #45 0x558a55cd8107 in type_call Objects/typeobject.c:1997\n\nSUMMARY: AddressSanitizer: heap-use-after-free Modules/expat/xmlparse.c:8660 in getRootParserOf\nShadow bytes around the buggy address:\n  0x7cda1ad7dd80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7de00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7de80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7df00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7df80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n=\u003e0x7cda1ad7e000: fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd\n  0x7cda1ad7e080: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa\n  0x7cda1ad7e100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\n  0x7cda1ad7e180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7e200: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x7cda1ad7e280: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\nShadow byte legend (one shadow byte represents 8 application bytes):\n  Addressable:           00\n  Partially addressable: 01 02 03 04 05 06 07 \n  Heap left redzone:       fa\n  Freed heap region:       fd\n  Stack left redzone:      f1\n  Stack mid redzone:       f2\n  Stack right redzone:     f3\n  Stack after return:      f5\n  Stack use after scope:   f8\n  Global redzone:          f9\n  Global init order:       f6\n  Poisoned by user:        f7\n  Container overflow:      fc\n  Array cookie:            ac\n  Intra object redzone:    bb\n  ASan internal:           fe\n  Left alloca redzone:     ca\n  Right alloca redzone:    cb\n==16187==ABORTING\n```\n\n\u003c/details\u003e\n\nMy understanding is that there is a bug in the graph of object relations and that the same parser instance is being freed twice as a consequence.\n\nCC @picnixz \n\n### CPython versions tested on:\n\n3.15, 3.14, 3.13, 3.12, 3.11, 3.10, 3.9\n\n### Operating systems tested on:\n\nOther, Windows, macOS, Linux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-139403\n* gh-139606\n* gh-139608\n* gh-139609\n* gh-139612\n* gh-139613\n* gh-139614\n* gh-139664\n* gh-139663\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/hartwork","@type":"Person","name":"hartwork"},"datePublished":"2025-09-28T17:40:00.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":19},"url":"https://github.com/139400/cpython/issues/139400"}