René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Crash in repr() for lists containing NULLs","articleBody":"# Crash report\n\n### What happened?\n\nIt's possible to segfault the interpreter from a double-free in `xml.etree.ElementTree.TreeBuilder`. Please let me know if the automated diagnosis is incorrect and whether you prefer me not to post it in new issues.\n\n### Automated diagnosis:\n\n**Missing Py_NewRef causes double-free in treebuilder_handle_end (line 2851)**: `Py_XSETREF(self-\u003elast_for_tail, self-\u003elast)` stores into `last_for_tail` without taking a new reference. Both fields point to the same object with only one reference. `treebuilder_gc_clear` decrements twice — a double-free triggered on every XML end tag. Lines 2882 and 2922 demonstrate the correct pattern with `Py_NewRef`.\nFix: `Py_XSETREF(self-\u003elast_for_tail, Py_NewRef(self-\u003elast));`\n\nMRE:\n```python\nimport xml.etree.ElementTree\nimport gc\n\nbuilder = xml.etree.ElementTree.TreeBuilder()\nfor x in range(10):\n    builder.start(\"a\", {})\nfor x in range(10):\n    builder.end(\"a\")\nroot = builder.close()\nprint(gc.get_referrers(root[0]))\n```\n\nBacktrace:\n```\nProgram received signal SIGSEGV, Segmentation fault.\n0x0000555555b5ca97 in Py_INCREF (op=0x0) at ./Include/refcount.h:281\n281         PY_UINT32_T cur_refcnt = op-\u003eob_refcnt;\n(gdb) bt\n#0  0x0000555555b5ca97 in Py_INCREF (op=0x0) at ./Include/refcount.h:281\n#1  _Py_NewRef (obj=0x0) at ./Include/refcount.h:529\n#2  list_repr_impl (v=0x7c6ff70471a0) at Objects/listobject.c:604\n#3  list_repr (self=0x7c6ff70471a0) at Objects/listobject.c:644\n#4  0x0000555555bf05dc in PyObject_Repr (v=v@entry=0x7c6ff70471a0) at Objects/object.c:781\n#5  0x0000555555cdc514 in PyUnicodeWriter_WriteRepr (writer=writer@entry=0x7c6ff71b2390, obj=obj@entry=0x7c6ff70471a0) at Objects/unicode_writer.c:390\n#6  0x0000555555b5cb0b in list_repr_impl (v=0x7c6ff70749a0) at Objects/listobject.c:615\n#7  list_repr (self=0x7c6ff70749a0) at Objects/listobject.c:644\n#8  0x0000555555bf028b in PyObject_Str (v=0x7c6ff70749a0) at Objects/object.c:823\n#9  0x0000555555b26f23 in PyFile_WriteObject (v=0x7c6ff70749a0, f=0x7d0ff70126a0, flags=flags@entry=1) at Objects/fileobject.c:119\n#10 0x0000555555e4de11 in builtin_print_impl (module=0x7ccff6fe6450, objects=0x7bfff5db6b28, objects_length=1, sep=0x0, end=0x0, file=0x7d0ff70126a0, flush=0) at Python/bltinmodule.c:2356\n#11 builtin_print (module=\u003coptimized out\u003e, args=\u003coptimized out\u003e, nargs=\u003coptimized out\u003e, kwnames=0x0) at Python/clinic/bltinmodule.c.h:1122\n#12 0x0000555555bde65c in cfunction_vectorcall_FASTCALL_KEYWORDS (func=func@entry=0x7c7ff70374c0, args=args@entry=0x7bfff5db6b28, nargsf=nargsf@entry=9223372036854775809,\n    kwnames=kwnames@entry=0x0) at Objects/methodobject.c:465\n#13 0x0000555555ab9e00 in _PyObject_VectorcallTstate (tstate=0x5555568f7b18 \u003c_PyRuntime+360664\u003e, callable=0x7c7ff70374c0, args=0x7bfff5db6b28, nargsf=9223372036854775809, kwnames=0x0)\n    at ./Include/internal/pycore_call.h:136\n#14 0x0000555555e588dd in _Py_VectorCallInstrumentation_StackRefSteal (callable=..., arguments=\u003coptimized out\u003e, total_args=1, kwnames=..., call_instrumentation=\u003coptimized out\u003e,\n    frame=\u003coptimized out\u003e, this_instr=\u003coptimized out\u003e, tstate=\u003coptimized out\u003e) at Python/ceval.c:770\n#15 0x0000555555e94263 in _PyEval_EvalFrameDefault (tstate=\u003coptimized out\u003e, frame=\u003coptimized out\u003e, throwflag=\u003coptimized out\u003e) at Python/generated_cases.c.h:1838\n#16 0x0000555555e57778 in _PyEval_EvalFrame (tstate=0x5555568f7b18 \u003c_PyRuntime+360664\u003e, frame=0x7e8ff6fe5220, throwflag=0) at ./Include/internal/pycore_ceval.h:118\n#17 _PyEval_Vector (tstate=\u003coptimized out\u003e, func=\u003coptimized out\u003e, locals=\u003coptimized out\u003e, args=\u003coptimized out\u003e, argcount=\u003coptimized out\u003e, kwnames=0x0) at Python/ceval.c:2134\n#18 0x0000555555e57195 in PyEval_EvalCode (co=\u003coptimized out\u003e, globals=\u003coptimized out\u003e, locals=0x7c7ff70884c0) at Python/ceval.c:681\n#19 0x0000555556061fb0 in run_eval_code_obj (tstate=tstate@entry=0x5555568f7b18 \u003c_PyRuntime+360664\u003e, co=co@entry=0x7d5ff6ffe990, globals=globals@entry=0x7c7ff70884c0,\n    locals=locals@entry=0x7c7ff70884c0) at Python/pythonrun.c:1368\n```\n\n### Claude explanation of the MRE and crash:\n\n1. The loop calls `builder.end(\"a\")` 10 times, each time executing line 2851: `Py_XSETREF(self-\u003elast_for_tail, self-\u003elast)` — missing `Py_NewRef`. Each end tag over-decrements the element it touches.\n2. After `builder.close()`, `root[0]` (the innermost `\u003ca\u003e`) has had its refcount corrupted — it's lower than it should be.\n3. `gc.get_referrers(root[0])` returns a list containing objects that reference that element. One of those referrers is the `TreeBuilder`'s internal stack list, which itself contains elements with corrupted refcounts.\n4. `print()` calls `list_repr` on that referrers list, which calls `Py_NewRef` on each item. When it hits an element whose refcount was decremented to 0 (already freed), `op` is `NULL` or points to freed memory → segfault at `Py_INCREF(op=0x0)`.\n\nThe backtrace confirms it: `list_repr_impl` at `listobject.c:604` tries to `Py_NewRef` a `NULL` pointer — an object that was prematurely freed due to the cumulative over-decrements from the 10 `end()` calls.\n\n----\n\nFound using [cpython-review-toolkit](https://github.com/devdanzin/cpython-review-toolkit).\n\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n### Output from running 'python -VV' on the command line:\n\nPython 3.15.0a7+ (heads/main:99e2c5eccd2, Mar 17 2026, 08:26:50) [Clang 21.1.2 (2ubuntu6)]\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-146062\n* gh-146129\n* gh-146155\n* gh-146167\n* gh-146204\n* gh-146213\n* gh-146271\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/devdanzin","@type":"Person","name":"devdanzin"},"datePublished":"2026-03-17T12:19:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":7},"url":"https://github.com/146056/cpython/issues/146056"}