René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"heap-use-after-free in _PyFunction_LookupByVersion","articleBody":"# Crash report\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nmacOS\n\n### Output from running 'python -VV' on the command line:\n\nPython 3.13.0a0 (heads/main:531930f47f, Aug 21 2023, 17:14:02) [Clang 14.0.3 (clang-1403.0.22.14.1)]\n\n### What happened?\n\nShortest repro I've found so far:\n```\n./python.exe -Xuops -m test test_opcache -v\n```\nI'm guessing I'm doing something wrong with the `func_version_cache` logic (the table contains borrowed pointers which are cleared by `func_dealloc`), but I can't quite figure out what. You can't subclass the function type so clearing the table entry in `func_dealloc` would *seem* to be safe. The output from address sanitizer hasn't jogged my memory yet.\n\nClick for address sanitizer output:\n\u003cdetails\u003e\n```\n0:00:00 load avg: 2.38 Run tests sequentially\n0:00:00 load avg: 2.38 [1/1] test_opcache\n=================================================================\n==48442==ERROR: AddressSanitizer: heap-use-after-free on address 0x610000f109e8 at pc 0x000102bf6885 bp 0x7ff7bd537070 sp 0x7ff7bd537068\nREAD of size 4 at 0x610000f109e8 thread T0\n    #0 0x102bf6884 in _PyFunction_LookupByVersion funcobject.c:285\n    #1 0x102fd3c74 in uop_optimize optimizer.c:792\n    #2 0x102fd0c8d in _PyOptimizer_BackEdge optimizer.c:169\n    #3 0x102e98232 in _PyEval_EvalFrameDefault generated_cases.c.h:2960\n    #4 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #5 0x102b87545 in _PyFunction_Vectorcall call.c\n    #6 0x102b84b0a in _PyObject_VectorcallDictTstate call.c:146\n    #7 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #8 0x102cdbd0e in slot_tp_init typeobject.c:9057\n    #9 0x102cc3411 in type_call typeobject.c:1700\n    #10 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #11 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #12 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #13 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #14 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #15 0x102b87545 in _PyFunction_Vectorcall call.c\n    #16 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #17 0x102b8d9af in method_vectorcall classobject.c:91\n    #18 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #19 0x102b86ebd in _PyObject_Call call.c:348\n    #20 0x102b8723c in PyObject_Call call.c:373\n    #21 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #22 0x102bd92f2 in gen_send_ex2 genobject.c:232\n    #23 0x102bd8c60 in gen_send_ex genobject.c:273\n    #24 0x102bda780 in _gen_throw genobject.c:561\n    #25 0x102bd999e in gen_throw genobject.c:598\n    #26 0x102baa5ab in method_vectorcall_FASTCALL descrobject.c:410\n    #27 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186\n    #28 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #29 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #30 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #31 0x102b87545 in _PyFunction_Vectorcall call.c\n    #32 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #33 0x102b8d8c3 in method_vectorcall classobject.c:61\n    #34 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186\n    #35 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #36 0x102e840b6 in _PyEval_EvalFrameDefault generated_cases.c.h:3519\n    #37 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #38 0x102b87545 in _PyFunction_Vectorcall call.c\n    #39 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #40 0x102b8d9af in method_vectorcall classobject.c:91\n    #41 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #42 0x102b86ebd in _PyObject_Call call.c:348\n    #43 0x102b8723c in PyObject_Call call.c:373\n    #44 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #45 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #46 0x102b87545 in _PyFunction_Vectorcall call.c\n    #47 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #48 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #49 0x102cd971b in slot_tp_call typeobject.c:8813\n    #50 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #51 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #52 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #53 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #54 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #55 0x102b87545 in _PyFunction_Vectorcall call.c\n    #56 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #57 0x102b8d9af in method_vectorcall classobject.c:91\n    #58 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #59 0x102b86ebd in _PyObject_Call call.c:348\n    #60 0x102b8723c in PyObject_Call call.c:373\n    #61 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #62 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #63 0x102b87545 in _PyFunction_Vectorcall call.c\n    #64 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #65 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #66 0x102cd971b in slot_tp_call typeobject.c:8813\n    #67 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #68 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #69 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #70 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #71 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #72 0x102b87545 in _PyFunction_Vectorcall call.c\n    #73 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #74 0x102b8d9af in method_vectorcall classobject.c:91\n    #75 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #76 0x102b86ebd in _PyObject_Call call.c:348\n    #77 0x102b8723c in PyObject_Call call.c:373\n    #78 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #79 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #80 0x102b87545 in _PyFunction_Vectorcall call.c\n    #81 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #82 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #83 0x102cd971b in slot_tp_call typeobject.c:8813\n    #84 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #85 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #86 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #87 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #88 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #89 0x102b87545 in _PyFunction_Vectorcall call.c\n    #90 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #91 0x102b8d9af in method_vectorcall classobject.c:91\n    #92 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #93 0x102b86ebd in _PyObject_Call call.c:348\n    #94 0x102b8723c in PyObject_Call call.c:373\n    #95 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #96 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #97 0x102b87545 in _PyFunction_Vectorcall call.c\n    #98 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #99 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #100 0x102cd971b in slot_tp_call typeobject.c:8813\n    #101 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #102 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #103 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #104 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #105 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #106 0x102b87545 in _PyFunction_Vectorcall call.c\n    #107 0x103199e70 in _PyObject_VectorcallTstate pycore_call.h:186\n    #108 0x10319ce35 in partial_vectorcall _functoolsmodule.c:230\n    #109 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186\n    #110 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #111 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #112 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #113 0x102b87545 in _PyFunction_Vectorcall call.c\n    #114 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #115 0x102b8d8c3 in method_vectorcall classobject.c:61\n    #116 0x102b86b77 in _PyVectorcall_Call call.c:285\n    #117 0x102b86ebd in _PyObject_Call call.c:348\n    #118 0x102b8723c in PyObject_Call call.c:373\n    #119 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #120 0x102e64cc6 in PyEval_EvalCode ceval.c:557\n    #121 0x102e5b47d in builtin_exec bltinmodule.c.h:540\n    #122 0x102c770b2 in cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:441\n    #123 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186\n    #124 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #125 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n    #126 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #127 0x102b87545 in _PyFunction_Vectorcall call.c\n    #128 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #129 0x102b86ebd in _PyObject_Call call.c:348\n    #130 0x102b8723c in PyObject_Call call.c:373\n    #131 0x10308900f in pymain_run_module main.c:300\n    #132 0x103086b09 in Py_RunMain main.c:688\n    #133 0x1030888b1 in pymain_main main.c:718\n    #134 0x103088bb0 in Py_BytesMain main.c:742\n    #135 0x1029c1918 in main python.c:15\n    #136 0x7ff80da2d41e in start+0x76e (dyld:x86_64+0xfffffffffff6e41e) (BuildId: 5db85b72c63a318291e55c942ec30e4832000000200000000100000000040d00)\n\n0x610000f109e8 is located 168 bytes inside of 184-byte region [0x610000f10940,0x610000f109f8)\nfreed by thread T0 here:\n    #0 0x10444cee9 in wrap_free+0xa9 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x48ee9) (BuildId: 756bb7515781379f84412f22c4274ffd2400000010000000000a0a0000030d00)\n    #1 0x102c8ca5b in _PyMem_RawFree obmalloc.c:74\n    #2 0x102c90ee5 in _PyMem_DebugFree obmalloc.c:2297\n    #3 0x102c8fb42 in PyObject_Free obmalloc.c:831\n    #4 0x10308eccd in PyObject_GC_Del gcmodule.c:2415\n    #5 0x102bf8228 in func_dealloc funcobject.c:934\n    #6 0x102c8c164 in _Py_Dealloc object.c:2656\n    #7 0x102c0628b in list_dealloc listobject.c:357\n    #8 0x102c8c164 in _Py_Dealloc object.c:2656\n    #9 0x102f594a8 in _PyFrame_ClearExceptCode frame.c:140\n    #10 0x102eb2819 in clear_thread_frame ceval.c:1486\n    #11 0x102eac816 in _PyEval_FrameClearAndPop ceval.c:1513\n    #12 0x102e95dab in _PyEval_EvalFrameDefault generated_cases.c.h:1064\n    #13 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #14 0x102b87545 in _PyFunction_Vectorcall call.c\n    #15 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #16 0x102b8d9af in method_vectorcall classobject.c:91\n    #17 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #18 0x102b86ebd in _PyObject_Call call.c:348\n    #19 0x102b8723c in PyObject_Call call.c:373\n    #20 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #21 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #22 0x102b87545 in _PyFunction_Vectorcall call.c\n    #23 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #24 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #25 0x102cd971b in slot_tp_call typeobject.c:8813\n    #26 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #27 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n    #28 0x102b86d4f in PyObject_Vectorcall call.c:327\n    #29 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759\n\npreviously allocated by thread T0 here:\n    #0 0x10444cda0 in wrap_malloc+0xa0 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x48da0) (BuildId: 756bb7515781379f84412f22c4274ffd2400000010000000000a0a0000030d00)\n    #1 0x102c8c9f4 in _PyMem_RawMalloc obmalloc.c:46\n    #2 0x102c90a21 in _PyMem_DebugMalloc obmalloc.c:2282\n    #3 0x102c8f9a7 in PyObject_Malloc obmalloc.c:802\n    #4 0x10308e2a2 in _PyObject_GC_New gcmodule.c:2337\n    #5 0x102bf5d2c in PyFunction_NewWithQualName funcobject.c:188\n    #6 0x102bf6aba in PyFunction_New funcobject.c:312\n    #7 0x102e71421 in _PyEval_EvalFrameDefault generated_cases.c.h:4621\n    #8 0x102e64cc6 in PyEval_EvalCode ceval.c:557\n    #9 0x1030261e8 in run_eval_code_obj pythonrun.c:1725\n    #10 0x103021d94 in run_mod pythonrun.c:1746\n    #11 0x10301f75f in PyRun_StringFlags pythonrun.c:1621\n    #12 0x102e5a6a1 in builtin_eval bltinmodule.c.h:456\n    #13 0x102f3995f in _PyUopExecute executor_cases.c.h:2417\n    #14 0x102e97857 in _PyEval_EvalFrameDefault generated_cases.c.h:2982\n    #15 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #16 0x102b87545 in _PyFunction_Vectorcall call.c\n    #17 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186\n    #18 0x102b8d9af in method_vectorcall classobject.c:91\n    #19 0x102b86bb1 in _PyVectorcall_Call call.c:273\n    #20 0x102b86ebd in _PyObject_Call call.c:348\n    #21 0x102b8723c in PyObject_Call call.c:373\n    #22 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601\n    #23 0x102e6511f in _PyEval_Vector ceval.c:1622\n    #24 0x102b87545 in _PyFunction_Vectorcall call.c\n    #25 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135\n    #26 0x102b87bbe in _PyObject_Call_Prepend call.c:504\n    #27 0x102cd971b in slot_tp_call typeobject.c:8813\n    #28 0x102b85226 in _PyObject_MakeTpCall call.c:242\n    #29 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184\n\nSUMMARY: AddressSanitizer: heap-use-after-free funcobject.c:285 in _PyFunction_LookupByVersion\nShadow bytes around the buggy address:\n  0x1c20001e20e0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\n  0x1c20001e20f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa\n  0x1c20001e2100: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\n  0x1c20001e2110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa\n  0x1c20001e2120: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\n=\u003e0x1c20001e2130: fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fa\n  0x1c20001e2140: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\n  0x1c20001e2150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa\n  0x1c20001e2160: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\n  0x1c20001e2170: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd\n  0x1c20001e2180: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd\nShadow byte legend (one shadow byte represents 8 application bytes):\n  Addressable:           00\n  Partially addressable: 01 02 03 04 05 06 07 \n  Heap left redzone:       fa\n  Freed heap region:       fd\n  Stack left redzone:      f1\n  Stack mid redzone:       f2\n  Stack right redzone:     f3\n  Stack after return:      f5\n  Stack use after scope:   f8\n  Global redzone:          f9\n  Global init order:       f6\n  Poisoned by user:        f7\n  Container overflow:      fc\n  Array cookie:            ac\n  Intra object redzone:    bb\n  ASan internal:           fe\n  Left alloca redzone:     ca\n  Right alloca redzone:    cb\n==48442==ABORTING\nFatal Python error: Aborted\n\nCurrent thread 0x00007ff851463640 (most recent call first):\n  File \"/Users/guido/cpython/Lib/traceback.py\", line 357 in _walk_tb_with_full_positions\n  File \"/Users/guido/cpython/Lib/traceback.py\", line 421 in _extract_from_extended_frame_gen\n  File \"/Users/guido/cpython/Lib/traceback.py\", line 697 in __init__\n  File \"/Users/guido/cpython/Lib/traceback.py\", line 139 in format_exception\n  File \"/Users/guido/cpython/Lib/test/support/testresult.py\", line 96 in __makeErrorDict\n  File \"/Users/guido/cpython/Lib/test/support/testresult.py\", line 113 in addFailure\n  File \"/Users/guido/cpython/Lib/unittest/case.py\", line 98 in _addError\n  File \"/Users/guido/cpython/Lib/unittest/case.py\", line 75 in testPartExecutor\n  File \"/Users/guido/cpython/Lib/contextlib.py\", line 159 in __exit__\n  File \"/Users/guido/cpython/Lib/unittest/case.py\", line 633 in run\n  File \"/Users/guido/cpython/Lib/unittest/case.py\", line 690 in __call__\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 122 in run\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 84 in __call__\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 122 in run\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 84 in __call__\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 122 in run\n  File \"/Users/guido/cpython/Lib/unittest/suite.py\", line 84 in __call__\n  File \"/Users/guido/cpython/Lib/test/support/testresult.py\", line 143 in run\n  File \"/Users/guido/cpython/Lib/test/support/__init__.py\", line 1116 in _run_suite\n  File \"/Users/guido/cpython/Lib/test/support/__init__.py\", line 1242 in run_unittest\n  File \"/Users/guido/cpython/Lib/test/libregrtest/runtest.py\", line 294 in _test_module\n  File \"/Users/guido/cpython/Lib/test/libregrtest/runtest.py\", line 330 in _runtest_inner2\n  File \"/Users/guido/cpython/Lib/test/libregrtest/runtest.py\", line 373 in _runtest_inner\n  File \"/Users/guido/cpython/Lib/test/libregrtest/runtest.py\", line 248 in _runtest\n  File \"/Users/guido/cpython/Lib/test/libregrtest/runtest.py\", line 278 in runtest\n  File \"/Users/guido/cpython/Lib/test/libregrtest/main.py\", line 483 in run_tests_sequential\n  File \"/Users/guido/cpython/Lib/test/libregrtest/main.py\", line 621 in run_tests\n  File \"/Users/guido/cpython/Lib/test/libregrtest/main.py\", line 799 in _main\n  File \"/Users/guido/cpython/Lib/test/libregrtest/main.py\", line 758 in main\n  File \"/Users/guido/cpython/Lib/test/libregrtest/main.py\", line 822 in main\n  File \"/Users/guido/cpython/Lib/test/__main__.py\", line 2 in \u003cmodule\u003e\n  File \"/Users/guido/cpython/Lib/runpy.py\", line 88 in _run_code\n  File \"/Users/guido/cpython/Lib/runpy.py\", line 198 in _run_module_as_main\n\nExtension modules: _testcapi, _testinternalcapi (total: 2)\nAbort trap: 6\n```\n\u003c/details\u003e\n\n\n### Error messages\n\n_No response_\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-108296\n* gh-108383\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/gvanrossum","@type":"Person","name":"gvanrossum"},"datePublished":"2023-08-22T00:27:13.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/108253/cpython/issues/108253"}