René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"BOLT optimizations fail on Linux aarch64","articleBody":"# Bug report\n\n### Bug description:\n\nWhen running the `--pgo` test suite with a BOLT instrumented binary, the interpreter crashes with\n\n```\n./python -m test --pgo --rerun --verbose3 --timeout=\npython: ../cpython-ro-srcdir/Python/generated_cases.c.h:1074: _PyEval_EvalFrameDefault: Assertion `tp-\u003etp_alloc == PyType_GenericAlloc' failed.\nAborted (core dumped)\n```\n\nI find this surprising since we include `_PyEval_EvalFrameDefault` in the BOLT skip functions — but am not familiar with the details.\n\nThe following patch successfully worked around that error\n\n```diff\ndiff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h\nindex 810beb61d0d..d24add0afab 100644\n--- a/Python/generated_cases.c.h\n+++ b/Python/generated_cases.c.h\n@@ -1071,7 +1071,7 @@\n                 DEOPT_IF(FT_ATOMIC_LOAD_UINT32_RELAXED(tp-\u003etp_version_tag) != type_version, CALL);\n                 assert(tp-\u003etp_new == PyBaseObject_Type.tp_new);\n                 assert(tp-\u003etp_flags \u0026 Py_TPFLAGS_HEAPTYPE);\n-                assert(tp-\u003etp_alloc == PyType_GenericAlloc);\n+                assert(tp-\u003etp_alloc == PyBaseObject_Type.tp_alloc);\n                 PyHeapTypeObject *cls = (PyHeapTypeObject *)callable_o;\n                 PyFunctionObject *init_func = (PyFunctionObject *)FT_ATOMIC_LOAD_PTR_ACQUIRE(cls-\u003e_spec_cache.init);\n                 PyCodeObject *code = (PyCodeObject *)init_func-\u003efunc_code;\n```\n\nThen, the profiling test run succeeded, but BOLT crashed during the apply step.\n\n```\n# Run bolt against the merged data to produce an optimized binary.\nfor bin in python; do \\\n  /usr/lib/llvm-19/bin/llvm-bolt \"${bin}.prebolt\" -o \"${bin}.bolt\" -data=\"${bin}.fdata\" -update-debug-sections -skip-funcs=_PyEval_EvalFrameDefault,sre_ucs1_match/1,sre_ucs2_match/1,sre_ucs4_match/1  -reorder-blocks=ext-tsp -reorder-functions=cdsort -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot ; \\\n  mv \"${bin}.bolt\" \"${bin}\"; \\\ndone\nBOLT-INFO: Target architecture: aarch64\nBOLT-INFO: BOLT version: \u003cunknown\u003e\nBOLT-INFO: first alloc address is 0x400000\nBOLT-INFO: enabling relocation mode\nBOLT-INFO: pre-processing profile using branch profile reader\nBOLT-INFO: number of removed linker-inserted veneers: 0\nBOLT-INFO: 8500 out of 12058 functions in the binary (70.5%) have non-empty execution profile\nBOLT-INFO: 41 functions with profile could not be optimized\nBOLT-INFO: profile for 1 objects was ignored\nBOLT-INFO: removed 1 empty block\nBOLT-INFO: ICF folded 678 out of 12439 functions in 5 passes. 0 functions had jump tables.\nBOLT-INFO: Removing all identical functions will save 46.23 KB of code space. Folded functions were called 3909549484 times based on profile.\nBOLT-INFO: ICP Total indirect calls = 1808544446, 153 callsites cover 99% of all indirect calls\n #0 0x0000aacc1be768cc (/usr/lib/llvm-19/bin/llvm-bolt+0x1ae68cc)\n #1 0x0000aacc1be74b80 (/usr/lib/llvm-19/bin/llvm-bolt+0x1ae4b80)\n #2 0x0000aacc1be77174 (/usr/lib/llvm-19/bin/llvm-bolt+0x1ae7174)\n #3 0x0000ff03feee37e0 (linux-vdso.so.1+0x7e0)\n #4 0x0000aacc1c397200 (/usr/lib/llvm-19/bin/llvm-bolt+0x2007200)\n #5 0x0000aacc1c39aa1c (/usr/lib/llvm-19/bin/llvm-bolt+0x200aa1c)\n #6 0x0000aacc1c39a9e4 (/usr/lib/llvm-19/bin/llvm-bolt+0x200a9e4)\n #7 0x0000aacc1c39a9e4 (/usr/lib/llvm-19/bin/llvm-bolt+0x200a9e4)\n #8 0x0000aacc1bf1ebc4 (/usr/lib/llvm-19/bin/llvm-bolt+0x1b8ebc4)\n #9 0x0000aacc1bf21328 (/usr/lib/llvm-19/bin/llvm-bolt+0x1b91328)\n#10 0x0000aacc1becfe3c (/usr/lib/llvm-19/bin/llvm-bolt+0x1b3fe3c)\n#11 0x0000aacc1aadf2f0 (/usr/lib/llvm-19/bin/llvm-bolt+0x74f2f0)\n#12 0x0000ff03fe8684c4 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3\n#13 0x0000ff03fe868598 call_init ./csu/../csu/libc-start.c:128:20\n#14 0x0000ff03fe868598 __libc_start_main ./csu/../csu/libc-start.c:347:5\n#15 0x0000aacc1aadd4f0 (/usr/lib/llvm-19/bin/llvm-bolt+0x74d4f0)\nPLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.\nStack dump:\n0.\tProgram arguments: /usr/lib/llvm-19/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -skip-funcs=_PyEval_EvalFrameDefault,sre_ucs1_match/1,sre_ucs2_match/1,sre_ucs4_match/1 -reorder-blocks=ext-tsp -reorder-functions=cdsort -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot\nSegmentation fault (core dumped)\n```\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux","author":{"url":"https://github.com/zanieb","@type":"Person","name":"zanieb"},"datePublished":"2025-01-15T15:13:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":10},"url":"https://github.com/128884/cpython/issues/128884"}