René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bolt instrumentation missing `-update-debug-sections`","articleBody":"# Bug report\n\n### Bug description:\n\nThe following warning is displayed during BOLT instrumentation\n\n```\nBOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.\n```\n\nThe flag is provided during \"apply\"\n\nhttps://github.com/python/cpython/blob/81376fef76a53fb79893bfa9c9db18d97c228fbe/configure.ac#L2184\n\nbut not during \"instrumentation\"\n\nhttps://github.com/python/cpython/blob/81376fef76a53fb79893bfa9c9db18d97c228fbe/configure.ac#L2170\n\nReproduced with\n\n```\nmake clean\n\nexport CC=clang\nexport CXX=clang++\n\n./configure py_cv_module__openssl=n/a py_cv_module__hashlib=n/a py_cv_module__gdbm=n/a py_cv_module__tkinter=n/a \\\n    --without-ensurepip \\\n    --enable-optimizations --enable-bolt\n\nmake -j8\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eAdditional logs\u003c/summary\u003e\n\u003cpre\u003e\n...\nmake profile-bolt-stamp\nmake[1]: Entering directory '/big/workspace/cpython'\n# Ensure a pristine, pre-BOLT copy of the binary and no profile data from last run.\nfor bin in python; do \\\n  prebolt=\"${bin}.prebolt\"; \\\n  if [ -e \"${prebolt}\" ]; then \\\n    echo \"Restoring pre-BOLT binary ${prebolt}\"; \\\n    mv \"${bin}.prebolt\" \"${bin}\"; \\\n  fi; \\\n  cp \"${bin}\" \"${prebolt}\"; \\\n  rm -f ${bin}.bolt.*.fdata ${bin}.fdata; \\\ndone\n# Instrument each binary.\nfor bin in python; do \\\n  /usr/bin/llvm-bolt \"${bin}\" -instrument -instrumentation-file-append-pid -instrumentation-file=/big/workspace/cpython/${bin}.bolt -o ${bin}.bolt_inst ; \\\n  mv \"${bin}.bolt_inst\" \"${bin}\"; \\\ndone\nBOLT-INFO: Target architecture: x86_64\nBOLT-INFO: BOLT version: 6a0964d75628b15bafd078342120888c0e6d126f\nBOLT-INFO: first alloc address is 0x400000\nBOLT-INFO: creating new program header table at address 0xa00000, offset 0x600000\nBOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.\nBOLT-INFO: enabling relocation mode\nBOLT-INFO: forcing -jump-tables=move for instrumentation\nBOLT-INFO: enabling lite mode\nBOLT-INFO: 0 out of 7748 functions in the binary (0.0%) have non-empty execution profile\nBOLT-INFO: validate-mem-refs updated 2 object references\nBOLT-INSTRUMENTER: Number of indirect call site descriptors: 1995\nBOLT-INSTRUMENTER: Number of indirect call target descriptors: 7719\nBOLT-INSTRUMENTER: Number of function descriptors: 7719\nBOLT-INSTRUMENTER: Number of branch counters: 150867\nBOLT-INSTRUMENTER: Number of ST leaf node counters: 47195\nBOLT-INSTRUMENTER: Number of direct call counters: 0\nBOLT-INSTRUMENTER: Total number of counters: 198062\nBOLT-INSTRUMENTER: Total size of counters: 1584496 bytes (static alloc memory)\nBOLT-INSTRUMENTER: Total size of string table emitted: 166904 bytes in file\nBOLT-INSTRUMENTER: Total size of descriptors: 10927480 bytes in file\nBOLT-INSTRUMENTER: Profile will be saved to file /big/workspace/cpython/python.bolt\nBOLT-INFO: 66759 instructions were shortened\nBOLT-INFO: removed 84 empty blocks\nBOLT-INFO: UCE removed 838 blocks and 51008 bytes of code\nBOLT-INFO: padding code to 0x1600000 to accommodate hot text\nBOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0x18e0950\nBOLT-INFO: clear procedure is 0x18dc390\nBOLT-INFO: patched build-id (flipped last bit)\nBOLT-INFO: setting _end to 0x190247c\nBOLT-INFO: setting _end to 0x190247c\nBOLT-INFO: setting __bolt_runtime_start to 0x18e0900\nBOLT-INFO: setting __bolt_runtime_fini to 0x18e0950\nBOLT-INFO: setting __hot_start to 0xc00000\nBOLT-INFO: setting __hot_end to 0x14b3986\n# Run instrumented binaries to collect data.\n./python -m test --pgo --timeout=\n...\n\u003c/pre\u003e\n\u003c/details\u003e\n\nThis was patched downstream in `python-build-standalone` https://github.com/astral-sh/python-build-standalone/pull/463 — I'm interesting in upstreaming that patch.\n\nA simple patch like\n\n```diff\ndiff --git a/configure.ac b/configure.ac\nindex ee034e5a962..d0245debd5b 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -2167,7 +2167,7 @@ AC_ARG_VAR(\n AC_MSG_CHECKING([BOLT_INSTRUMENT_FLAGS])\n if test -z \"${BOLT_INSTRUMENT_FLAGS}\"\n then\n-  BOLT_INSTRUMENT_FLAGS=\n+  BOLT_INSTRUMENT_FLAGS=\"-update-debug-sections\"\n fi\n AC_MSG_RESULT([$BOLT_INSTRUMENT_FLAGS])\n ```\n \n is sufficient — but downstream a `BOLT_COMMON_FLAGS` variable was added which was useful for subsequent fixes.\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-128455\n* gh-128512\n* gh-128513\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/zanieb","@type":"Person","name":"zanieb"},"datePublished":"2025-01-03T07:56:56.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/128437/cpython/issues/128437"}