René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Supporting stack unwinding in the JIT compiler","articleBody":"# TLDR\r\n\r\nThis is a lot of text because the issue is complex but if you want the gist:\r\n\r\nTo not break a lot of tools that rely on unwinding, I propose to fix this by **compiling the JIT stencils with frame pointers** which has a **trivial maintenance cost** (2 lines) and only **involves a 2% hit** on speed when using the JIT while making **almost all existing debuggers and profilers just work™** in the presence of the JIT.\r\n\r\n**Although this doesn't fix everything** sadly I think is the **best compromise that I can find** (and we are quite lucky to have it as normally fixing this is a nightmare - as in thousands of complex lines of code and 50% slowdown nightmare).\r\n\r\n------\r\n\r\n# The issue + proposal\r\n\r\nCPython's JIT compiler must provide robust stack unwinding support to maintain compatibility with the Python ecosystem's debugging and profiling tools. This requirement is particularly critical given Python's reliance on native extensions written in C, C++, and Rust. As more performance-critical code moves to these native implementations, the ability to properly unwind through mixed Python and native frames becomes fundamental for effective debugging and profiling. This capability is even more crucial with the introduction of both JIT compilation and free-threaded modes, which significantly increase the complexity of runtime behavior. Without proper unwinding support, debugging te issues that can appear in the presence of these new modes becomes extremely challenging – developers would be unable to generate meaningful stack traces, hampering their ability to understand where and why their programs failed. Consider debugging a deadlock in free threadded code and a Rust extension, or investigating a crash in native code called from JIT-compiled Python – without proper unwinding support, error reports would be incomplete or misleading, making production issues significantly harder to diagnose and fix. The ability to get complete, accurate stack traces in these scenarios is not just a convenience; it's essential for maintaining production applications where Python increasingly interacts with native code through multiple execution modes.\r\n\r\nProfiling is equally affected – modern performance analysis tools like py-spy, Austin, or eBPF-based solutions need to understand the complete call stack, including JIT-compiled code, native extensions, and regular Python frames, to provide accurate performance insights.\r\n\r\nSome of the popular tools that use native unwinding in some way or form:\r\n\r\n- Austin (statistical profiler). Uses libunwind\r\n- py-spy (statistical profiler).Uses libunwind\r\n- memray: uses C++ unwinder for macOS and libunwind for Linux\r\n- pystack: uses elfutils (as is the only one supporting core files).\r\n- EBPf unwinders : normally rely on frame pointers or rely on the C++ unwinder (https://israelo.io/blog/ehframes/, https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-based-stack-walking-using-ebpf).\r\n- Additionally, most JIT compilers implement not only frame pointers to make possible to profile and debug, but also implement either C++ exception unwinder  via eh_frames or even more hardcore support for gdb and other debuggers.\r\n- Gdb and hdb-helpers: a combination of libdw, libunwind or custom\r\n- Perf: frame pointers, lubunwind or libdw\r\n- Lldb: libunwind or custom\r\n- valgrind: uses a custom unwinder based on libdw \r\n\r\nThis doesn't include the considerable amount of custom tools that are not open source either in the debugging or profiling space. \r\n\r\n## Unwinding Libraries and Their Capabilities\r\n\r\nSeveral libraries provide stack unwinding capabilities, each with its own strengths and limitations. \r\n\r\n| Library | Platforms | Local Unwinding | Remote Unwinding | Core Files | Registration API | Remote Registration Support | Main Users |\r\n|---------|-----------|-----------------|------------------|------------|------------------|---------------------------|------------|\r\n| libunwind | Linux only | Yes (fast) | Yes | No | `_U_dyn_register` | No | Profilers, debuggers |\r\n| Native Unwinder (LLVM/GDB) | Linux, MacOS | Yes (optimized) | No | No | `__register_frame` | No | C++ exceptions, `backtrace()` |\r\n| libdw (elfutils) | Linux only | Yes | Yes | Yes | None (ELF parsing) | No | GDB, debuggers like pystack |\r\n| GDB/LLDB built-in | Cross-platform | Yes | Yes | Yes | `__jit_debug_register_code` | No | Debug tools |\r\n\r\n# Implementation Plan for Stack Unwinding Support\r\n\r\nAfter extensive experimentation and analysis (believe me this was **a lot of very difficult research work**, @brandtbucher and I found that since we have `preserve_none` and LLVM 19 we can just compile the JIT stencils with frame pointers and that makes *most* tools just work. Based on this I propose implementing frame pointer support as the primary strategy for stack unwinding in CPython's JIT. This approach has proven to be the most pragmatic and effective solution, providing broad compatibility with existing tools while maintaining reasonable performance characteristics. I do think we got very lucky this fixes most of the tools since adding unwinding support for JITs otherwise is **very** challenging (see following sections).\r\n\r\nFrame pointer support can be enabled through a minimal change to the JIT compiler flags:\r\n\r\n```diff\r\ndiff --git a/Tools/jit/_targets.py b/Tools/jit/_targets.py\r\nindex d8dce0a905c..7c3b2e5aab7 100644\r\n--- a/Tools/jit/_targets.py\r\n+++ b/Tools/jit/_targets.py\r\n@@ -135,6 +135,8 @@ async def _compile(\r\n             # Don't call stack-smashing canaries that we can't find or patch:\r\n             \"-fno-stack-protector\",\r\n             \"-std=c11\",\r\n+            \"-fno-omit-frame-pointer\",\r\n+            \"-mno-omit-leaf-frame-pointer\",\r\n             \"-o\",\r\n             f\"{o}\",\r\n             f\"{c}\",\r\n```\r\n\r\nThis simple change enables compatibility with a wide range of tools:\r\n- libunwind (both local and remote unwinding)\r\n- libdw (supporting core file analysis, remote and local unwinding)\r\n- GDB and LLDB\r\n- eBPF-based unwinders\r\n\r\nMost of the popular tools will just work with this change, instead of having to implement tons of different unwinding information support, [which is a nightmare](https://www.corsix.org/content/libunwind-dynamic-code-x86-64))\r\n\r\n## Performance Impact\r\n\r\nhttps://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20241114-3.14.0a1%2B-925b70b-JIT/bm-20241114-linux-x86_64-brandtbucher-justin_frame_pointer-3.14.0a1%2B-925b70b-vs-base.md\r\n\r\nThe performance impact of this change has been carefully measured:\r\n- AMD64: Approximately 2% overhead, as shown in [recent benchmarks](https://raw.githubusercontent.com/faster-cpython/benchmarking-public/refs/heads/main/results/bm-20241114-3.14.0a1%2B-925b70b-JIT/bm-20241114-linux-x86_64-brandtbucher-justin_frame_pointer-3.14.0a1%2B-925b70b-vs-base.svg)\r\n- ARM64 and macOS: Even lower overhead due to the presence of dedicated link registers\r\n\r\nThis modest performance cost is significantly outweighed by the expected JIT performance improvements. Moreover, the maintenance burden is minimal compared to alternative approaches like implementing custom unwinding support for each tool.\r\n\r\n## Additional Optimization\r\n\r\nWe can further enhance compatibility by generating eh-frames in the JIT stencils and calling `__register_frame_table` (a bulk version of `__register_frame`). Benchmarks show this additional feature has neutral performance impact while enabling native unwinder support. This makes the solution even more robust without additional overhead.\r\n\r\nGiven these considerations - the minimal implementation complexity, broad tool compatibility, reasonable performance characteristics, and the ability to extend support to native unwinders - frame pointer support represents the optimal path forward for CPython's JIT implementation.\r\n\r\n## Why this must be activated by default\r\n\r\nThe decision to enable frame pointers by default is driven by two critical production requirements.\r\n\r\n* When applications crash or hang, tools like pystack need to analyze program state without the luxury of reproduction – you cannot simply \"run it again with debug options enabled.\" You need to unwind the stack either remotely or from a core file. \r\n* Performance profiling in production requires safely inspecting program state from another process or the kernel. Tools like Austin, py-spy, and eBPF-based profilers need to periodically sample stack traces without interrupting the target process. Both scenarios require frame pointers to work reliably.\r\n\r\nThis must be enabled by default because most Python users don't compile Python themselves – they install it through package managers or tools like uv. Unless frame pointers are enabled by default, these critical debugging and profiling capabilities won't be available in most Python installations, severely impacting the quality of bug reports to both CPython and C extensions. Given the minimal performance impact of 2% frame pointers should be enabled by default, following the principle of being \"safe by default.\" While we should provide the option to disable them, this should be a conscious decision made by end users who understand they're trading away the ability to properly profile and debug their applications in production – not a choice made by intermediate distributors or package managers.\r\n\r\n## Additional information\r\n\r\nUsing frame pointers aligns with a broader industry trend where major tech companies and Linux distributions are reverting previous optimization decisions and re-enabling frame pointers. Companies like Meta and Google now compile their entire software stack, including their Python installations, with frame pointers enabled. This shift is driven by the recognition that observability and debugging capabilities in production environments far outweigh the minor performance impact of frame pointers. Ubuntu has also adopted this approach, now compiling virtually all packages with frame pointers enabled (with Python being a notable exception we aim to address). This industry-wide move reflects a fundamental reality: in modern production environments, the ability to profile, monitor, and debug applications effectively is crucial, and frame pointers provide the most reliable and universal way to achieve this. The performance cost, typically around 1-5%, is normally considered to be a worthwhile trade-off for the improved observability they enable. This is particularly relevant for server workloads where understanding performance characteristics and debugging production issues is far more valuable than the small overhead frame pointers introduce. The fact that major tech companies maintain this policy even for performance-critical services demonstrates that the benefits of comprehensive profiling and debugging support outweigh the minimal performance impact. Some links:\r\n\r\nhttps://ubuntu.com/blog/ubuntu-performance-engineering-with-frame-pointers-by-default\r\nhttps://www.brendangregg.com/blog/2024-03-17/the-return-of-the-frame-pointers.html\r\nhttps://www.polarsignals.com/blog/posts/2023/12/13/embracing-frame-pointers-in-ubuntu-24-04-lts\r\n\r\n-------------\r\n\r\n# Background on Stack Unwinding Support Requirements\r\n\r\nAdding JIT compilation to CPython requires careful consideration of stack unwinding support. Without proper unwinding capabilities, we risk breaking compatibility with essential development tools that Python developers rely on daily. To understand the scope of this requirement, we need to examine the landscape of tools that depend on stack unwinding.\r\n\r\n## Tools Requiring Stack Unwinding\r\n\r\nStack unwinding is a critical capability used by three major categories of tools in the Python ecosystem:\r\n\r\n* **Debuggers** (GDB, LLDB, pystack) rely on unwinding to show developers where their program is currently executing. These tools deal with remote processes or core files.\r\n\r\n* **Profilers** use unwinding to understand program performance. \r\n\r\n* **C++ Exception Handling** depends on unwinding information to:\r\n  - Propagate exceptions through the stack\r\n  - Clean up resources during stack unwinding\r\n\r\n## Unwinding Mechanisms\r\n\r\nStack unwinding occurs in two fundamentally different contexts:\r\n\r\n* **In-process unwinding**:\r\n  - Code examines its own stack\r\n  - Used by tracing profilers and exception handlers\r\n  - Has direct access to program memory and CPU registers\r\n  - Primarily used for live analysis\r\n\r\n* **Out-of-process unwinding**:\r\n  - External program examines another program's stack\r\n  - Used by debuggers and statistical profilers\r\n  - Includes core dump analysis\r\n\r\n## Types of Analysis Tools\r\n\r\nThe tools that perform unwinding can be categorized by their data collection method:\r\n\r\n* **Tracing Profilers**:\r\n  - Run inside the target process\r\n  - Instrument function entries and exits\r\n  - Collect detailed execution information\r\n  - Higher overhead but more precise\r\n  - Rely on local unwinding capabilities\r\n  - Examples: cProfile, memray\r\n\r\n* **Statistical Profilers**:\r\n  - Run outside the target process\r\n  - Sample program state periodically\r\n  - Much lower overhead\r\n  - Suitable for production use\r\n  - Growing eBPF ecosystem:\r\n    - Enables efficient system-wide analysis\r\n    - Becoming standard for production monitoring\r\n    - Requires reliable stack unwinding support\r\n    - Examples: perf, py-spy, bcc tools\r\n\r\n# Unwinding Libraries and Their Capabilities\r\n\r\nSeveral libraries provide stack unwinding capabilities, each with its own strengths and limitations. The strategies to support unwinding for JIT compiled code are:\r\n\r\n* Allow frame pointers: makes libdw, gdb, lldb and lubinwind work. libunwind and libdw work in remote mode, including core files.\r\n\r\n* Per unwinder support for dynamically generated code using DWARF (only local mode and only if it supports it - libdw has no support):\r\n    * libunwind: Construct ELF .debug_frame data structures, and table_entry data structures, and a unw_dyn_table_info data structure, and a unw_dyn_info_t structure, then call _U_dyn_register.\r\n    *  C++ exception unwinder: Construct ELF .eh_frame data structures, then call __register_frame \r\n    *  GDB: Construct a full in-memory ELF object, manually maintain a doubly-linked list of all such objects in a global variable called __jit_debug_descriptor, and call a global function called __jit_debug_register_code when this list is changed\r\n\r\nAny other library  has **no support**. \r\n\r\n## Other JITs\r\n\r\nMost JIT compilers implement not only frame pointers to make it possible to profile and debug, but also implement either C++ exception unwinder via eh_frames or even more hardcore support for gdb and other debuggers. Some links:\r\n\r\nLuaJIT: https://github.com/LuaJIT/LuaJIT/blob/fe71d0fb54ceadfb5b5f3b6baf29e486d97f6059/src/lj_err.c#L620\r\nDotNet: https://github.com/dotnet/coreclr/pull/468/files\r\nDotNet remote unwinding: https://github.com/dotnet/runtime/blob/d9855f27f7d5d5138452009e9f0dd7a81c5c8b74/src/coreclr/pal/src/exception/remote-unwind.cpp\r\nJulia (frame pointers and __register_frame): https://github.com/JuliaLang/julia/blob/caa2f7d52b430f50c8038a7f6766edba28a3fb65/src/debuginfo.cpp#L554\r\nLLVM's ORC jit (frame pointers, __register_frame and much more): https://github.com/llvm/llvm-project/blob/935d753c6dca0cd9bc5ea14fde5b00386ebcc5be/compiler-rt/lib/orc/elfnix_platform.cpp#L191\r\nV8: Frame pointers, __register_frame and much more: https://www.kvakil.me/posts/2022-10-17-ustackjs-unwind-javascript-stacks-in-ebpf.html, https://github.com/v8/v8/blob/b70457462cb22753a011096c9c9be20275dc4437/src/diagnostics/gdb-jit.cc#L1674, https://github.com/v8/v8/blob/b70457462cb22753a011096c9c9be20275dc4437/src/diagnostics/unwinder.cc#L100, https://github.com/v8/v8/blob/b70457462cb22753a011096c9c9be20275dc4437/src/diagnostics/eh-frame.cc\r\n\r\n\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-143548\n* gh-144137\n* gh-146071\n* gh-146520\n* gh-146521\n* gh-146524\n* gh-147945\n* gh-147994\n* gh-148002\n* gh-148872\n* gh-149103\n* gh-149104\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/pablogsal","@type":"Person","name":"pablogsal"},"datePublished":"2024-11-16T19:33:22Z","url":"https://github.com/python/cpython/issues/126910"}