René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Allow the linux perf profiler to see Python calls","articleBody":"The linux `perf` profiler is a very powerful tool but unfortunately is not able to see Python calls (only the C stack) and therefore it cannot be used (neither its very complete ecosystem) to profile Python applications and extensions. \r\n\r\nTurns out that node and the [JVM](https://github.com/jvm-profiling-tools/perf-map-agent/blob/master/README.md) have developed a way to leverage the `perf` profiler for the Java and javascript frames. They use their JIT compilers to generate a unique area in memory where they place assembly code that in turn calls the frame evaluator function. This JIT compiled areas are unique per function/code object. They use the perf maps (perf allows to place a map in `/temp/perf-PID.map` with information mapping the JIT-ed areas to a string that identifies them and this allows perf to map java/javascript names to the JIT-ed areas, basically showing the non-native function names on the stack.\r\n\r\nWe can do a simple version of this idea in Python by using a very simple JIT compiler that compiles a assembly template that is the used to jump to `PyEval_EvalFrameDefault` and we can place the code names and filenames in the special `perf` file. This allows perf to see Python calls as well:\r\n\r\n![perf_names](https://user-images.githubusercontent.com/11718525/185669431-e8edaa0c-ab98-4faf-ae59-0cedc9560a10.png)\r\n\r\nAnd this works with all the tools in the perf ecosystem, like flamegraphs:\r\n\r\n![perf_flame](https://user-images.githubusercontent.com/11718525/185669454-a710dd61-557e-46cb-b1d3-53586defe374.png)\r\n\r\n\r\nSee also:\r\nhttps://www.brendangregg.com/Slides/KernelRecipes_Perf_Events.pdf\r\n","author":{"url":"https://github.com/pablogsal","@type":"Person","name":"pablogsal"},"datePublished":"2022-08-20T17:32:57.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/96143/cpython/issues/96143"}