René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Top-of-stack caching in the JIT","articleBody":"Unlike https://github.com/python/cpython/issues/131498 which was a wash for performance, TOS caching in the JIT promises substantial performance improvements. This is because we can create several stencils for each uop, tailored for the number of registers and dynamically vary the number of values cached.\nFor example, in this code:\n```\n  LOAD_FAST_BORROW\n  LOAD_FAST_BORROW\n  BINARY_OP_ADD_INT\n  STORE_FAST\n```\nwe can tailor each version to the number of registers cached:\n```\n  LOAD_FAST_BORROW_0_1  ( 0 -> 1 registers)\n  LOAD_FAST_BORROW_1_2  ( 1 -> 2 registers)\n  BINARY_OP_ADD_INT_2_1 ( 2 -> 1 registers)\n  STORE_FAST_1_0        ( 1 -> 0 registers)\n```\nthus avoiding any memory traffic to and from the stack at all.\n\nThe exact number of variants per uop will need to be determined empirically.\nHaving more stencils allows more freedom when generating code, but excessive numbers of stencils would cause bloat both at runtime, and in any repository containing the stencils. \n\n#### Spilling and reloading\n\nThere will be an upper bound to the number of values cached and some uops may need a minimum number of values in the cache.\nTo handle those we will need to insert spill and reload uops. Spills will reduce the number of cached values, saving them to the in-memory stack and reloads will do the opposite moving values from the in-memory stack to the cache.\n\nE.g.\n```\n  LOAD_FAST_BORROW\n  BINARY_OP_ADD_INT\n```\n`BINARY_OP_ADD_INT` expects two inputs, but we only have one cached (from the `LOAD_FAST_BORROW`) so we need to insert a `RELOAD`:\n```\n  LOAD_FAST_BORROW  ( 0 -> 1 registers)\n  RELOAD_1_2        ( 1 -> 2 registers)\n  BINARY_OP_ADD_INT ( 2 -> 1 registers)\n```\n\n`SPILL` and `RELOAD` are semantic no-ops, and will be generated automatically.\n\n#### Deferred references\n\nFor this to work the code generator must spill any cached values to the in-memory stack when GC could occur. Fortunately, the code generator already does this (as part of the work for  https://github.com/python/cpython/issues/131498).\n\nSee https://github.com/faster-cpython/ideas/issues/711\n\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-135384\n* gh-135465\n* gh-135563\n* gh-135668\n* gh-135706\n* gh-135707\n* gh-135714\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/markshannon","@type":"Person","name":"markshannon"},"datePublished":"2025-06-11T09:03:10Z","url":"https://github.com/python/cpython/issues/135379"}