Title: Use dual stacks to separate control and data · Issue #157559 · python/cpython · GitHub
Open Graph Title: Use dual stacks to separate control and data · Issue #157559 · python/cpython
X Title: Use dual stacks to separate control and data · Issue #157559 · python/cpython
Description: Currently, in CPython, the stack is implemented as a linked list of frames. These frames are mostly allocated in large chunks, but may not be contiguous as generator frames are allocated as part of the generator. Each frame's size depend...
Open Graph Description: Currently, in CPython, the stack is implemented as a linked list of frames. These frames are mostly allocated in large chunks, but may not be contiguous as generator frames are allocated as part of...
X Description: Currently, in CPython, the stack is implemented as a linked list of frames. These frames are mostly allocated in large chunks, but may not be contiguous as generator frames are allocated as part of...
Opengraph URL: https://github.com/python/cpython/issues/157559
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Use dual stacks to separate control and data","articleBody":"Currently, in CPython, the stack is implemented as a linked list of frames. These frames are mostly allocated in large chunks, but may not be contiguous as generator frames are allocated as part of the generator.\n\nEach frame's size depends on its code object, which makes scanning the scan slow and requires dynamic memory allocation to account for varying stack sizes, making it hard for external profilers, like Tachyon, to snapshot the stack without relying on undocumented assumptions about the VM.\n\nTachyon is advertised as zero-overhead, but it isn't. It requires extra code on the fast path of yields and returns and couples the VM and profiler in undocumented and hard to maintain ways preventing improvements in the VM e.g. https://github.com/python/cpython/pull/148681\n\nBy splitting the stack into two parts, a control stack and an object stack, we can make the control stack simpler, smaller and more regular, which would:\n* make it easier and faster for Tachyon to copy and parse, and keep the object stack layout flexible, and\n* give the VM freedom to arrange the object stack however is best for performance.\n\n### Performance impact\n\nHaving two stacks, means two stack pointers, using a register in the interpreter and JIT.\nHowever, having a control stack of fixed-sized control frames, would simplify recursion depth checking and other bookkeeping tasks.\n\nI expect the additional costs and the savings to largely cancel out.\n\nPrimarily, this is about decoupling profilers from the VM design, not performance, so a small initial slowdown would be acceptable.\n\nHaving the object stack being purely composed of object pointers, and no control, may allow some additional optimizations across calls, slightly reduced stack memory use, and slightly faster stack scanning for the GC, so this might eventually give a small performance boost.\n\n### Control frames\n\n#### Profiler view\n\nTo a profiler, or other out-of-process tool, a control frame will look like this:\n```C\ntypedef struct {\n PyObject *executable; // The code object, or non-Python callable, for this frame.\n char padding[CONTROL_FRAME_SIZE - sizeof(void *)];\n} _PyControlFrame;\n```\nIf callable is a code object, then additional information is available:\n```C\ntypedef struct {\n PyCodeObject *code; // The code for this frame\n _Py_CODEUNIT *instr_ptr; /* Instruction currently executing (may be approximate) */\n char padding2[CONTROL_FRAME_SIZE - 2 * sizeof(void *)];\n} _PyPythonControlFrame;\n```\n\n#### Implementation\n\nThe control frame contains all the data that isn't object pointers from the current interpreter frame, plus the pointer to the code object:\n```C\ntypedef struct {\n PyObject *executable; /* Borrowed reference */\n _Py_CODEUNIT *instr_ptr;\n _PyInterpreterDataFrame *framepointer;\n _PyStackRef *stackpointer;\n uint8_t owner;\n uint8_t visited; /* For GC */\n uint16_t return_offset; /* Only relevant during a function call */\n /* 32 bits used for tlbc_index in FT build */\n} _PyControlFrameInternal;\n\ntypedef struct {\n _PyStackRef f_funcobj; /* Deferred or strong reference. */\n _PyStackRef f_globals; /* Borrowed reference. */\n _PyStackRef f_builtins; /* Borrowed reference. */\n _PyStackRef f_locals; /* Strong reference, may be NULL. */\n _PyStackRef frame_obj; /* Strong reference, may be NULL. */\n /* Locals and stack */\n _PyStackRef localsplus[1]; \n} _PyInterpreterDataFrame;\n```\n\n### Debug info and guarantees for profilers\n\nOut of process debuggers and profilers require information to traverse internal data structures. The VM will provide this information:\n* `uint32_t control_frame_offset` offset of the control frame pointer in the thread state\n* `uint32_t control_base_offset` offset of the pointer to the base of the stack in the thread state\n* `uint32_t control_frame_size` the size (in bytes) of a control frame == `CONTROL_FRAME_SIZE` above.\n\nAny changes to the base pointer changes will be protected by a memory fence, so other processors will see the change.\nThe control frame pointer will not be synchronized, so may appear out-of-date to other processors.\n\n\n------------------------------------\n\nPrior discussion focused on implementation: https://github.com/faster-cpython/ideas/issues/675\nhttps://github.com/python/cpython/issues/115946 explains why this helps profilers.\n","author":{"url":"https://github.com/markshannon","@type":"Person","name":"markshannon"},"datePublished":"2026-09-15T14:43:52Z","url":"https://github.com/python/cpython/issues/157559"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:847a719f-5fc5-b873-204e-7f5da0cf3a07 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | B088:1148D3:415F04A:5341929:6B022C44 |
| html-safe-nonce | 74ac00707e315ea64be31fb880588d36bf996aedc5d5d269b324971ca80f89d2 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMDg4OjExNDhEMzo0MTVGMDRBOjUzNDE5Mjk6NkIwMjJDNDQiLCJ2aXNpdG9yX2lkIjoiNzAyODcxMDcyNjExNDA5NDE0OCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 9e8cd658c440995f44336fe27eb5ee8325e9770351032777d8fc4f45c538bfe0 |
| hovercard-subject-tag | issue:5463466342 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/157559/issue_layout |
| twitter:image | https://opengraph.githubassets.com/ce4c0484f5ade942bbbfd75abeb38d698fb0b0dff368f4eff59854a71cd56546/python/cpython/issues/157559 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/ce4c0484f5ade942bbbfd75abeb38d698fb0b0dff368f4eff59854a71cd56546/python/cpython/issues/157559 |
| og:image:alt | Currently, in CPython, the stack is implemented as a linked list of frames. These frames are mostly allocated in large chunks, but may not be contiguous as generator frames are allocated as part of... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | markshannon |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width