René's URL Explorer Experiment


Title: Executor management code lacks proper invariants leading to bugs. · Issue #142513 · python/cpython · GitHub

Open Graph Title: Executor management code lacks proper invariants leading to bugs. · Issue #142513 · python/cpython

X Title: Executor management code lacks proper invariants leading to bugs. · Issue #142513 · python/cpython

Description: Bug report Bug description: Our lifetime management of JIT executors is ad-hoc and has some logical flaws. This results in bugs, and is hard to fix as fixing one bug often seems to produce another. Some examples: #139540 #141658 #141786 ...

Open Graph Description: Bug report Bug description: Our lifetime management of JIT executors is ad-hoc and has some logical flaws. This results in bugs, and is hard to fix as fixing one bug often seems to produce another....

X Description: Bug report Bug description: Our lifetime management of JIT executors is ad-hoc and has some logical flaws. This results in bugs, and is hard to fix as fixing one bug often seems to produce another....

Opengraph URL: https://github.com/python/cpython/issues/142513

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Executor management code lacks proper invariants leading to bugs.","articleBody":"# Bug report\n\n### Bug description:\n\nOur lifetime management of JIT executors is ad-hoc and has some logical flaws.\nThis results in bugs, and is hard to fix as fixing one bug often seems to produce another.\n\nSome examples:\n\nhttps://github.com/python/cpython/issues/139540\nhttps://github.com/python/cpython/issues/141658\nhttps://github.com/python/cpython/issues/141786\n\n\n## Executor management overview\n\nExecutors are reference counted `PyObject` objects allowing them to be interacted with from Python.\nFor performance reasons, the references to currently executing executors are deferred (not counted).\nExecutors can be invalidated at any time, even when it would be unsafe to deallocate an object.\nInvalidation means that the machine code for that executor is no longer valid. The executor object remains a valid object.\n\n### Invalidation\n\nInvalidating an executor does not make it unreachable, but in order to free executors as soon as\npossible, when an executor is made invalid, we delete references to all executors that are side exits\nfrom the invalidated executor. The reference from the code object is also deleted.\nInvalidation can happen at any time, so care must be taken not to break any invariants when\ninvalidating executors.\n\n### Refcount dropping to zero\n\nBecause references to currently executing executors are deferred, executors may still be live when\ntheir reference count drops to zero.\nTo handle this, when an executor's refcount drops to zero, it should not be deallocated, but moved to a \"zero count\" list.\nAlso, deferring deallocation means that it safe to call `Py_DECREF()` on an executor, even if it would not be safe to deallocate it.\n\n### Cycles\n\nSince cycles can be formed between executors, we need the cycle GC to handle executors.\nWhen executors become invalid, we should break any cycles involving them.\n\n## Current implementation\n\nThe current implementation (2025-12-01) maintains a doubly linked list of all executors.\nThis list is scanned when a dependency is broken, `_Py_Executors_InvalidateDependency` and `_Py_Executors_InvalidateAll`, to invalidate exectors.\nThere is also a second double linked list, `executor_deletion_list`.\nWhen the reference count of an executor reacahes zero, it is moved from the main list\nto the deletion list. This helps the GC find any executors with a zero reference count quickly.\n\nWhen the `executor_deletion_list` reaches a certain capacity, we clear it.\nThis is broken. We should only do this during cycle GC.\n\nNote that invalidation and unreachability (being garbage) are two different things.\nWhen an executor becomes invalid, it is useful to drop some references to and from it,\nbut it could still be being executed and may still be reachable, so its integrity as an\nobject must be maintained.\nWhen an executor becomes unreachable, it should be GC'd like any other object, but\nwhen its reference count drops to zero it may still be being executed and should be kept\nintact.\n\n## Proposed implementation\n\n### Invalidation\n\n1. Detach the executor from the code object\n2. Clear all exits.\n\n### Refcount dropping to zero\n\n* Move the executor to the `executor_deletion_list`\n* Invalidate the executor\n\n### `tp_clear` \n\n* Clear all exits and any constants\n\n### Change to the cycle GC\n\n* Move all currently executing executors in the `executor_deletion_list` to the `currently_executing_list`, incrementing the reference count of each.\n* Deallocate all executors in the  `executor_deletion_list` \n* Perform cycle GC as normal\n* For all executors in the `currently_executing_list`:\n    *  decrement the refcount, which will move it back to the `executor_deletion_list` if its refcount drops to zero.\n\n\n\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-142931\n* gh-143073\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/markshannon","@type":"Person","name":"markshannon"},"datePublished":"2025-12-10T14:11:59Z","url":"https://github.com/python/cpython/issues/142513"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:8f496222-811c-9496-781a-1d01c5b38862
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idCE7A:2C4BED:479FBCD:5AC15A2:6B024469
html-safe-nonceeb374ee0a6fce7cd80dfb279b96ad567ff23402cdd01b9356897953eab4aba43
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDRTdBOjJDNEJFRDo0NzlGQkNEOjVBQzE1QTI6NkIwMjQ0NjkiLCJ2aXNpdG9yX2lkIjoiMzY2OTMzMDc0MTkxMDcxNzU0NSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmace8fda06975c7fc28d7389b483054be3f91215af5e0f48e529d5bcede59adef68
hovercard-subject-tagissue:3715373230
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/142513/issue_layout
twitter:imagehttps://opengraph.githubassets.com/11bb4df9c854e9883884b269ca93613dfe79996987c0a1c576aae8c0a16b0b8a/python/cpython/issues/142513
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/11bb4df9c854e9883884b269ca93613dfe79996987c0a1c576aae8c0a16b0b8a/python/cpython/issues/142513
og:image:altBug report Bug description: Our lifetime management of JIT executors is ad-hoc and has some logical flaws. This results in bugs, and is hard to fix as fixing one bug often seems to produce another....
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamemarkshannon
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/cpython/issues/142513#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F142513
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F142513
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/issues/142513
Reloadhttps://github.com/python/cpython/issues/142513
Reloadhttps://github.com/python/cpython/issues/142513
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
https://github.com/python/cpython
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/142513
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/142513
Executor management code lacks proper invariants leading to bugs.https://github.com/python/cpython/issues/142513#top
3.15bugs and security fixeshttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%223.15%22
interpreter-core(Objects, Python, Grammar, and Parser dirs)https://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22interpreter-core%22
topic-JIThttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-JIT%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
https://github.com/markshannon
markshannonhttps://github.com/markshannon
on Dec 10, 2025https://github.com/python/cpython/issues/142513#issue-3715373230
#139540https://github.com/python/cpython/issues/139540
#141658https://github.com/python/cpython/issues/141658
#141786https://github.com/python/cpython/issues/141786
GH-142513: Reimplement executor management #142931https://github.com/python/cpython/pull/142931
GH-142513: fix missing return in executor_clear #143073https://github.com/python/cpython/pull/143073
markshannonhttps://github.com/markshannon
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
interpreter-core(Objects, Python, Grammar, and Parser dirs)https://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22interpreter-core%22
3.15bugs and security fixeshttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%223.15%22
topic-JIThttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-JIT%22
on Dec 10, 2025https://github.com/python/cpython/issues/142513#event-21480172543
markshannonhttps://github.com/markshannon
on Dec 18, 2025https://github.com/python/cpython/issues/142513#event-21648422633
GH-142513: Reimplement executor management (GH-142931)https://github.com/python/cpython/commit/e4058d7cb115286c3c53e0a0be2a7b60af46e073
GH-142513https://github.com/python/cpython/issues/142513
GH-142931https://github.com/python/cpython/pull/142931
e4058d7https://github.com/python/cpython/commit/e4058d7cb115286c3c53e0a0be2a7b60af46e073
cocolatohttps://github.com/cocolato
on Dec 18, 2025https://github.com/python/cpython/issues/142513#event-21649672494
pythonGH-142513: Reimplement executor management (pythonGH-142931)https://github.com/cocolato/cpython/commit/07eee1740cfa79b08130888f51d28e2c7f8f6d40
pythonGH-142513https://github.com/python/cpython/issues/142513
pythonGH-142931https://github.com/python/cpython/pull/142931
07eee17https://github.com/cocolato/cpython/commit/07eee1740cfa79b08130888f51d28e2c7f8f6d40
Fidget-Spinnerhttps://github.com/Fidget-Spinner
on Dec 22, 2025https://github.com/python/cpython/issues/142513#event-21709150451
GH-142513: fix missing return in executor_clear (GH-143073)https://github.com/python/cpython/commit/700e9fad70da3f1da008c3231749e3861fbce897
GH-142513https://github.com/python/cpython/issues/142513
GH-143073https://github.com/python/cpython/pull/143073
700e9fahttps://github.com/python/cpython/commit/700e9fad70da3f1da008c3231749e3861fbce897
markshannonhttps://github.com/markshannon
completedhttps://github.com/python/cpython/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Jan 5, 2026https://github.com/python/cpython/issues/142513#event-21855405217
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/cpython/issues/142513
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/cpython/issues/142513
3.15bugs and security fixeshttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%223.15%22
interpreter-core(Objects, Python, Grammar, and Parser dirs)https://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22interpreter-core%22
topic-JIThttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-JIT%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.