René's URL Explorer Experiment


Title: gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters by ericsnowcurrently · Pull Request #103084 · python/cpython · GitHub

Open Graph Title: gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters by ericsnowcurrently · Pull Request #103084 · python/cpython

X Title: gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters by ericsnowcurrently · Pull Request #103084 · python/cpython

Description: Sharing mutable (or non-immortal) objects between interpreters is generally not safe. We can work around that but not easily. There are two restrictions that are critical for objects that break interpreter isolation. The first is that the object's state be guarded by a global lock. For now the GIL meets this requirement, but a granular global lock is needed once we have a per-interpreter GIL. The second restriction is that the object (and, for a container, its items) be deallocated/resized only when the interpreter in which it was allocated is the current one. This is because every interpreter has (or will have, see gh-101660) its own object allocator. Deallocating an object with a different allocator can cause crashes. The dict for the cache of module defs is completely internal, which simplifies what we have to do to meet those requirements. To do so, we do the following: add a mechanism for re-using a temporary thread state tied to the main interpreter in an arbitrary thread add _PyRuntime.imports.extensions.main_tstate add _PyThreadState_InitDetached() and _PyThreadState_ClearDetached() (pystate.c) add _PyThreadState_BindDetached() and _PyThreadState_UnbindDetached() (pystate.c) make sure the cache dict (_PyRuntime.imports.extensions.dict) and its items are all owned by the main interpreter) add a placeholder using for a granular global lock Note that the cache is only used for legacy extension modules and not for multi-phase init modules. (This PR was derived from gh-102925, which I recently reverted. Also, there was a similar, more complex PR that I've closed: gh-102938.) Issue: gh-100227

Open Graph Description: Sharing mutable (or non-immortal) objects between interpreters is generally not safe. We can work around that but not easily. There are two restrictions that are critical for objects that break in...

X Description: Sharing mutable (or non-immortal) objects between interpreters is generally not safe. We can work around that but not easily. There are two restrictions that are critical for objects that break in...

Opengraph URL: https://github.com/python/cpython/pull/103084

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/commits/:range(.:format)
route-controllerpull_requests
route-actioncommits
fetch-noncev2:252035fe-76d2-f48d-90a7-a78acef15afc
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idE236:206B44:287C1C:3590FB:6994F187
html-safe-nonce47116f9fa11c9ac6e4ebb709049570e164af66817fecb1865b7740dc7e6f7478
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFMjM2OjIwNkI0NDoyODdDMUM6MzU5MEZCOjY5OTRGMTg3IiwidmlzaXRvcl9pZCI6IjU2NzAwODYxNjA4ODc5MDI1OTkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac6e7d9772cee47467e5a9ea7a4b63316e7d7247ebca1b88d5ff49aa813914d230
hovercard-subject-tagpull_request:1293167297
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/commits
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
twitter:imagehttps://avatars.githubusercontent.com/u/1152074?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/1152074?s=400&v=4
og:image:altSharing mutable (or non-immortal) objects between interpreters is generally not safe. We can work around that but not easily. There are two restrictions that are critical for objects that break in...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None4bd759bc5f83244e2a0de29b937365905c0fefd238b6f077c24a49830375b4df
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release48c364f96c0b8ac95ea493e8f02d59e2a5af92a3
ui-targetcanary-1
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F103084%2Fcommits%2Fd79afe586c31bdb342f5aa7289f9320ed7d91c9d
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F103084%2Fcommits%2Fd79afe586c31bdb342f5aa7289f9320ed7d91c9d
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Fcommits&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
Reloadhttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
Reloadhttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
python https://github.com/python
cpythonhttps://github.com/python/cpython
Please reload this pagehttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
Notifications https://github.com/login?return_to=%2Fpython%2Fcpython
Fork 34.1k https://github.com/login?return_to=%2Fpython%2Fcpython
Star 71.5k https://github.com/login?return_to=%2Fpython%2Fcpython
Code https://github.com/python/cpython
Issues 5k+ https://github.com/python/cpython/issues
Pull requests 2.1k https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects 31 https://github.com/python/cpython/projects
Security 0 https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Code https://github.com/python/cpython
Issues https://github.com/python/cpython/issues
Pull requests https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
ericsnowcurrentlyhttps://github.com/ericsnowcurrently
python:mainhttps://github.com/python/cpython/tree/main
ericsnowcurrently:extensions-dict-owned-by-main-interpreter-2https://github.com/ericsnowcurrently/cpython/tree/extensions-dict-owned-by-main-interpreter-2
Conversation 1 https://github.com/python/cpython/pull/103084
Commits 14 https://github.com/python/cpython/pull/103084/commits
Checks 0 https://github.com/python/cpython/pull/103084/checks
Files changed https://github.com/python/cpython/pull/103084/files
Please reload this pagehttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#top
Show all changes 14 commits https://github.com/python/cpython/pull/103084/files
ff4eb36 Make _PyRuntime.imports.extensions a struct. ericsnowcurrently Mar 27, 2023 https://github.com/python/cpython/pull/103084/commits/ff4eb3621768e59066c0d25144f9ac07ac63ab24
e558d76 Factor out add_threadstate(). ericsnowcurrently Mar 21, 2023 https://github.com/python/cpython/pull/103084/commits/e558d765ecb03972d95f076daf3a73e9d5b998b5
acf5269 Add _PyThreadState_InitDetached(). ericsnowcurrently Mar 21, 2023 https://github.com/python/cpython/pull/103084/commits/acf5269bdf94009f72ae8650f6253398d9b6fbb7
181106e Add _PyThreadState_ClearDetached(). ericsnowcurrently Mar 21, 2023 https://github.com/python/cpython/pull/103084/commits/181106e3eef47cdfb5b933b8bd3b6f71da48d36c
b8ffbba Add extensions_lock_acquire() and extensions_lock_release(). ericsnowcurrently Mar 27, 2023 https://github.com/python/cpython/pull/103084/commits/b8ffbba3b81b9b9241c23541057a1abba10e047a
a86aea5 Add _PyRuntime.imports.extensions.main_tstate. ericsnowcurrently Mar 27, 2023 https://github.com/python/cpython/pull/103084/commits/a86aea5b46d956efe68215ca91fb0320d116431c
d79afe5 Add _PyThreadState_BindDetached() and _PyThreadState_UnbindDetached(). ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
5182186 Share the extensions dict safely. ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/51821860affc43f26478f6faddd9007424200132
f6db0a2 Intern the name and filename in the main interpreter. ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/f6db0a270b40c1ce230bdd1442b67a0a505e9269
dac9c23 Fix the asserts in _PyThreadState_BindDetached() and _PyThreadState_U… ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/dac9c23e4194a6232fdd97cb6bd47c2ad32fda6e
8d442e2 Swap back when done using main_tstate. ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/8d442e2d79d7bc517f9eba4bec5f256be9d09b7f
ee70a96 Disable a prematurely-restrictive assert. ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/ee70a96101040dac9fc07105a8022f40f557d317
e9c37fe Fix a copy-and-paste bug. ericsnowcurrently Mar 28, 2023 https://github.com/python/cpython/pull/103084/commits/e9c37fe6e04abefdba37a9472004488236adc99d
8e005bf Decref name & filename if copied. ericsnowcurrently Mar 29, 2023 https://github.com/python/cpython/pull/103084/commits/8e005bf00be6f7f4e1ef2e3db154fd279e053ec3
Clear filters https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
Please reload this pagehttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
Please reload this pagehttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
pycore_pystate.h https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
pystate.c https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
Prev https://github.com/python/cpython/pull/103084/commits/a86aea5b46d956efe68215ca91fb0320d116431c
Next https://github.com/python/cpython/pull/103084/commits/51821860affc43f26478f6faddd9007424200132
Please reload this pagehttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d
https://github.com/ericsnowcurrently
ericsnowcurrentlyhttps://github.com/python/cpython/commits?author=ericsnowcurrently
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L318
Include/internal/pycore_pystate.hhttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
View file https://github.com/ericsnowcurrently/cpython/blob/d79afe586c31bdb342f5aa7289f9320ed7d91c9d/Include/internal/pycore_pystate.h
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/103084/commits/{{ revealButtonHref }}
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L318
Python/pystate.chttps://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
View file https://github.com/ericsnowcurrently/cpython/blob/d79afe586c31bdb342f5aa7289f9320ed7d91c9d/Python/pystate.c
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/103084/commits/{{ revealButtonHref }}
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/pull/103084/commits/d79afe586c31bdb342f5aa7289f9320ed7d91c9d#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.