René's URL Explorer Experiment


Title: gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache by ericsnowcurrently · Pull Request #106974 · python/cpython · GitHub

Open Graph Title: gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache by ericsnowcurrently · Pull Request #106974 · python/cpython

X Title: gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache by ericsnowcurrently · Pull Request #106974 · python/cpython

Description: This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed? We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter. We also remove the accommodations for "detached" thread states, which were a dubious idea to start with. (This approach does not have the same potential performance penalty as the linked list we used in gh-106899.) Issue: gh-105699

Open Graph Description: This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the...

X Description: This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the...

Opengraph URL: https://github.com/python/cpython/pull/106974

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:b4cc8c4c-bac3-c905-bcf9-00e94638c81c
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idDDA4:9CC2A:EB699C:146CA00:696A862B
html-safe-nonce6bd452f17989212061ec715594881f168fa234e2cb2fcc163568fd756bc99015
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEREE0OjlDQzJBOkVCNjk5QzoxNDZDQTAwOjY5NkE4NjJCIiwidmlzaXRvcl9pZCI6IjEwNzk2MTA2NTY3OTc4MzQ2NyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacf2d3e99f6dc66af1cb36d89f33782e284ca888ca2d121048f411cd3ecf7b6ce6
hovercard-subject-tagpull_request:1445012911
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/106974/files
twitter:imagehttps://avatars.githubusercontent.com/u/1152074?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/1152074?s=400&v=4
og:image:altThis fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None913560fa317c3c5a71e34f9b19253c9f09d02b4b958a86c2a56f4c8541116377
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release5998c30593994bf2589055aef7b22d368a499367
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/106974/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F106974%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F106974%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/106974/files
Reloadhttps://github.com/python/cpython/pull/106974/files
Reloadhttps://github.com/python/cpython/pull/106974/files
python https://github.com/python
cpythonhttps://github.com/python/cpython
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
Notifications https://github.com/login?return_to=%2Fpython%2Fcpython
Fork 33.9k https://github.com/login?return_to=%2Fpython%2Fcpython
Star 71.1k https://github.com/login?return_to=%2Fpython%2Fcpython
Code https://github.com/python/cpython
Issues 5k+ https://github.com/python/cpython/issues
Pull requests 2.1k https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects 31 https://github.com/python/cpython/projects
Security Uh oh! There was an error while loading. Please reload this page. https://github.com/python/cpython/security
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
Insights https://github.com/python/cpython/pulse
Code https://github.com/python/cpython
Issues https://github.com/python/cpython/issues
Pull requests https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
ericsnowcurrentlyhttps://github.com/ericsnowcurrently
python:mainhttps://github.com/python/cpython/tree/main
ericsnowcurrently:fix-extensions-tstate-althttps://github.com/ericsnowcurrently/cpython/tree/fix-extensions-tstate-alt
Conversation 20 https://github.com/python/cpython/pull/106974
Commits 14 https://github.com/python/cpython/pull/106974/commits
Checks 0 https://github.com/python/cpython/pull/106974/checks
Files changed https://github.com/python/cpython/pull/106974/files
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache https://github.com/python/cpython/pull/106974/files#top
Show all changes 14 commits https://github.com/python/cpython/pull/106974/files
3397777 Add a comment. ericsnowcurrently Jul 19, 2023 https://github.com/python/cpython/pull/106974/commits/33977774c6f8229eb098417da1d0f21a61357924
670e632 Add a TODO. ericsnowcurrently Jul 19, 2023 https://github.com/python/cpython/pull/106974/commits/670e632496b32dc638117e3a5d42df877aae0182
485d130 Use a low-level hashtable instead of a dict. ericsnowcurrently Jul 21, 2023 https://github.com/python/cpython/pull/106974/commits/485d130637503a207b0440a4f600a42fffb32042
77d5e8f Drop _PyThreadState_*Detached(). ericsnowcurrently Jul 19, 2023 https://github.com/python/cpython/pull/106974/commits/77d5e8f298ae4dcfd5aa5e8cf7f6cd3f0254e8fc
04a19c4 Set "byname" properly for the reload case. ericsnowcurrently Jul 24, 2023 https://github.com/python/cpython/pull/106974/commits/04a19c456b6b63aab9a1272d0ea08c66b6e46875
5fde127 Use a flat hashtable instead of a nested one. ericsnowcurrently Jul 27, 2023 https://github.com/python/cpython/pull/106974/commits/5fde127bbd6fe6d332f45f91fc655ebd6289cea4
818d828 Merge branch 'main' into fix-extensions-tstate-alt ericsnowcurrently Jul 27, 2023 https://github.com/python/cpython/pull/106974/commits/818d828ca7458fe2637f8000dc13c4d691366712
e83b0cc Use the right allocator. ericsnowcurrently Jul 27, 2023 https://github.com/python/cpython/pull/106974/commits/e83b0cccc1429b92091f6576349b33e13b6ec5bd
9e81cd5 Use PyUnicode_AsUTF8AndSize() (and fix the assert). ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/9e81cd5bcf45c147c05de6503d4339c1dde2692e
430716b Raise MemoryError where appropriate. ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/430716b1b27f5ead37fc6d765e68292d0f401cba
647fbc2 Add a comment. ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/647fbc2c81dbd09b00aa39d489ead78580fdcf00
bbe5eda Use a constant for the cache separator character. ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/bbe5eda9b7f833ac487f05591798295601bcd966
1d7cbd0 Decref when clearing the hashtable. ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/1d7cbd00cc85c154567f4718fd965b981944a5b3
20c8b9d Make the cached module defs immortal. ericsnowcurrently Jul 28, 2023 https://github.com/python/cpython/pull/106974/commits/20c8b9d9415c81a65bd292c2474b050e531e1dc9
Clear filters https://github.com/python/cpython/pull/106974/files
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
pycore_import.h https://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
pycore_pystate.h https://github.com/python/cpython/pull/106974/files#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
pycore_runtime_init.h https://github.com/python/cpython/pull/106974/files#diff-84d4b72a69538f6e60865f5c0301f102c0dca577e47759ac07d7a9a034ee75a2
import.c https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
pystate.c https://github.com/python/cpython/pull/106974/files#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L260
Include/internal/pycore_import.hhttps://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
View file https://github.com/ericsnowcurrently/cpython/blob/20c8b9d9415c81a65bd292c2474b050e531e1dc9/Include/internal/pycore_import.h
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/106974/{{ revealButtonHref }}
https://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
https://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
https://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
https://github.com/python/cpython/pull/106974/files#diff-f0d0635a621a67c4105f6b306ecabc7b6f8e1fe1518ae49baf006c26f71520b5
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L311
Include/internal/pycore_pystate.hhttps://github.com/python/cpython/pull/106974/files#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
View file https://github.com/ericsnowcurrently/cpython/blob/20c8b9d9415c81a65bd292c2474b050e531e1dc9/Include/internal/pycore_pystate.h
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/106974/{{ revealButtonHref }}
https://github.com/python/cpython/pull/106974/files#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
https://github.com/python/cpython/pull/106974/files#diff-76818a128869986e3de4f16dae0257c430008546f04764cd7ad8254abfa5ce83
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L312
Include/internal/pycore_runtime_init.hhttps://github.com/python/cpython/pull/106974/files#diff-84d4b72a69538f6e60865f5c0301f102c0dca577e47759ac07d7a9a034ee75a2
View file https://github.com/ericsnowcurrently/cpython/blob/20c8b9d9415c81a65bd292c2474b050e531e1dc9/Include/internal/pycore_runtime_init.h
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/106974/{{ revealButtonHref }}
https://github.com/python/cpython/pull/106974/files#diff-84d4b72a69538f6e60865f5c0301f102c0dca577e47759ac07d7a9a034ee75a2
https://github.com/python/cpython/pull/106974/files#diff-84d4b72a69538f6e60865f5c0301f102c0dca577e47759ac07d7a9a034ee75a2
https://github.com/python/cpython/blob/main/.github/CODEOWNERS#L261
Python/import.chttps://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
View file https://github.com/ericsnowcurrently/cpython/blob/20c8b9d9415c81a65bd292c2474b050e531e1dc9/Python/import.c
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/106974/{{ revealButtonHref }}
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
https://github.com/python/cpython/pull/106974/files#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3
Please reload this pagehttps://github.com/python/cpython/pull/106974/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.