René's URL Explorer Experiment


Title: Faster, parallel type checking · Issue #933 · python/mypy · GitHub

Open Graph Title: Faster, parallel type checking · Issue #933 · python/mypy

X Title: Faster, parallel type checking · Issue #933 · python/mypy

Description: If we have a sufficiently large program to type check we should be able to speed up type checking significantly by using multiple type checker processes that work in parallel. One potential way to implement this: Have a coordinator proce...

Open Graph Description: If we have a sufficiently large program to type check we should be able to speed up type checking significantly by using multiple type checker processes that work in parallel. One potential way to ...

X Description: If we have a sufficiently large program to type check we should be able to speed up type checking significantly by using multiple type checker processes that work in parallel. One potential way to ...

Opengraph URL: https://github.com/python/mypy/issues/933

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Faster, parallel type checking","articleBody":"If we have a sufficiently large program to type check we should be able to speed up type checking significantly by using multiple type checker processes that work in parallel.\n\nOne potential way to implement this:\n- Have a coordinator process that maintains a module dependency graph and keeps track of a work list of modules that haven't been type checked. Initially this will contain all modules in the program.\n- Have multiple worker processes that repeatedly get (sets of cyclically dependent) modules from the coordinator that have their dependencies processed, type check them and mark them as processed on the coordinator.\n\nFor this to really work we are going to need a way of efficiently communicating type check results for a module between processes (to avoid type checking shared dependencies multiple times). Having a JSON serialization format (see #932) would be sufficient.\n\nAdditionally we need a quick way of figuring out the dependency graph of all modules (or at least an approximation of it). We'll probably have to cache that between runs and update it incrementally, similar to the approach outlined in #932.\n\nSo how much would this help? Obviously this depends on the shape of the dependency graph. Under reasonable assumptions we should be able to hit an effective level of parallelism of at least 4 for larger programs, but I wouldn't be surprised if we could get even better than that. Cyclic module dependencies can add a limit to how far we can parallelize. We can probably estimate the achievable level of parallelism for a particular program by analyzing the module dependency graph.\n\nThis is probably only worth implementing after we have incremental type checking (#932), and we should preserve incrementalism -- i.e., we'd only type check modules modified since the last run and modules that depend on them.\n","author":{"url":"https://github.com/JukkaL","@type":"Person","name":"JukkaL"},"datePublished":"2015-10-15T06:30:19Z","url":"https://github.com/python/mypy/issues/933"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:68cd45d0-0142-1f25-1ccf-1b82b8f02d22
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id92FC:D064:20B796B:2C5D5FD:6B001CD6
html-safe-nonce6ca4108ffd870f3675d0e996f9e7a09d7787f6c5984dde37c16259332697c752
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5MkZDOkQwNjQ6MjBCNzk2QjoyQzVENUZEOjZCMDAxQ0Q2IiwidmlzaXRvcl9pZCI6IjEyMjk1MDI1NzcxNzU5NjA3OTAiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac2548c2947017142f0e226dccf74019de993e9685c1ff2fc063374089ccd9889e
hovercard-subject-tagissue:111553782
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/933/issue_layout
twitter:imagehttps://opengraph.githubassets.com/f4e327032416717bc20e98409e8d8bb678b4da6c1fd0a66223d40811d6d16f72/python/mypy/issues/933
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/f4e327032416717bc20e98409e8d8bb678b4da6c1fd0a66223d40811d6d16f72/python/mypy/issues/933
og:image:altIf we have a sufficiently large program to type check we should be able to speed up type checking significantly by using multiple type checker processes that work in parallel. One potential way to ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameJukkaL
hostnamegithub.com
expected-hostnamegithub.com
None4eb4cba0d69f045374ac15de9666d22b0961e1d555cc3824f36b7734c00c78e0
turbo-cache-controlno-preview
go-importgithub.com/python/mypy git https://github.com/python/mypy.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id7053637
octolytics-dimension-repository_nwopython/mypy
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id7053637
octolytics-dimension-repository_network_root_nwopython/mypy
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release44c658b24abf00491fb7ae5f56d42c9f38cbeab2
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/mypy/issues/933#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F933
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F933
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fmypy
Reloadhttps://github.com/python/mypy/issues/933
Reloadhttps://github.com/python/mypy/issues/933
Reloadhttps://github.com/python/mypy/issues/933
pythonhttps://github.com/python
mypyhttps://github.com/python/mypy
Codehttps://github.com/python/mypy
Issues2.7k (2.7k)https://github.com/python/mypy/issues
Pull requests546 (546)https://github.com/python/mypy/pulls
Actionshttps://github.com/python/mypy/actions
Projectshttps://github.com/python/mypy/projects
Wikihttps://github.com/python/mypy/wiki
Security and qualityhttps://github.com/python/mypy/security
Insightshttps://github.com/python/mypy/insights
https://github.com/python/mypy
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/933
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/933
#20280https://github.com/python/mypy/pull/20280
Faster, parallel type checkinghttps://github.com/python/mypy/issues/933#top
#20280https://github.com/python/mypy/pull/20280
needs discussionhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22needs%20discussion%22
performancehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22performance%22
topic-incrementalhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-incremental%22
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Oct 15, 2015https://github.com/python/mypy/issues/933#issue-111553782
#932https://github.com/python/mypy/issues/932
#932https://github.com/python/mypy/issues/932
#932https://github.com/python/mypy/issues/932
Nexthttps://github.com/python/mypy/issues/933?timeline_page=1
JukkaLhttps://github.com/JukkaL
needs discussionhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22needs%20discussion%22
on Oct 15, 2015https://github.com/python/mypy/issues/933#event-436041281
https://github.com/jhance
jhancehttps://github.com/jhance
on Oct 17, 2015https://github.com/python/mypy/issues/933#issuecomment-148879872
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Oct 17, 2015https://github.com/python/mypy/issues/933#issuecomment-148948807
#932https://github.com/python/mypy/issues/932
gvanrossumhttps://github.com/gvanrossum
topic-incrementalhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-incremental%22
on Apr 8, 2016https://github.com/python/mypy/issues/933#event-620551445
gvanrossumhttps://github.com/gvanrossum
on Apr 8, 2016https://github.com/python/mypy/issues/933#event-620551566
gvanrossumhttps://github.com/gvanrossum
priority-2-lowhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-2-low%22
on Jan 6, 2017https://github.com/python/mypy/issues/933#event-914699303
gvanrossumhttps://github.com/gvanrossum
on Mar 29, 2017https://github.com/python/mypy/issues/933#event-1021274339
ilevkivskyihttps://github.com/ilevkivskyi
performancehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22performance%22
on May 11, 2019https://github.com/python/mypy/issues/933#event-2334966682
https://github.com/fzyzcjy
fzyzcjyhttps://github.com/fzyzcjy
on Dec 15, 2022https://github.com/python/mypy/issues/933#issuecomment-1353188284
https://github.com/manmartgarc
manmartgarchttps://github.com/manmartgarc
on Apr 25, 2023https://github.com/python/mypy/issues/933#issuecomment-1521173244
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Apr 25, 2023https://github.com/python/mypy/issues/933#issuecomment-1521905253
@manmartgarchttps://github.com/manmartgarc
JukkaLhttps://github.com/JukkaL
priority-2-lowhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-2-low%22
on May 4, 2023https://github.com/python/mypy/issues/933#event-9168480378
https://github.com/jgarvin
jgarvinhttps://github.com/jgarvin
on Jan 20, 2024https://github.com/python/mypy/issues/933#issuecomment-1901730751
https://github.com/aiven-anton
aiven-antonhttps://github.com/aiven-anton
on Jan 22, 2024https://github.com/python/mypy/issues/933#issuecomment-1903579845
@jgarvinhttps://github.com/jgarvin
https://github.com/jgarvin
jgarvinhttps://github.com/jgarvin
on Jan 22, 2024https://github.com/python/mypy/issues/933#issuecomment-1904596196
https://github.com/ilevkivskyi
ilevkivskyihttps://github.com/ilevkivskyi
on Sep 5, 2025https://github.com/python/mypy/issues/933#issuecomment-3259946028
https://github.com/ilevkivskyi
ilevkivskyihttps://github.com/ilevkivskyi
on Sep 6, 2025https://github.com/python/mypy/issues/933#issuecomment-3261885783
https://github.com/ilevkivskyi
ilevkivskyihttps://github.com/ilevkivskyi
on Sep 6, 2025https://github.com/python/mypy/issues/933#issuecomment-3262169723
#19798https://github.com/python/mypy/pull/19798
ilevkivskyihttps://github.com/ilevkivskyi
on Sep 12, 2025https://github.com/python/mypy/issues/933#event-19662293453
Write cache for modules with errors (#19820)https://github.com/python/mypy/commit/8bfecd4e9fbbcb26390b4b851bf2c6f9e9e34e00
#19820https://github.com/python/mypy/pull/19820
8bfecd4https://github.com/python/mypy/commit/8bfecd4e9fbbcb26390b4b851bf2c6f9e9e34e00
ilevkivskyihttps://github.com/ilevkivskyi
on Sep 30, 2025https://github.com/python/mypy/issues/933#event-20006829430
Do not update states after writing cache (#19936)https://github.com/python/mypy/commit/a12565c8253f6b0996a9b11162776c9083aaceb8
#19936https://github.com/python/mypy/pull/19936
a12565chttps://github.com/python/mypy/commit/a12565c8253f6b0996a9b11162776c9083aaceb8
ilevkivskyihttps://github.com/ilevkivskyi
on Oct 16, 2025https://github.com/python/mypy/issues/933#event-20322200788
Switch to a more dynamic SCC processing logic (#20053)https://github.com/python/mypy/commit/e1643aec5229e37cf62700dabae294b9a5344c97
#20053https://github.com/python/mypy/pull/20053
e1643aehttps://github.com/python/mypy/commit/e1643aec5229e37cf62700dabae294b9a5344c97
ilevkivskyihttps://github.com/ilevkivskyi
completedhttps://github.com/python/mypy/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#20280https://github.com/python/mypy/pull/20280
on Dec 1, 2025https://github.com/python/mypy/issues/933#event-21281368384
ilevkivskyihttps://github.com/ilevkivskyi
on Dec 1, 2025https://github.com/python/mypy/issues/933#event-21281368928
Initial implementation of parallel type checking (#20280)https://github.com/python/mypy/commit/985dc7f5b03f384066d91e54065d21cfbc063edd
#20280https://github.com/python/mypy/pull/20280
985dc7fhttps://github.com/python/mypy/commit/985dc7f5b03f384066d91e54065d21cfbc063edd
RiyaPrabhakar-githttps://github.com/RiyaPrabhakar-git
GC-Content-Calculatorhttps://github.com/users/RiyaPrabhakar-git/projects/1
on Dec 12, 2025https://github.com/python/mypy/issues/933#event-21527464401
github-project-automationhttps://github.com/apps/github-project-automation
GC-Content-Calculatorhttps://github.com/users/RiyaPrabhakar-git/projects/1
on Dec 12, 2025https://github.com/python/mypy/issues/933
github-project-automationhttps://github.com/apps/github-project-automation
GC-Content-Calculatorhttps://github.com/users/RiyaPrabhakar-git/projects/1
on Dec 12, 2025https://github.com/python/mypy/issues/933
ilevkivskyihttps://github.com/ilevkivskyi
on Jan 29, 2026https://github.com/python/mypy/issues/933#event-22390438949
Refactor State creation logic (#20691)https://github.com/python/mypy/commit/ba3e0649f1c20ef3d4f68000b68196a5ddb0f6d2
#20691https://github.com/python/mypy/pull/20691
ba3e064https://github.com/python/mypy/commit/ba3e0649f1c20ef3d4f68000b68196a5ddb0f6d2
Add boilerplate for graph broadcasting (#20715)https://github.com/python/mypy/commit/84e8f1f723db749302e152b0988c0a9c960375bf
#20715https://github.com/python/mypy/pull/20715
84e8f1fhttps://github.com/python/mypy/commit/84e8f1f723db749302e152b0988c0a9c960375bf
ilevkivskyihttps://github.com/ilevkivskyi
on Feb 13, 2026https://github.com/python/mypy/issues/933#event-22767616531
Support parallel type checking on Windows (#20777)https://github.com/python/mypy/commit/67350214a7c41821c59c40238c7a440e0c4d3b6e
#20777https://github.com/python/mypy/pull/20777
6735021https://github.com/python/mypy/commit/67350214a7c41821c59c40238c7a440e0c4d3b6e
Send early errors to workers (#20822)https://github.com/python/mypy/commit/70de10e410bd0f72de814533f703d150d57ec1a8
#20822https://github.com/python/mypy/pull/20822
70de10ehttps://github.com/python/mypy/commit/70de10e410bd0f72de814533f703d150d57ec1a8
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/mypy/issues/933
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/mypy/issues/933
needs discussionhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22needs%20discussion%22
performancehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22performance%22
topic-incrementalhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-incremental%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.