René's URL Explorer Experiment


Title: Feature: Reduce Mypy's Cache Size · Issue #15731 · python/mypy · GitHub

Open Graph Title: Feature: Reduce Mypy's Cache Size · Issue #15731 · python/mypy

X Title: Feature: Reduce Mypy's Cache Size · Issue #15731 · python/mypy

Description: Reducing Mypy's Cache Size This is a meta-issue talking about different ways to reduce Mypy's cache size. I've been working on a branch in my free time, though it is probably too big for a PR, so I thought it would be best to get everyon...

Open Graph Description: Reducing Mypy's Cache Size This is a meta-issue talking about different ways to reduce Mypy's cache size. I've been working on a branch in my free time, though it is probably too big for a PR, so I...

X Description: Reducing Mypy's Cache Size This is a meta-issue talking about different ways to reduce Mypy's cache size. I've been working on a branch in my free time, though it is probably too big fo...

Opengraph URL: https://github.com/python/mypy/issues/15731

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Feature: Reduce Mypy's Cache Size","articleBody":"## Reducing Mypy's Cache Size\r\n\r\nThis is a meta-issue talking about different ways to reduce Mypy's cache size. I've been working on a [branch](https://github.com/dosisod/mypy/tree/shrink-cache) in my free time, though it is probably too big for a PR, so I thought it would be best to get everyone's opinion on which optimizations (if any) would be worth-while.\r\n\r\nIn short, I've reduced Mypy's filesystem cache by about 27% using a few different techniques.\r\n\r\nHere is a breakdown of each of the commits, what I did to reduce the cache size, and by how much. We probably don't need to include all of these techniques since a lot of them only marginally reduce the cache size. The cache I've been using as a comparison is Mypy's own cache when checking itself.\r\n\r\n## The Numbers\r\n\r\n| Commit      | Total Size | Diff  | Percent \\* | Optimization Technique |\r\n|-------------|------------|------:|--------:|------------------------|\r\n| `master`    | 26.9MB     |       |         |                        |\r\n| `763a94d5e` | 26.5MB     | 0.4MB |    1.6% | Use ints instead of bools for certain truthy/falsey values |\r\n| `623266f47` | 24.2MB     | 2.2MB |    8.4% | Don't store fields that are nullable |\r\n| `6cefbfb27` | 23.4MB     | 0.8MB |    3.0% | Drop `builtins.` prefix for common types |\r\n| `27e9e0d56` | 23.4MB     |  47KB |    0.2% | Don't cache `builtins.object` in MRO because everything derives from it |\r\n| `d2d0aa005` | 22.2MB     | 1.2MB |    4.5% | Don't store empty list/tuples in TypeInfo classes |\r\n| `cce01a60f` | 22.1MB     |  17KB |    0.1% | Don't store empty symbol tables |\r\n| `88b5b6a3d` | 21.3MB     | 0.8MB |    3.0% | Don't store `arg_names` and `arg_kinds` for func defs with type info |\r\n| `4275d51b5` | 21.1MB     | 0.2MB |    0.8% | Store LDEF/GDEF/MDEF as ints |\r\n| `0a4dfeab2` | 19.9MB     | 1.2MB |    4.4% | Replace `.class` key with empty string |\r\n| `abf66951c` | 19.9MB     | 0.1MB |    0.5% | Store `NoneType` node as `\"None\"` string literal |\r\n| `22b91d94d` | 19.6MB     | 0.1MB |    0.6% | Optimize `def_extras` usage |\r\n| `98c4ce817` | 19.6MB     |   5KB |    0.0% | Don't store `Instance` args if they're empty |\r\n|             |            |       |         |                        |\r\n| Total       |            | 7.0MB |   27.0% |                        |\r\n\r\n> \\* Percent is calculated based on the original cache size of the `master` branch\r\n\r\nThe best techniques based on total savings are:\r\n\r\n* Don't store fields that are nullable (8.4%)\r\n* Don't store empty list/tuples in TypeInfo classes (4.5%)\r\n* Replace `.class` key with empty string (4.4%)\r\n* Drop `builtins.` prefix for common types (3.0%)\r\n* Don't store `arg_names` and `arg_kinds` for func defs with type info (3.0%)\r\n\r\nAfter that point everything starts to drop off, though they still might be worth including.\r\n\r\n## Backwards Compatibility\r\n\r\nI made sure to check that my changes would not break backwards compatibility. These new techniques allow for loading of both old and new caches, but of course the new cache format will be used when cached files needs to be rebuild.\r\n\r\n## Why?\r\n\r\nI was poking around the cache folder to see why the cache was the size it was, and I noticed that there was a lot of optimizations that could be made. In theory, smaller caches are quicker to save and load, while taking up less space on the users computer. In CI systems where storage space is metered, having a smaller cache size will speed up CI workflows and use up less cloud storage.\r\n\r\nLet me know if this is something you guys would be interested in! If so I'll start splitting this into separate PR's. Thanks!","author":{"url":"https://github.com/dosisod","@type":"Person","name":"dosisod"},"datePublished":"2023-07-21T05:23:40Z","url":"https://github.com/python/mypy/issues/15731"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:c473ee12-61f2-6041-1905-ad50a95abeed
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idCBA6:22FAD4:45D7A9:5D9936:6B002C8B
html-safe-nonce84b265e664f5b4bd4a732a1659d00b1f72b50588409b781d673530df48400462
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDQkE2OjIyRkFENDo0NUQ3QTk6NUQ5OTM2OjZCMDAyQzhCIiwidmlzaXRvcl9pZCI6IjQxNjk4OTc2OTk4MTE3OTgxNTUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac0eb61afb462cc12ff1b53f4907df15211d8bf242ca1c81c997f01278d7c54296
hovercard-subject-tagissue:1815173691
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/15731/issue_layout
twitter:imagehttps://opengraph.githubassets.com/7224fa26e60feb9e5fc8e9ecb546c5a7526012fca1c6f234b452fb168519b959/python/mypy/issues/15731
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/7224fa26e60feb9e5fc8e9ecb546c5a7526012fca1c6f234b452fb168519b959/python/mypy/issues/15731
og:image:altReducing Mypy's Cache Size This is a meta-issue talking about different ways to reduce Mypy's cache size. I've been working on a branch in my free time, though it is probably too big for a PR, so I...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamedosisod
hostnamegithub.com
expected-hostnamegithub.com
None7d0829bbde9e48b2486cb5dd6d147896eb01d32134627091f818958a935ef3f0
turbo-cache-controlno-preview
go-importgithub.com/python/mypy git https://github.com/python/mypy.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id7053637
octolytics-dimension-repository_nwopython/mypy
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id7053637
octolytics-dimension-repository_network_root_nwopython/mypy
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasef3f3a43ca095cdea3abbb394112e971f9fdb94fa
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/mypy/issues/15731#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F15731
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F15731
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fmypy
Reloadhttps://github.com/python/mypy/issues/15731
Reloadhttps://github.com/python/mypy/issues/15731
Reloadhttps://github.com/python/mypy/issues/15731
pythonhttps://github.com/python
mypyhttps://github.com/python/mypy
Codehttps://github.com/python/mypy
Issues2.7k (2.7k)https://github.com/python/mypy/issues
Pull requests547 (547)https://github.com/python/mypy/pulls
Actionshttps://github.com/python/mypy/actions
Projectshttps://github.com/python/mypy/projects
Wikihttps://github.com/python/mypy/wiki
Security and qualityhttps://github.com/python/mypy/security
Insightshttps://github.com/python/mypy/insights
https://github.com/python/mypy
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/15731
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/15731
Feature: Reduce Mypy's Cache Sizehttps://github.com/python/mypy/issues/15731#top
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
https://github.com/dosisod
dosisodhttps://github.com/dosisod
on Jul 21, 2023https://github.com/python/mypy/issues/15731#issue-1815173691
branchhttps://github.com/dosisod/mypy/tree/shrink-cache
dosisodhttps://github.com/dosisod
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
on Jul 21, 2023https://github.com/python/mypy/issues/15731#event-9881512033
https://github.com/ikonst
ikonsthttps://github.com/ikonst
on Jul 21, 2023https://github.com/python/mypy/issues/15731#issuecomment-1645644160
https://github.com/dosisod
dosisodhttps://github.com/dosisod
on Jul 21, 2023https://github.com/python/mypy/issues/15731#issuecomment-1646345301
https://github.com/dosisod
dosisodhttps://github.com/dosisod
on Aug 27, 2023https://github.com/python/mypy/issues/15731#issuecomment-1694584039
3240da4https://github.com/python/mypy/commit/3240da455e06a292669de22b0ef313dad43cb094
5328896https://github.com/python/mypy/commit/532889661f93d8afdeb9b2216fa00a4c4bd9290e
5328896https://github.com/python/mypy/commit/532889661f93d8afdeb9b2216fa00a4c4bd9290e
#932 (comment)https://github.com/python/mypy/issues/932#issuecomment-148726912
https://github.com/hauntsaninja
hauntsaninjahttps://github.com/hauntsaninja
on Aug 28, 2023https://github.com/python/mypy/issues/15731#issuecomment-1694839003
dosisodhttps://github.com/dosisod
on Aug 29, 2023https://github.com/python/mypy/issues/15731#event-10215387670
Compress cached JSON files:https://github.com/dosisod/mypy/commit/b2b636b54cf57807e272d844ac10310bbef02d8d
https://docs.github.com/articles/closing-issues-via-commit-messages
b2b636bhttps://github.com/dosisod/mypy/commit/b2b636b54cf57807e272d844ac10310bbef02d8d
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Aug 29, 2023https://github.com/python/mypy/issues/15731#issuecomment-1697440830
https://github.com/dosisod
dosisodhttps://github.com/dosisod
on Aug 30, 2023https://github.com/python/mypy/issues/15731#issuecomment-1698300205
010da0bhttps://github.com/python/mypy/commit/010da0b2f48dc92be2f79495fd4551c92351868f
936ba7dhttps://github.com/python/mypy/commit/936ba7d413a9aa8ab8521683fdae06eec1365936
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Feb 23, 2026https://github.com/python/mypy/issues/15731#issuecomment-3943914642
#20758https://github.com/python/mypy/pull/20758
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/mypy/issues/15731
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/mypy/issues/15731
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.