Title: Exponential increase in compile time and memory usage · Issue #38339 · microsoft/TypeScript · GitHub
Open Graph Title: Exponential increase in compile time and memory usage · Issue #38339 · microsoft/TypeScript
X Title: Exponential increase in compile time and memory usage · Issue #38339 · microsoft/TypeScript
Description: This bug report is a follow-up to this tweet where I write about OOM crashes in 3.9.1-rc. Daniel Rosenwasser (@DanielRosenwasser) asked me to report an issue, but I find it really hard to narrow it down. I have a project with ~2k lines o...
Open Graph Description: This bug report is a follow-up to this tweet where I write about OOM crashes in 3.9.1-rc. Daniel Rosenwasser (@DanielRosenwasser) asked me to report an issue, but I find it really hard to narrow it...
X Description: This bug report is a follow-up to this tweet where I write about OOM crashes in 3.9.1-rc. Daniel Rosenwasser (@DanielRosenwasser) asked me to report an issue, but I find it really hard to narrow it...
Opengraph URL: https://github.com/microsoft/TypeScript/issues/38339
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Exponential increase in compile time and memory usage","articleBody":"> This bug report is a follow-up to [this tweet](https://twitter.com/squidfunk/status/1256149893007147009) where I write about OOM crashes in 3.9.1-rc. @DanielRosenwasser asked me to report an issue, but I find it really hard to narrow it down.\r\n\r\nI have a project with ~2k lines of code. It uses another internal package with ~18k lines of code. The problem is that the compiler doesn't terminate anymore. I then tried to compile a subset of the project (which is a recursive decent parser, BTW), which runs in under 6 seconds. If I add more and more files, it will get slower and slower to a point where it will run into OOM. The interesting thing is that there's no big bump in files, lines, types, instantiations or symbols:\r\n\r\n```\r\nFiles: 1588 1597 1600\r\nLines: 65199 80674 81049\r\nNodes: 278678 349628 350339\r\nIdentifiers: 90620 91032 91288\r\nSymbols: 64556 465900 478162\r\nTypes: 3027 109800 134328\r\nInstantiations: 31435 115295 222586\r\nMemory used: 125785K 295562K 952063K\r\nI/O read: 0.19s 0.13s 0.90s\r\nI/O write: 0.02s 0.05s 0.06s\r\nParse time: 3.78s 3.27s 4.77s\r\nBind time: 0.80s 0.78s 1.07s\r\nCheck time: 0.96s 5.37s 94.43s\r\nEmit time: 0.25s 0.74s 0.77s\r\nTotal time: 5.79s 10.15s 101.04s\r\n```\r\n\r\nYes, there are twice as many instantiations, which I assume results from the use of rather large discriminated unions (.e.g when restructuring them), but the increase in memory is almost three times as high. The CPU profile when compiling looks like this:\r\n\r\n\r\n\r\nMy current guess is that the exponential increase is somehow related to __Instantiations ⨉ Instantiations__, as most of the time is spent in `someTypeRelatedToType`.\r\n\r\nI tried breaking it down to a reproducible case, but as the project with which I'm experiencing this issue is not Open Source, stripping everything down takes too much time. I understand that this is not an actionable error report, but maybe there's an obvious reason why this recent change in behavior may be causing this. If not, feel free to close it\r\n\r\n**TypeScript Version:** 3.9.1-rc\r\n**Search Terms:** crash, out of memory, oom\r\n\r\n**Code** \r\n\r\nCould not be provided\r\n\r\n**Expected behavior:**\r\n\r\nTypeScript compiles the code\r\n\r\n**Actual behavior:**\r\n\r\nTypeScript runs out of memory\r\n\r\n**UPDATE**:\r\n\r\nI managed to get all packages to build with 3.9.1 by removing mapped discriminated union types, i.e. functions with signatures like `function\u003cT extends FooKind>(kind: T): FooLike\u003cT>`, whereas `FooLike\u003cT>` uses conditionals to select a specific type from a huge union with ~1k types. Build times now are:\r\n\r\n```\r\npackage-c 3.8.3 3.9.1\r\n\r\nFiles: 1626 1628\r\nLines: 84005 84000\r\nNodes: 357333 357139\r\nIdentifiers: 93411 93314\r\nSymbols: 488307 487405\r\nTypes: 128763 124300\r\nInstantiations: - 167777\r\nMemory used: 1092025K 979008K\r\nI/O read: 0.17s 0.33s\r\nI/O write: 0.13s 0.22s\r\nParse time: 3.29s 5.44s\r\nBind time: 1.20s 1.64s\r\nCheck time: 60.46s 92.90s\r\nEmit time: 2.41s 2.85s\r\nTotal time: 67.37s 102.82s\r\n\r\npackage-b 3.8.3 3.9.1\r\n\r\nFiles: 1588 1590\r\nLines: 64154 64149\r\nNodes: 276974 276780\r\nIdentifiers: 89681 89584\r\nSymbols: 64608 64983\r\nTypes: 3484 3621\r\nInstantiations: - 9232\r\nMemory used: 131259K 130999K\r\nI/O read: 0.25s 0.22s\r\nI/O write: 0.10s 0.08s\r\nParse time: 4.70s 4.62s\r\nBind time: 0.99s 1.14s\r\nCheck time: 0.99s 1.12s\r\nEmit time: 1.16s 1.38s\r\nTotal time: 7.83s 8.26s\r\n\r\npackage-a 3.8.3 3.9.1\r\n\r\nFiles: 1577 1579\r\nLines: 115025 115020\r\nNodes: 329307 329113\r\nIdentifiers: 88729 88632\r\nSymbols: 154484 151390\r\nTypes: 45980 45135\r\nInstantiations: - 36085\r\nMemory used: 372082K 274693K\r\nI/O read: 0.27s 1.54s\r\nI/O write: 3.98s 3.98s\r\nParse time: 6.17s 7.64s\r\nBind time: 1.63s 1.77s\r\nCheck time: 21.55s 4.72s\r\nEmit time: 13.94s 15.12s\r\nTotal time: 43.29s 29.25s\r\n```","author":{"url":"https://github.com/squidfunk","@type":"Person","name":"squidfunk"},"datePublished":"2020-05-05T14:06:41Z","url":"https://github.com/microsoft/TypeScript/issues/38339"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:4a5af765-f7a3-af8a-cbb8-9ab8cd88c76c |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | C8AC:3DA1E8:5DF99C:7D1FAA:6B01A235 |
| html-safe-nonce | ce7562a35ff1299905d2d8d1f0111ec28739dc8f57d72e5c697cdafb5512e58f |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDOEFDOjNEQTFFODo1REY5OUM6N0QxRkFBOjZCMDFBMjM1IiwidmlzaXRvcl9pZCI6IjI2NTI3NTc4MDE2NTg1ODk3NDkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | ab58f84ee864406a5e5e70ce8178b8f3ffa8efea707d2b9874ead410e575a4bd |
| hovercard-subject-tag | issue:612630795 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/TypeScript/38339/issue_layout |
| twitter:image | https://opengraph.githubassets.com/2dd576dce2e936da2d11dc6bd9883d97fe95884e6e038b2cca5716b061dc8dc6/microsoft/TypeScript/issues/38339 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/2dd576dce2e936da2d11dc6bd9883d97fe95884e6e038b2cca5716b061dc8dc6/microsoft/TypeScript/issues/38339 |
| og:image:alt | This bug report is a follow-up to this tweet where I write about OOM crashes in 3.9.1-rc. Daniel Rosenwasser (@DanielRosenwasser) asked me to report an issue, but I find it really hard to narrow it... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | squidfunk |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/TypeScript git https://github.com/microsoft/TypeScript.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 20929025 |
| octolytics-dimension-repository_nwo | microsoft/TypeScript |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 20929025 |
| octolytics-dimension-repository_network_root_nwo | microsoft/TypeScript |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width