René's URL Explorer Experiment

Title: Build improvements by sheetalkamat · Pull Request #48784 · microsoft/TypeScript · GitHub

Open Graph Title: Build improvements by sheetalkamat · Pull Request #48784 · microsoft/TypeScript

X Title: Build improvements by sheetalkamat · Pull Request #48784 · microsoft/TypeScript

Description: Commits: 16cef4a, 59ad6ef, c8327da During tsc --build and other scenarios where we would do existence check as well as stat which resulted in two stat calls, we just check modified time Commits: cb7aca3 In tsc --build while determining upto date ness status, we now check if referenced projects have errors so that we dont need to go through input and output timestamps in that case Commits: 89d2d4c During watch especially during polling file watch and in some scenarios during native file-system based watch we stat on files and that time is now passed through as part of file watcher event so tsc --build can use it. Commits: 6e0c916 Apart from this in watch mode we save timestamps for all the watched files. These get updated through watcher event callback or are repopulated if watcher event didn't query the time. So if 2 files changed out of 20 input files then at max we will requery timestamps for those two files (if watch callback didn't already give us updated time) Fixes #45082 Commits: 7cb0f40, 1a8abac, 59f2b5c, dc21283, 5bccee8, If its incremental build, tsbuildInfo alone determines upto date ness of the build. For this we now store changeFileSet, dtsChangeTime and emitSignatures for the output files in the buildInfo when in multi file emit scenario. We store fileVersions, outSignature, dtsChangeTime in single file output scenario. The emitSignatures and outSignature are the d.ts hash of the output that last changed to the disk. Since any composite project already has declaration as must we can store this easily and save having to read files on emit to see if the d.ts file has changed. This makes sure that whether you are in tsc --build or tsc --build --watch mode the state is persisted so we know exactly if we need to really build or just update timestamps. In case of multi file emit scenario, tsbuildinfo timestamp, changeFileSet and errors in the buildInfo determine whether we need to rebuild project or not. This way we dont have to update timestamps for anything except buildInfo when thing appear uptodate but timestamp doesnt reflect that. (eg upsteam project changed but didnt change d.ts files) In case of single file emit scenario, tsbuildInfo timestamp is enough because we cant emit tsbuildinfo without emitting other output files. Earlier we checked input file timestamps and found newest file, now in case of incremental build, we first check buildInfo as missing it, or errors is clear indication of requiring rebuild. After that we check each input file timestamp against buildInfo so that the number of checks can be smaller. Fixes #46677 Fixes #46661 Commits: 2f2e370, fcf07f8, 15fe24e In non incremental project, we still need to update timestamps for output files that didnt change as there is no other way to tell if project is upto date or not. But we dont need to read d.ts files to see if .d.ts files have chaned since these projects cannot be referenced by other projects. (composite also means incremental which is must for referenced proejcts) Commits: 6198fa3, With so much depending on buildInfo, it is now cached and passed through emit as additional data so we dont have to reparse the json or identify which file is tsbuildinfo Commits: 62c687b We store the hash of files written in bundle emit to ensure that when we try to manipulate the sources for pseudo update (because d.ts of referenced project didnt change so only prepends need to concatenated in right order), we can check if file hasnt changed. This is now needed because bundle buildInfo also stores .d.ts change time and if incremental was toggled between compilations we might use wrong file text and get incorrect d.ts Look at 62c687b for details and 62c687b#diff-6b83fa9d09e38f88c61b24a70aec8668c68e0d8322e8404dbd85c90120f14eeeL1016 shows test that failed without this change Commits: 4fb6773 We also cache output file timestamps so that we dont have to requery as we are the one who write those files. Commits: 0f7903d, 437619e This also has some refactoring of BuilderState, BuilderProgramState etc structures so its correctly reflected. With this change we also have flags for emitting compiler option key into the buildInfo based on multi file or single file emit Commits: e6a3ee8 tsc --build --force now doesnt query input file stamps (it never did query output file stamps) Commits: b32d2eb In builder, earlier we use to copy too many structures as part of backup which is internal method to ensure we can restore state if declaration emit fails. So now we are only storing emit state instead of reference map etc Commits: 303824e Other change in builder is that earlier we use to maintain new delta of changes for affected files new signatures as well as new exported modules map. We use to commit it once all the affected files for the changed file is handled. (This was due to our API usage where we could cancel the operations in between and would still need to be able to reuse the state as much as possible). Now i have changed this to instead maintain copy of old signatures and old export map so that all we do is clear these these maps instead of having iterate through each and update them into the latest copy on commit Commits: 28a9ff3 During tsc --build if file timestamp is changed but its text has not, we read the file and do pseudo update (only in incremental build). Fixes #31932 Commits: e4e6672 - Delete only tsbuildinfo instead of all output files when doing clean on incremental project Here are the numbers from run on solution with large (1994) number of projects.   main PR Delta % Projects Built Timestamps Updated Total projects tsc -b 120.21 93.38 -26.83 -22.32% 255 0 1994 tsc -b (Fix one error) 39.13 31.50 -7.63 -19.50% 22 0 1994 tsc -b (Fix multiple errors in files) 43.96 35.74 -8.22 -18.70% 43 0 1994 tsc -b -w 373.25 326.22 -47.03 -12.60% 484 0 1994 Fix error 19.69 17.76 -1.93 -9.80% 11 0 1994 Fix error 61.48 60.71 -0.77 -1.25% 66 0 1994 Fix error 11.53 8.97 -2.56 -22.20% 3 0 1994 Fix error 7.19 6.18 -1.01 -14.05% 2 0 1994 tsc -b 620.07 480.23 -139.84 -22.55% 796 0 1994 tsc -b No change 26.78 19.59 -7.20 -26.87% 6 0 1994 Local Change 36.70 20.14 -16.57 -45.13% 7 477 1994 D.ts change Average 74.60 54.47 -20.12 -26.98% 138 346 1994 tsc -b -w 630.56 557.41 -73.15 -11.60% 796 0 1994 Local Change 8.98 7.52 -1.46 -16.21% 5 131 1994 D.ts change Average 89.21 70.56 -18.65 -20.90% 136 133 1994 tsc -b -w no change 37.58 26.37 -11.21 -29.83% 6 0 1994 On Typescript Code base: Total Projects In the main PR Delta % Projects Built Timestamps Updated Bundles updated Total Project tsc -b 60.73 49.03 -11.70 -19.26% 18 0 0 19 tsc -b no change 0.15 0.09 -0.06 -41.56% 0 0 0 19 tsc -b local change 33.42 26.17 -7.25 -21.68% 1 4 9 19 tsc -b dts change 58.87 45.63 -13.24 -22.48% 14 0 0 19

Open Graph Description: Commits: 16cef4a, 59ad6ef, c8327da During tsc --build and other scenarios where we would do existence check as well as stat which resulted in two stat calls, we just check modified time Commits: ...

X Description: Commits: 16cef4a, 59ad6ef, c8327da During tsc --build and other scenarios where we would do existence check as well as stat which resulted in two stat calls, we just check modified time Commits: ...

Opengraph URL: https://github.com/microsoft/TypeScript/pull/48784

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width