Title: perf(tests): mock the tool registry globally, drop the dead deps optimizer by waleedlatif1 · Pull Request #6117 · simstudioai/sim · GitHub
Open Graph Title: perf(tests): mock the tool registry globally, drop the dead deps optimizer by waleedlatif1 · Pull Request #6117 · simstudioai/sim
X Title: perf(tests): mock the tool registry globally, drop the dead deps optimizer by waleedlatif1 · Pull Request #6117 · simstudioai/sim
Description: Summary The test suite spent far more time importing modules than running them — import was 1,399s aggregate against 88s of actual tests. This cuts the full suite 45%, with zero coverage lost. Lint and Test is currently the CI critical path (p50 189s vs Build App's 150s), and ~143s of it is the test run. Where the cost was — measured per area lib/core touches no registry and sets the floor at 0.09s/file. Everything that reaches the tool registry graph: area files import per-file × floor blocks 25 177.9s 7.12s 79× providers 53 193.2s 3.65s 41× executor 64 196.2s 3.07s 34× tools 78 159.9s 2.05s 23× app/api 204 214.7s 1.05s 12× lib/core 50 4.6s 0.09s — The tool registry is 4,351 entries pulling ~5,907 modules, and almost nothing under test needs the real thing. @/blocks/registry was already globally mocked for exactly this reason — this extends the same pattern to @/tools/registry. Result — full suite, same commit, same machine baseline after Duration 166.21s 91.56s −45% import 1399.17s 617.08s −56% transform 141.80s 61.98s −56% setup 259.00s 203.61s −21% Test Files 1 failed / 1252 passed 1 failed / 1252 passed identical Tests 1 failed / 16873 passed 1 failed / 16873 passed identical The single failure reproduces on unmodified staging — cloud-review-tools.test.ts can't find rg from its spawned python3 on macOS. CI installs ripgrep explicitly and it passes there. No coverage was removed Four files genuinely assert tool registration or tool params. They opt out with vi.unmock('@/tools/registry') rather than being weakened or deleted — outlook, azure_devops, and the two search-replace suites. Each carries a comment explaining why, so the pattern is discoverable for the next person who needs the real registry. These are exactly the tests that catch blocks and tools drifting apart, which is the kind of breakage that ships silently. Deleting them to make the number work would have been trading correctness for speed. deps.optimizer was dead config — measured, not assumed It was set to web, which only applies to client environments (jsdom/happy-dom), while 985 of 1,219 files declare @vitest-environment node. Vitest's docs are explicit that node/edge use ssr. I measured both ways rather than just "fixing" it to ssr: config Duration import baseline (web) 19.33s 135.30s removed 19.23s 136.37s switched to ssr + include list for openai / Anthropic / Google / Bedrock SDKs 19.19s 136.69s All identical within noise. The cost is first-party module graph, which the optimizer doesn't touch — it only bundles node_modules. So the honest move is deleting it, not leaving config that reads as if it does something. Also fixes a latent mock gap Adds the missing getBlockRegistry accessor to the @/blocks/registry mock. #6083 renamed that export and the mock was never updated, so any test reaching those three consumers would have hit getBlockRegistry is not a function. Nothing exercises them today — this closes it before someone trips on it. Rejected, with evidence isolate: false is ~20% faster (import 136.6s → 110.4s on a 145-file set) but leaks state between files — it broke two doc-servable tests on the first run that pass under isolation. Correctness over speed. Type of Change Performance Improvement (dead config removal, mock fidelity) Testing Full suite run both ways on the same machine (table above). tsc --noEmit exit 0. biome check clean. Each of the four opt-out files verified individually: outlook 21/21, azure-devops 28/28, indexer 58/58, replacements 32/32.
Open Graph Description: Summary The test suite spent far more time importing modules than running them — import was 1,399s aggregate against 88s of actual tests. This cuts the full suite 45%, with zero coverage lost. Lint...
X Description: Summary The test suite spent far more time importing modules than running them — import was 1,399s aggregate against 88s of actual tests. This cuts the full suite 45%, with zero coverage lost. Lint...
Opengraph URL: https://github.com/simstudioai/sim/pull/6117
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c48c3193-bb5e-9d8a-6831-bb49b610e904 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | B4E6:AF687:832243:B4F671:6AFB469E |
| html-safe-nonce | 3d48c7f46f120a17febd48ed764b3d12e64ba651e44ba240636413475a02cc2c |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNEU2OkFGNjg3OjgzMjI0MzpCNEY2NzE6NkFGQjQ2OUUiLCJ2aXNpdG9yX2lkIjoiMzY2MTgzMDQwNjE2MTUxNjE5MCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | f39410fc2dd5db8d414b41671437063aa01df068bb93ed16d8191b612c4fbc69 |
| hovercard-subject-tag | pull_request:4174600221 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/6117/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Summary The test suite spent far more time importing modules than running them — import was 1,399s aggregate against 88s of actual tests. This cuts the full suite 45%, with zero coverage lost. Lint... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | acf7616d06cc051644705b379878c9bb13dc5e70ecf4ffd9b34140f8ac01d7e1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 26504db649b58525278f77a52b80ba5f10d5f4de |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width