René's URL Explorer Experiment


Title: improvement(ci): collect idle node_modules sticky disks at 3 days by waleedlatif1 · Pull Request #7513 · simstudioai/sim · GitHub

Open Graph Title: improvement(ci): collect idle node_modules sticky disks at 3 days by waleedlatif1 · Pull Request #7513 · simstudioai/sim

X Title: improvement(ci): collect idle node_modules sticky disks at 3 days by waleedlatif1 · Pull Request #7513 · simstudioai/sim

Description: What's going on The Docker layer-cache caps we shipped last week worked — layer caches gave back 431 GB. But total storage barely moved, because node_modules sticky disks grew +350 GB over the same period and ate the win. They're now 44 disks / 601 GB (~$305/mo) and the single largest storage line, with total storage back over the 500 GB quota. Why they grew — checked, not assumed Two possible causes, and they need different fixes: Disks accumulating cruft over their life → the fix would be in test-build.yml, not retention. More disks alive at once → the fix is retention. It's the second. Disks created yesterday are already 14.8 GB, and the whole population now clusters tightly at 14.7–14.9 GB rather than the bimodal spread it had before. They're born that size — no accumulation. What changed is churn: bun.lock moved 24 times in 14 days on staging alone, and disks are created at ~6.3/day. Steady state is new disks/day × GB/disk × retention: predicted observed 7d (today) 606 GB 601 GB 3d (this PR) 260 GB — The model reproduces reality to within 1%, so the projection is trustworthy. Why 3 days Not a judgement call — the data has a cliff: Retention Disks kept GB kept Reclaimed now 7d 41 558 43 GB 4d 34 458 144 GB 3d 20 277 348 GB 2d 19 263 362 GB Going 4d→3d reclaims another 204 GB; going 3d→2d reclaims only 14 GB more. So 3 days captures essentially all of the win, and it clears a long weekend — a PR whose last CI run was Friday afternoon still hits a warm disk on Monday. Expected: ~$173/mo steady-state, ~$176/mo reclaimed on the first run. Why age-based and never PR-triggered The key contains no PR identifier. Every open PR whose checkout has the same bun.lock mounts the same disk — there are far more open PRs than distinct lockfile hashes (28 hashes across 44 disks), so sharing is the common case. A pull_request: [closed] hook would destroy a disk many other open PRs are actively using, most aggressively for the most-shared disk. Safety — every guard was executed, not reasoned about Test Result Normal run at 3d against the live listing 24/44 selected, 348 GB retention_days: 0 refused retention_days: "3d" (non-numeric) refused Every disk crafted to look idle refused — that's a listing/clock fault, not idleness A turbo-cache-build-push key injected into the listing 0 selected Two independent filters: a server-side --search, then a local regex re-proving the full key shape. The event segment is [a-z_]+ rather than an enumerated push|pull_request — the key interpolates ${{ github.event_name }}, and a workflow_dispatch disk already exists that an enumerated list would have skipped forever. Other choices worth a look in review: GitHub-hosted runner, so collection keeps working during a Blacksmith outage or CI_PROVIDER break-glass switch — exactly when disks idle and still bill. CLI pinned by version + SHA256, verified against the vendor's published .sha256 sidecar, not curl | sh; the job holds an org-wide token. Auto-update disabled so the pin holds. permissions: {}, no checkout. Group by key before the staleness test, since delete without --arch removes every arch variant. Worst case of a wrong delete is one cold bun install — and the bun cache disk is not lockfile-keyed, so it never idles, is never collected, and a refill links from a warm store with no network fetch. Setup required before this does anything A GitHub org admin runs blacksmith org-token create --label ci-stickydisk-gc --organization simstudioai (printed once). Add it as repository secret BLACKSMITH_CLI_TOKEN. The job fails with a clear message if it's missing. Must land on main — schedule only runs the default-branch copy. First run via Actions → Sticky Disk GC → Run workflow → dry_run = true; confirm every listed key is a node-modules- one. Type of Change Improvement Testing actionlint clean including its shellcheck pass. The selection logic was executed against the live account listing and against crafted listings for each guard (table above). Pinned CLI URL returns 200 and its SHA256 matches the published sidecar. Diff is one YAML file with no JS, so the repo audits run in CI. Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Open Graph Description: What's going on The Docker layer-cache caps we shipped last week worked — layer caches gave back 431 GB. But total storage barely moved, because node_modules sticky disks grew +350 GB over the ...

X Description: What's going on The Docker layer-cache caps we shipped last week worked — layer caches gave back 431 GB. But total storage barely moved, because node_modules sticky disks grew +350 GB over ...

Opengraph URL: https://github.com/simstudioai/sim/pull/7513

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:d3c86346-70f3-7de1-3cf6-8c7a9aeded9a
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idA480:63A32:24818F:337A05:6AFB51DB
html-safe-nonce3509b38980ad279db466ae4b85a8e3cd1533243e08615c9880ceb3229f71c754
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBNDgwOjYzQTMyOjI0ODE4RjozMzdBMDU6NkFGQjUxREIiLCJ2aXNpdG9yX2lkIjoiNjI0NTczOTI3ODk3NDUzODIwMyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac068ab59dff12f88fe02d96b0724832dab6c74a4cedd8f8d58d4db76434d5884f
hovercard-subject-tagpull_request:4448448787
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/7513/files
twitter:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
og:image:altWhat's going on The Docker layer-cache caps we shipped last week worked — layer caches gave back 431 GB. But total storage barely moved, because node_modules sticky disks grew +350 GB over the ...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/simstudioai/sim git https://github.com/simstudioai/sim.git
octolytics-dimension-user_id199344406
octolytics-dimension-user_loginsimstudioai
octolytics-dimension-repository_id912559512
octolytics-dimension-repository_nwosimstudioai/sim
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id912559512
octolytics-dimension-repository_network_root_nwosimstudioai/sim
turbo-body-classeslogged-out env-production page-responsive
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasecbf66c024bb6d10e6955baa7413c54cf13a201bb
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/simstudioai/sim/pull/7513/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7513%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7513%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=simstudioai%2Fsim
Reloadhttps://github.com/simstudioai/sim/pull/7513/files
Reloadhttps://github.com/simstudioai/sim/pull/7513/files
Reloadhttps://github.com/simstudioai/sim/pull/7513/files
simstudioaihttps://github.com/simstudioai
simhttps://github.com/simstudioai/sim
Codehttps://github.com/simstudioai/sim
Issues129 (129)https://github.com/simstudioai/sim/issues
Pull requests197 (197)https://github.com/simstudioai/sim/pulls
Actionshttps://github.com/simstudioai/sim/actions
Projectshttps://github.com/simstudioai/sim/projects
Security and qualityhttps://github.com/simstudioai/sim/security
Insightshttps://github.com/simstudioai/sim/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
waleedlatif1https://github.com/waleedlatif1
staginghttps://github.com/simstudioai/sim/tree/staging
ci/reclaim-node-modules-diskshttps://github.com/simstudioai/sim/tree/ci/reclaim-node-modules-disks
Conversation 9 https://github.com/simstudioai/sim/pull/7513
Commits 2 https://github.com/simstudioai/sim/pull/7513/commits
Checks 30 https://github.com/simstudioai/sim/pull/7513/checks
Files changed https://github.com/simstudioai/sim/pull/7513/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7513/files
improvement(ci): collect idle node_modules sticky disks at 3 days https://github.com/simstudioai/sim/pull/7513/files#top
Show all changes 2 commits https://github.com/simstudioai/sim/pull/7513/files
3629e36 improvement(ci): collect idle node_modules sticky disks at 3 days waleedlatif1 Sep 5, 2026 https://github.com/simstudioai/sim/pull/7513/commits/3629e36f43fdfb9a69e6ad024395e63818c83a07
ff61d54 fix(ci): collect fork disks, and count like with like in the bulk guard waleedlatif1 Sep 5, 2026 https://github.com/simstudioai/sim/pull/7513/commits/ff61d54495bbab3706cdb49e11d72b56929f2bbb
Clear filters https://github.com/simstudioai/sim/pull/7513/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7513/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7513/files
.github/workflows/stickydisk-gc.ymlhttps://github.com/simstudioai/sim/pull/7513/files#diff-043f9e6aa4eca0e375528c97dfb85ebe1c600fa19e443971617ed2e9741fe875
View file https://github.com/simstudioai/sim/blob/ff61d54495bbab3706cdb49e11d72b56929f2bbb/.github/workflows/stickydisk-gc.yml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7513/{{ revealButtonHref }}
Please reload this pagehttps://github.com/simstudioai/sim/pull/7513/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.