Title: perf(ci): scope the Docker layer cache per image and platform by waleedlatif1 · Pull Request #6501 · simstudioai/sim · GitHub
Open Graph Title: perf(ci): scope the Docker layer cache per image and platform by waleedlatif1 · Pull Request #6501 · simstudioai/sim
X Title: perf(ci): scope the Docker layer cache per image and platform by waleedlatif1 · Pull Request #6501 · simstudioai/sim
Description: app.Dockerfile's deps stage rebuilds on every build (~300-465s, the longest step in CI) even when no dependency changed. The cause is not the Dockerfile: it is that setup-docker-builder@v1 keys the Blacksmith sticky disk on the repository name alone (stickyDiskKey = GITHUB_REPO_NAME), so every Docker build in the repo shares one disk. What that means in practice, from the job logs: all five matrix jobs clone the same parent snapshot (Sticky disk parent snapshot: commit-01KZP43K…, byte-identical across the app/db/realtime/pii/cron jobs of run 31416995583), and at most one commit per wave becomes the next parent. Matching the 11 snapshot heads minted over 08-07/08-08 against job completion times, 9 of them landed within 2-4s of the fastest job finishing (pii, ~23s; cron, ~18s) and only 2 came from an app build. It is first-writer-wins against the parent generation, not last-writer-wins, which is exactly why the app build (637-919s) essentially never survives. Run 31416995583 cloned a head minted at 15:19:23 by a pii job, discarding the app builds that finished at 15:29, 15:59, 16:37 and 17:41 in between. The fix derives a distinct cache-key per Dockerfile per platform and moves to setup-docker-builder@v2, where cache-key exists as a required input: - name: Resolve Docker layer cache key id: cache-key if: inputs.provider == '' || inputs.provider == 'blacksmith' shell: bash env: FILE: ${{ inputs.file }} PLATFORMS: ${{ inputs.platforms }} run: echo "value=${GITHUB_REPOSITORY##*/}/${FILE#./}/${PLATFORMS//\//-}" >> "$GITHUB_OUTPUT" - name: Set up Blacksmith builder if: inputs.provider == '' || inputs.provider == 'blacksmith' uses: useblacksmith/setup-docker-builder@a5256a73e30f09e37e3eceb8ca36043d17621d24 # v2 with: cache-key: ${{ steps.cache-key.outputs.value }} That yields sim/docker/app.Dockerfile/linux-amd64, sim/docker/db.Dockerfile/linux-amd64, sim/docker/app.Dockerfile/linux-arm64, and so on: ten keys, one per concurrent builder, so no two jobs that run at the same time ever contend for the same snapshot lineage again. Computing it inside the composite means all three call sites (build-dev, build-amd64, build-ghcr-arm64) are covered with no caller changes and no way for a new call site to drift. Platform is part of the key deliberately. On main, build-amd64 and build-ghcr-arm64 build the same Dockerfiles concurrently; keying only by Dockerfile would reproduce the identical clobbering between them, and amd64/arm64 share no layers, so splitting costs nothing. The deps layer is genuinely cacheable, which is what makes this worth fixing: re-running turbo prune sim --docker at both reference commits (156ee3eb -> 5da48d01) produces byte-identical out/json and bun.lock, so the only reason #11 COPY --from=pruner /app/out/json/ ./ misses is that the previous app build's records are not in the snapshot this build cloned. Nothing was evicting, so this is not a pruning problem: the teardown log reports Skipping BuildKit cache pruning (max-cache-size-mb not set) with 91.77 GB retained and fully reclaimable, including five separate ~2.8 GB deps layers from five earlier app builds, each with USAGE COUNT 1. max-cache-size-mb no longer exists on v2; the cache is garbage-collected automatically (layers unused for 8 days). Worth watching separately: org sticky disk usage is currently 942 GB against a 500 GB allowance. The first build on each new key is cold, and there are ten of them. Everything after that hits a cache that only its own image writes to. The other Blacksmith runners on blacksmith-* labels and the docker/* fallback path for CI_PROVIDER != blacksmith are untouched; the new step carries the same predicate as the builder step it feeds.
Open Graph Description: app.Dockerfile's deps stage rebuilds on every build (~300-465s, the longest step in CI) even when no dependency changed. The cause is not the Dockerfile: it is that setup-docker-builder@v1 keys...
X Description: app.Dockerfile's deps stage rebuilds on every build (~300-465s, the longest step in CI) even when no dependency changed. The cause is not the Dockerfile: it is that setup-docker-builder@v1 ...
Opengraph URL: https://github.com/simstudioai/sim/pull/6501
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:1bb4c210-f0eb-ba92-a8ef-ebfb21c4b3e3 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | A328:3A88AC:6476AE:8F9DB9:6AFB42BB |
| html-safe-nonce | 8161cec11ad2766cebee29357ab6b7e9f067cd52167beece1a10b1a1fbbc09b3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMzI4OjNBODhBQzo2NDc2QUU6OEY5REI5OjZBRkI0MkJCIiwidmlzaXRvcl9pZCI6IjE5NTM1NjAzMzU4NzI2Mzk2NzUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 1dec7697dc3afb64365d519e27444f9e51bfdd9da93d12a08a1dc1ab8a4d013b |
| hovercard-subject-tag | pull_request:4247916448 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/6501/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | app.Dockerfile's deps stage rebuilds on every build (~300-465s, the longest step in CI) even when no dependency changed. The cause is not the Dockerfile: it is that setup-docker-builder@v1 keys... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | d849b928f455c800985398e7f1b659425adb312a5fa14fba5d9a14e62e9fe5dd |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | a6c2f87cbcb6414452377b4967858ed6a66f1109 |
| ui-target | canary-2 |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width