René's URL Explorer Experiment

Title: improvement(ci): decouple image builds from the test gate by waleedlatif1 · Pull Request #5701 · simstudioai/sim · GitHub

Open Graph Title: improvement(ci): decouple image builds from the test gate by waleedlatif1 · Pull Request #5701 · simstudioai/sim

X Title: improvement(ci): decouple image builds from the test gate by waleedlatif1 · Pull Request #5701 · simstudioai/sim

Description: Problem Push-to-production-deploy takes ~13.5 minutes because the pipeline is fully serial: tests (5m22) → migrate (0m46) → AMD64 image builds (4m48–7m01) → ECR tag push → CodePipeline. The builds only start after the gate clears, though nothing about building requires it. (Timings from run 29459378914.) Design Standard "build once, promote by retag" artifact promotion: images build in parallel with tests under immutable sha tags; passing the gate promotes the exact tested artifact by moving a tag (docker buildx imagetools create, a seconds-long manifest copy) — never by rebuilding. Invariant: no mutable/documented tag moves, and nothing deploys, until tests + migrations pass. Only immutable sha tags exist before the gate. ci.yml build-amd64 and build-ghcr-arm64 start immediately and push only sha tags (ECR :, GHCR :-amd64 / :-arm64). This also closes a pre-existing hole: latest-arm64 used to be pushed before test results existed. New promote-images job (after test-build + migrate) retags sha → latest/staging in ECR — the tag CodePipeline's EventBridge rules trigger on. Single job, not a matrix: it verifies all four sha manifests exist before moving any tag, so a partial mixed-version deploy can't happen. create-ghcr-manifests (after promote + arm64 build) now owns every mutable GHCR tag: latest-{amd64,arm64}, latest, version tags, and the multi-arch sha manifest — all built from the immutable sha-arch tags. Stale-run guards on both promoting jobs: re-running an old run used to be able to retag latest back to old code and trigger an immediate prod deploy of it. Now a superseded commit skips mutable-tag moves with a warning (immutable sha/version tags still publish, so releases can't be blocked by a racing push). test-build.yml Tests (2m05) and next build (2m26) were serial in one job; now parallel jobs. They share the warm bun-cache/node_modules sticky disks (identical content; Blacksmith disks are clone-on-mount + last-writer-wins commit, so concurrent mounts are safe) — but the build job gets its own Turbo cache key, since with a shared key the two jobs' new entries would evict each other every run. Verification CodePipeline triggers confirmed tag-filtered in AWS (image-tag: ["latest"|"staging"|"dev"]) for all 9 pipelines — sha pushes trigger nothing ECR lifecycle policy expires images after 7 days → sha tags self-clean (the promote verify step also fails loudly if a sha image has expired rather than half-promoting) actionlint clean (except pre-existing runner-label/SC2086-style notes shared with the old file) Independent adversarial review of the full diff; all real findings fixed in the second commit No branch protection/rulesets pin the old job names PR CI run validates the test/build split (first run: green, jobs in parallel) Expected impact Path Before After PR feedback ~5m20 ~3m30 Push → prod deploy trigger ~13m30 ~7m (bounded by slowest image build) Full main run ~15m ~8m Accepted trade-off Untested immutable sha-arch tags now exist on public GHCR for commits whose tests later fail (already true for arm64 today). They're plumbing tags nothing documents or deploys, but GHCR has no lifecycle expiry — if we care about accumulation, a scheduled ghcr untagged/sha cleanup job is a small follow-up. Watch on merge The push path (sha push + promote) first executes on the merge to staging — watch that run's promote-images before this reaches main.

Open Graph Description: Problem Push-to-production-deploy takes ~13.5 minutes because the pipeline is fully serial: tests (5m22) → migrate (0m46) → AMD64 image builds (4m48–7m01) → ECR tag push → CodePipeline. The builds ...

X Description: Problem Push-to-production-deploy takes ~13.5 minutes because the pipeline is fully serial: tests (5m22) → migrate (0m46) → AMD64 image builds (4m48–7m01) → ECR tag push → CodePipeline. The builds ...

Opengraph URL: https://github.com/simstudioai/sim/pull/5701

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width