René's URL Explorer Experiment

Title: perf(ci): version refactor (remove Xref/status.sh) by davdhacs · Pull Request #19424 · stackrox/stackrox · GitHub

Open Graph Title: perf(ci): version refactor (remove Xref/status.sh) by davdhacs · Pull Request #19424 · stackrox/stackrox

X Title: perf(ci): version refactor (remove Xref/status.sh) by davdhacs · Pull Request #19424 · stackrox/stackrox

Description: Problem: -X ldflags break Go's build and test cache Go's build cache keys every action (compile, link, test) on an ActionID — a hash of all inputs that affect the output. The link ActionID includes the full -ldflags string (exec.go:linkActionID() — see fmt.Fprintf(h, "link\n") ... fmt.Fprintf(h, "ldflags %q\n", ldflags)). The old status.sh + //XDef: mechanism injected version data via -X ldflags: -X "github.com/stackrox/rox/pkg/version/internal.MainVersion=4.11.x-431-gbbcc7d3be2" -X "github.com/stackrox/rox/pkg/version/internal.GitShortSha=bbcc7d3be24" Since MainVersion and GitShortSha change on every commit, the ldflags string changes, which changes the link ActionID. This has two effects: Build cache miss at link step: every binary re-links on every commit, even if no source code changed. Go's compile cache uses dependency contentID (output hash) rather than actionID, so compile steps can still cache-hit. But the link step always re-runs because its ActionID includes the changed ldflags. On CI, this means ~100s spent re-linking binaries that haven't actually changed. Test cache miss at Phase 1: Go's test result cache computes a test binary identity from the test binary's build ActionID, which includes the link ActionID (test.go:builderTest() — the test binary hash flows through packageActionID() → linkActionID()). When the link ActionID changes, the test binary identity changes, and Go discards all cached test results — every test re-runs from scratch. Fix: generated zversion.go instead of -X ldflags Replace -X ldflags with a generated pkg/version/internal/zversion.go file that sets version data via init(). This eliminates -X from the link step entirely, making ldflags stable across commits (-s -w only). For tests, go-tool.sh writes only the base tag (e.g. "4.11.x") — stable across commits on the same release branch. For builds, it writes the full git describe output (e.g. "4.11.x-431-gbbcc7d3be2"). The generated zversion.go is a source file, so it flows through the compile ActionID (content-hashed). When its content doesn't change (test mode), the compile output is byte-identical, giving the same contentID, so all 4400+ downstream packages get cache hits at both compile and link. Only 2 packages recompile (the version packages themselves, whose output is byte-identical anyway). Result: build compilation step drops from 100s → 27s (3.7x), and this improvement works independently of the test cache fix. Companion PRs PR #19395: Phase 2 fix (stabilize file mtimes for test input validation). PR #19585: disable -buildvcs (split out from this PR). Test caching requires both Phase 1 (this PR) and Phase 2 (#19395). The build cache improvement from this PR works on its own. Prior art: PR #15609 PR #15609 was closed due to two blockers (comment). Both are now resolved: -coverprofile broke test caching (Go <1.25): fixed in Go 1.25 (golang/go#23565). We are on Go 1.25.5. release vs non-release GOTAGS sharing one cache: our research (PR #19576) showed they share 93% of entries — only 6/1952 packages have release build tags. Go correctly recompiles those when GOTAGS changes. Separating caches would duplicate 93% of entries. Measured results (both PRs combined) Job Master warm Combined warm Speedup go (GOTAGS="") 35m ~8m 4.5x go-postgres 31m ~36s 52x sensor-integration 26m ~2.5m 11x Build compilation step (this PR alone): 100s → 27s (3.7x), with 99.95% cache hit rate (2/4403 packages recompile). Testing and quality CI results are inspected Automated testing added unit tests added e2e tests added regression tests added compatibility tests modified existing tests How I validated my change Verified locally: go-build.sh generates full version, go-test.sh generates stable base tag Build produces correct version strings in binaries Prior CI runs confirmed 4.5-52x test speedup and 3.7x build speedup when combined with mtime fix Operator builds pass CI with the new BUILD_TAG mechanism

Open Graph Description: Problem: -X ldflags break Go's build and test cache Go's build cache keys every action (compile, link, test) on an ActionID — a hash of all inputs that affect the output. The link ActionID ...

X Description: Problem: -X ldflags break Go&#39;s build and test cache Go&#39;s build cache keys every action (compile, link, test) on an ActionID — a hash of all inputs that affect the output. The link A...

Opengraph URL: https://github.com/stackrox/stackrox/pull/19424

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width