Title: Bench oracle: an absorbed marginal trace spot is only visible in a green CI step log · Issue #356 · purescript-lua/purescript-lua · GitHub
Open Graph Title: Bench oracle: an absorbed marginal trace spot is only visible in a green CI step log · Issue #356 · purescript-lua/purescript-lua
X Title: Bench oracle: an absorbed marginal trace spot is only visible in a green CI step log · Issue #356 · purescript-lua/purescript-lua
Description: Problem ./bench/ci compares freshly generated LuaJIT counter reports against committed oracles in bench/goldens/. One of those reports, the per-spec trace report, records which artifact source locations ended up carrying a compiled trace...
Open Graph Description: Problem ./bench/ci compares freshly generated LuaJIT counter reports against committed oracles in bench/goldens/. One of those reports, the per-spec trace report, records which artifact source loca...
X Description: Problem ./bench/ci compares freshly generated LuaJIT counter reports against committed oracles in bench/goldens/. One of those reports, the per-spec trace report, records which artifact source loca...
Opengraph URL: https://github.com/purescript-lua/purescript-lua/issues/356
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bench oracle: an absorbed marginal trace spot is only visible in a green CI step log","articleBody":"## Problem\n\n`./bench/ci` compares freshly generated LuaJIT counter reports against committed oracles in `bench/goldens/`. One of those reports, the per-spec trace report, records which artifact source locations ended up carrying a compiled trace or a blacklist, and whether a given location does is a per-process dice roll for spots sitting on LuaJIT's hot-count boundary. `bench/tools/diff_counters` absorbs that by re-measuring a report that differs from its golden only in the nondeterministic sets, passing if a fresh measurement matches.\n\nWhen it does absorb a flip it prints the entries that flickered, deliberately — a golden entry that is not decisive is the one property a reviewer cannot see by reading the committed goldens, so swallowing it silently would hide the fact that the oracle is pinning something marginal. But the notice goes to stderr and the script exits 0, so it lands in a *green* step's log:\n\n```\ntrace_ref_loop.txt: differs in the measured sets; re-measuring (1 of 2)\ntrace_ref_loop.txt: matched on re-measurement 1 -- the entries below are trace-formation noise, not a codegen change\n@@ -5,7 +5,6 @@\n Bench.RefLoop.lua:40 -- NYI: bytecode FNEW\n bytecode end state (J*=compiled, I*=blacklisted):\n Bench.RefLoop.lua:10 JFUNCF\n- Bench.RefLoop.lua:32 JFUNCF\n Bench.RefLoop.lua:34 JFORI\n Bench.RefLoop.lua:34 JFORL\n-counts: aborts=1 compiled=4 blacklisted=0\n+counts: aborts=1 compiled=3 blacklisted=0\n```\n\nNobody expands a green step. Measured at roughly 3% of runs on the current goldens, that means the signal is emitted a few times a year into a place no one reads.\n\n## Approach\n\nEmit a GitHub Actions annotation alongside the stderr notice, so an absorbed flip appears on the PR itself rather than only in the step log:\n\n```bash\nif [ -n \"${GITHUB_ACTIONS:-}\" ]; then\n printf '::warning::%s\\n' \"$name: absorbed a marginal trace spot on re-measurement $attempt\"\nfi\n```\n\nGuarded on `GITHUB_ACTIONS` so a local `./bench/ci` is unchanged.\n\nThe cost is that a vendor-neutral bench tool grows a branch that knows about one CI provider, which is why it was left out of the change that introduced the re-measurement rather than folded in. That is the decision to make here: accept the coupling for the visibility, or find a vendor-neutral equivalent (for instance, having `bench/ci` exit with a distinct non-failing status or write a marker file that a workflow step turns into an annotation, keeping the provider knowledge in the workflow).\n\n## Prerequisites / Relations\n\nIndependent; follows the re-measurement introduced in #346. Confined to `bench/` plus possibly `.github/workflows/ci.yaml`; no compiler code involved.\n\n## Acceptance criteria\n\n- An absorbed marginal trace spot is visible without expanding a green CI step's log.\n- A local `./bench/ci` run's output is unchanged.\n- Whichever way the vendor coupling is resolved is recorded next to the existing explanation in `bench/README.md`.\n","author":{"url":"https://github.com/Unisay","@type":"Person","name":"Unisay"},"datePublished":"2026-07-29T17:11:36Z","url":"https://github.com/purescript-lua/purescript-lua/issues/356"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:99759dd0-7939-0bd2-c6a7-a4c153632a87 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | C186:2E124E:4BAA53:621480:6B00D93D |
| html-safe-nonce | e7443d182bd6ab018e5febd9e5ebc319c7505eee07369e4939514c042cb4e724 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDMTg2OjJFMTI0RTo0QkFBNTM6NjIxNDgwOjZCMDBEOTNEIiwidmlzaXRvcl9pZCI6IjQwNDg2ODk2NzI1NjQ1NjIyMzciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | fb8dd9f41d8d7b1cd237ce406e8471b6a970489fb9fc2cb8c708b36251e0b4e3 |
| hovercard-subject-tag | issue:5011833782 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/purescript-lua/purescript-lua/356/issue_layout |
| twitter:image | https://opengraph.githubassets.com/70b15749c2d63492dcb232a9805a5873cca8e624329835c3f65b741ad9caa6c7/purescript-lua/purescript-lua/issues/356 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/70b15749c2d63492dcb232a9805a5873cca8e624329835c3f65b741ad9caa6c7/purescript-lua/purescript-lua/issues/356 |
| og:image:alt | Problem ./bench/ci compares freshly generated LuaJIT counter reports against committed oracles in bench/goldens/. One of those reports, the per-spec trace report, records which artifact source loca... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | Unisay |
| hostname | github.com |
| expected-hostname | github.com |
| None | a481f62a1e04c9b9bdcb5628230682c66d59006f5b13b080f79cba84a176905b |
| turbo-cache-control | no-preview |
| go-import | github.com/purescript-lua/purescript-lua git https://github.com/purescript-lua/purescript-lua.git |
| octolytics-dimension-user_id | 131581790 |
| octolytics-dimension-user_login | purescript-lua |
| octolytics-dimension-repository_id | 587200487 |
| octolytics-dimension-repository_nwo | purescript-lua/purescript-lua |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 587200487 |
| octolytics-dimension-repository_network_root_nwo | purescript-lua/purescript-lua |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 84c8c5cc25e2fb7f2308428458538436baecc6d2 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width