Title: docs(run-workflow): explain that {{KEY}} in run output is a redaction by mzxchandra · Pull Request #11 · simstudioai/skills · GitHub
Open Graph Title: docs(run-workflow): explain that {{KEY}} in run output is a redaction by mzxchandra · Pull Request #11 · simstudioai/skills
X Title: docs(run-workflow): explain that {{KEY}} in run output is a redaction by mzxchandra · Pull Request #11 · simstudioai/skills
Description: Problem A {{KEY}} reference resolves to the real secret value at execution. Sim then builds a separate log-facing copy in which exact occurrences of that value are masked — and the mask it writes is the {{KEY}} label itself. That masked copy is what workflows runs get and logs get return. Coding agents driving the sim CLI read it, see {{OPENAI_API_KEY}} where a value should be, and conclude the reference never resolved. They then "fix" a workflow that already works: rewriting {{KEY}} into environmentVariables.KEY, hardcoding a literal, or adding a print to prove resolution. This has been observed repeatedly in practice. No skill in the repo mentioned masking at all — redact, masked, and scrub had zero hits. Verified end to end against a live workflow before writing it up: Function block code Run status What the trace shows {{ANTHROPIC_API_KEY}} completed, len: 108 {{ANTHROPIC_API_KEY}} {{ZZ_DOES_NOT_EXIST_XYZ}} completed, len: 25 {{ZZ_DOES_NOT_EXIST_XYZ}} A resolved secret and a name that never existed render identically, both in a successful run. Why the claim is hedged An adversarial pass killed three stronger claims from the first draft, all checked against the sim source: unredacted: false does not mean "this is a mask." MIN_SUBSTITUTABLE_LITERAL_LENGTH is 8, so a secret shorter than 8 characters is never masked at all and its {{KEY}} is always an unresolved reference. secrets list proves a name exists, never that it resolved in this run. The original line would have told an agent to leave a genuinely broken reference in place. Singling out --follow implied the default run output is masked. It is not — a plain run and a --follow stream both POST the same executeWorkflow operation, and the execute route contains no redaction call. The section now says every live run response is unmasked and only runs get / logs get return the masked copy. "Passes through unchanged" is wrong for one of three languages. In a shell block an unresolved placeholder becomes the empty string (shell.ts, resolved ? shellExpansion(...) : ''); JavaScript and Python do leave it literal. Net effect is a weaker but true claim: a {{KEY}} in a masked log is usually a resolved secret, not proof of one — and the section says what to check when it isn't. Scope 13 lines added to skills/run-workflow/SKILL.md. Body only — the frontmatter description is unchanged, since this is about behaviour once the skill is routed to, not about routing. Merged main (the new "four properties of runs and run records" section) and kept both: those bullets are an unheaded continuation of ## Diagnose failures, so they stay attached to the numbered steps and the masking section follows as a peer H2. No contradiction — main's first bullet notes a completed run can sit in redacting status before its content is readable, which is the same pipeline this section describes once it is readable. bun run check:skills, bun run test, and bun run type-check all pass.
Open Graph Description: Problem A {{KEY}} reference resolves to the real secret value at execution. Sim then builds a separate log-facing copy in which exact occurrences of that value are masked — and the mask it writes i...
X Description: Problem A {{KEY}} reference resolves to the real secret value at execution. Sim then builds a separate log-facing copy in which exact occurrences of that value are masked — and the mask it writes i...
Opengraph URL: https://github.com/simstudioai/skills/pull/11
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:768dfec5-3b20-26c8-2ae4-e6aee88b29af |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | C7CA:FBC06:D122B3:1205E79:6AFB5BF1 |
| html-safe-nonce | 274b7467b2d1b0f97f5a9f39f55b8c977804ddd05ed50810f4c181fd5b1df1be |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDN0NBOkZCQzA2OkQxMjJCMzoxMjA1RTc5OjZBRkI1QkYxIiwidmlzaXRvcl9pZCI6Ijc1NTk4NjM2MDY0MTg1NjIwMzMiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 64efea22939851fe17c0643942e2da32be7c4fb446b83d4f5bc756fe230a330c |
| hovercard-subject-tag | pull_request:4531874209 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/skills/pull/11/files |
| twitter:image | https://avatars.githubusercontent.com/u/129460234?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/129460234?s=400&v=4 |
| og:image:alt | Problem A {{KEY}} reference resolves to the real secret value at execution. Sim then builds a separate log-facing copy in which exact occurrences of that value are masked — and the mask it writes i... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | ea33d13c96d0ff66bd8fa906b193d2213536b78cb06b0c50a4632c58b1418f94 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/skills git https://github.com/simstudioai/skills.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 1348964237 |
| octolytics-dimension-repository_nwo | simstudioai/skills |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 1348964237 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/skills |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 59ef97164c52cf59bd10558ddec9c4f482763988 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width