René's URL Explorer Experiment


Title: feat: validate documented experiment keys (DOCS-139) by nickvigilante · Pull Request #29316 · coder/coder · GitHub

Open Graph Title: feat: validate documented experiment keys (DOCS-139) by nickvigilante · Pull Request #29316 · coder/coder

X Title: feat: validate documented experiment keys (DOCS-139) by nickvigilante · Pull Request #29316 · coder/coder

Description: Summary The docs name specific experiment keys wherever an individual experimental feature is documented, and nothing verifies those keys still exist in the code. When an experiment is renamed or graduated, the enablement example keeps telling operators to set a key the server no longer accepts. This adds scripts/checkexperimentkeys, which fails when a key named in docs/** is absent from codersdk.ExperimentsKnown, and wires it into make lint as lint/check-experiment-keys. The data flows one way. The check reads keys the docs already name and validates them against code. It does not generate a page, add a manifest route, or write any key into docs/. That is deliberate: #29113 retired the generated feature-stage lists because the team decided not to advertise which features are currently experimental or beta. A guard against stale keys is compatible with that decision; an index is not. The five key references the check currently guards are the per-feature ones that decision left in place: chat-advisor, chat-virtual-desktop, auto-fill-parameters, agent-lifecycle-hooks, and mcp-server-http. Recognition rules Only explicit enablement forms are matched: --experiments= and CODER_EXPERIMENTS=, including every key in a comma-separated list. Values may be single- or double-quoted. Explicit shell line continuations are joined before matching, preserving the first physical line for a finding. Bare identifiers in prose are ignored on purpose. A check that guesses at prose produces false positives, and a check with false positives gets deleted by the first person it blocks. *, feature1, feature2, and angle-bracketed values are recognized placeholders. The recognized forms are documented in the script's README. Known gap: the space-separated --experiments form is not matched. No prose uses it today, so this is latent rather than broken, and closing it is a small follow-up if the form reappears. Also fixed ExperimentMCPToolSearch had no case in DisplayName(), so it rendered as a title-cased guess in the product UI as well as anywhere else display names surface. TestExperimentDisplayNames now pins the display name of every known experiment against an expected map, with a length assertion, so a new experiment cannot silently fall back to the title-cased key. Verification go test ./scripts/checkexperimentkeys ./codersdk passes. Coverage includes known and unknown keys, quoted values, shell continuations, placeholders, multi-key lists, bare-prose exclusion, and failure-output formatting. golangci-lint run ./scripts/checkexperimentkeys/... ./codersdk/... reports 0 issues. make lint/check-experiment-keys is clean against current docs/. Negative control: a fixture containing CODER_EXPERIMENTS=not-a-real-key exits 1 with the expected finding. The full pre-commit hook passes, including lint/check-experiment-keys inside aggregate make lint. History: the deferred lint wiring, now resolved This PR previously landed the tool and a standalone make target while leaving the one-line addition to aggregate make lint for a follow-up, and stayed draft, because three documented oauth2 keys were stale and owned by another PR: docs/admin/integrations/oauth2-provider.md:34 docs/ai-coder/mcp-server.md:108 docs/ai-coder/mcp-server.md:114 #29268 merged on 2026-09-15 and removed all three in favor of CODER_OAUTH2_PROVIDER_ENABLE. The branch is rebased onto that, the check is green against main, the wiring is folded into this PR, and the merge-order constraint recorded as CRF-5 no longer applies. The rebase also collided with DisplayName(), which gained new cases on main. Resolved in favor of main's switch rather than this branch's earlier map refactor, keeping the production diff to the one missing case and moving the exhaustiveness expectations into the test. Why this replaced a generated reference page DOCS-139 originally asked for a generated docs/reference/experiments.md listing every key in ExperimentsKnown. That was built and opened as #29311, then closed without merging. Commit b23d73d (#29113, six days earlier) had removed both generated feature-stage indexes from docs/install/releases/feature-stages.md, recording that the decision, after discussion with David, was not to advertise either list, and describing them as "drift surfaces that promise a completeness the docs cannot maintain." It explicitly rejected the superseded #28949, which generated the list from ExperimentsKnown with descriptions parsed from Go constant comments, for making docs track "an internal, unstable surface." #29311 was that rejected design rebuilt. The ticket predated the decision; the page should not have been built on top of it. This PR is the part of the original goal that survives: the drift protection, without the publication. Local pre-push note The pre-push hook's test stage cannot run in this workspace: GNU Make 4.4.1 segfaults on the repo's .SHELLFLAGS = $@ -ceu configuration for a target named test, tracked in DOCS-903 with a minimal reproduction. Verified instead with the package tests, golangci-lint, and the full pre-commit hook. No --no-verify. Generated with Coder Agents on behalf of @nickvigilante. Linear: DOCS-139

Open Graph Description: Summary The docs name specific experiment keys wherever an individual experimental feature is documented, and nothing verifies those keys still exist in the code. When an experiment is renamed or g...

X Description: Summary The docs name specific experiment keys wherever an individual experimental feature is documented, and nothing verifies those keys still exist in the code. When an experiment is renamed or g...

Opengraph URL: https://github.com/coder/coder/pull/29316

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:bf9ac5a6-13ee-231e-c077-bdb814a6277f
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idD7B6:2C074:3DD7B:51946:6B017DB9
html-safe-nonce14e4ecd4d027567defb3351e0c9908c7643c2322f03aa80be7fa3523242402af
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEN0I2OjJDMDc0OjNERDdCOjUxOTQ2OjZCMDE3REI5IiwidmlzaXRvcl9pZCI6IjI4MDY2ODkzNzgwMDc2MTI4NTciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac6c967b106480c4da4771cc7643b598616eb1ba9a46370b362901711a1e6a42c1
hovercard-subject-tagpull_request:4531495054
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/coder/coder/pull/29316/files
twitter:imagehttps://avatars.githubusercontent.com/u/13631471?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/13631471?s=400&v=4
og:image:altSummary The docs name specific experiment keys wherever an individual experimental feature is documented, and nothing verifies those keys still exist in the code. When an experiment is renamed or g...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Nonecc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/coder/coder git https://github.com/coder/coder.git
octolytics-dimension-user_id95932066
octolytics-dimension-user_logincoder
octolytics-dimension-repository_id440752086
octolytics-dimension-repository_nwocoder/coder
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id440752086
octolytics-dimension-repository_network_root_nwocoder/coder
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8c77c9e8439031ad54c75a250694842892f693e1
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/coder/coder/pull/29316/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcoder%2Fcoder%2Fpull%2F29316%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcoder%2Fcoder%2Fpull%2F29316%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=coder%2Fcoder
Reloadhttps://github.com/coder/coder/pull/29316/files
Reloadhttps://github.com/coder/coder/pull/29316/files
Reloadhttps://github.com/coder/coder/pull/29316/files
coderhttps://github.com/coder
coderhttps://github.com/coder/coder
Codehttps://github.com/coder/coder
Issues320 (320)https://github.com/coder/coder/issues
Pull requests470 (470)https://github.com/coder/coder/pulls
Discussionshttps://github.com/coder/coder/discussions
Actionshttps://github.com/coder/coder/actions
Projectshttps://github.com/coder/coder/projects
Security and quality49 (49)https://github.com/coder/coder/security
Insightshttps://github.com/coder/coder/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fcoder%2Fcoder%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fcoder%2Fcoder%2Fissues%2Fnew%2Fchoose
nickvigilantehttps://github.com/nickvigilante
mainhttps://github.com/coder/coder/tree/main
vigilante/docs-139-validate-documented-experiment-keys-against-experimentsknownhttps://github.com/coder/coder/tree/vigilante/docs-139-validate-documented-experiment-keys-against-experimentsknown
Conversation 31 https://github.com/coder/coder/pull/29316
Commits 7 https://github.com/coder/coder/pull/29316/commits
Checks 41 https://github.com/coder/coder/pull/29316/checks
Files changed https://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
feat: validate documented experiment keys (DOCS-139) https://github.com/coder/coder/pull/29316/files#top
Show all changes 7 commits https://github.com/coder/coder/pull/29316/files
a29e0bd feat: validate documented experiment keys (DOCS-139) nickvigilante Sep 14, 2026 https://github.com/coder/coder/pull/29316/commits/a29e0bd2249c0a5105202fa8e204f3d1cff5652a
39f06e4 fix: cover experiment key validation edge cases (DOCS-139) nickvigilante Sep 15, 2026 https://github.com/coder/coder/pull/29316/commits/39f06e46b449be482b0a48b678559bbfccc75238
e93b5a2 fix(scripts/checkexperimentkeys): ignore empty experiment keys nickvigilante Sep 15, 2026 https://github.com/coder/coder/pull/29316/commits/e93b5a2667d9e11e5309ca5836fd25c9d927ec1b
cad0d28 fix: defer experiment key lint wiring nickvigilante Sep 15, 2026 https://github.com/coder/coder/pull/29316/commits/cad0d28d9cf003a2eec217bd8e4bb346374a9302
5493532 feat(Makefile,codersdk,scripts/checkexperimentkeys): wire experiment … nickvigilante Sep 22, 2026 https://github.com/coder/coder/pull/29316/commits/5493532c297d53e9bf6f88027ed922f9c39b1578
d0d6b1d docs(docs/admin/integrations/oauth2-provider): drop the oauth2 example nickvigilante Oct 1, 2026 https://github.com/coder/coder/pull/29316/commits/d0d6b1d9b0ee8a31f869985ae6f6a976e54998b6
3223e0b test(codersdk): pin display names for the newer experiments nickvigilante Oct 1, 2026 https://github.com/coder/coder/pull/29316/commits/3223e0b37a84f1125deae203dead8826b524f21d
Clear filters https://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
Makefile https://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
deployment.go https://github.com/coder/coder/pull/29316/files#diff-49e2ae4570ed056423733e44ef29562a04ba64272a31121cca564d099958be2b
deployment_test.go https://github.com/coder/coder/pull/29316/files#diff-f0e9fb3415525c73dd8c4d34598fe915efc5c47dff9e73a3c57c17ee879f8b1c
index.md https://github.com/coder/coder/pull/29316/files#diff-c1d4d4a7444f7e7e8d085f49e5c7c1b413e29691fd9694b530ce194e97fa39d4
README.md https://github.com/coder/coder/pull/29316/files#diff-aaabd0bf880cc956a7a35fc62ff0a4ad773728af4d7dd2cdb50c859932016ea2
main.go https://github.com/coder/coder/pull/29316/files#diff-d94d48d32c6ec891b8164904ea58131d2dfcd70de189e4c7f820c9f522126f36
main_test.go https://github.com/coder/coder/pull/29316/files#diff-06476d5c6957ea2b4f83663e66d184f3ea211eb512ebabbf4303f3a7a70a375f
Makefilehttps://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/Makefile
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
https://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
https://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
https://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
https://github.com/coder/coder/pull/29316/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52
codersdk/deployment.gohttps://github.com/coder/coder/pull/29316/files#diff-49e2ae4570ed056423733e44ef29562a04ba64272a31121cca564d099958be2b
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/codersdk/deployment.go
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
https://github.com/coder/coder/pull/29316/files#diff-49e2ae4570ed056423733e44ef29562a04ba64272a31121cca564d099958be2b
https://github.com/coder/coder/pull/29316/files#diff-49e2ae4570ed056423733e44ef29562a04ba64272a31121cca564d099958be2b
codersdk/deployment_test.gohttps://github.com/coder/coder/pull/29316/files#diff-f0e9fb3415525c73dd8c4d34598fe915efc5c47dff9e73a3c57c17ee879f8b1c
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/codersdk/deployment_test.go
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
https://github.com/coder/coder/pull/29316/files#diff-f0e9fb3415525c73dd8c4d34598fe915efc5c47dff9e73a3c57c17ee879f8b1c
https://github.com/coder/coder/pull/29316/files#diff-f0e9fb3415525c73dd8c4d34598fe915efc5c47dff9e73a3c57c17ee879f8b1c
docs/admin/integrations/oauth2-provider/index.mdhttps://github.com/coder/coder/pull/29316/files#diff-c1d4d4a7444f7e7e8d085f49e5c7c1b413e29691fd9694b530ce194e97fa39d4
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/docs/admin/integrations/oauth2-provider/index.md
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
https://github.com/coder/coder/pull/29316/files#diff-c1d4d4a7444f7e7e8d085f49e5c7c1b413e29691fd9694b530ce194e97fa39d4
https://github.com/coder/coder/pull/29316/files#diff-c1d4d4a7444f7e7e8d085f49e5c7c1b413e29691fd9694b530ce194e97fa39d4
scripts/checkexperimentkeys/README.mdhttps://github.com/coder/coder/pull/29316/files#diff-aaabd0bf880cc956a7a35fc62ff0a4ad773728af4d7dd2cdb50c859932016ea2
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/scripts/checkexperimentkeys/README.md
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
scripts/checkexperimentkeys/main.gohttps://github.com/coder/coder/pull/29316/files#diff-d94d48d32c6ec891b8164904ea58131d2dfcd70de189e4c7f820c9f522126f36
View file https://github.com/coder/coder/blob/3223e0b37a84f1125deae203dead8826b524f21d/scripts/checkexperimentkeys/main.go
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/coder/coder/pull/29316/{{ revealButtonHref }}
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
Please reload this pagehttps://github.com/coder/coder/pull/29316/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.