René's URL Explorer Experiment

Title: Gate the test matrix and retry setup-uv's flaky manifest fetch by maxisbey · Pull Request #3080 · modelcontextprotocol/python-sdk · GitHub

Open Graph Title: Gate the test matrix and retry setup-uv's flaky manifest fetch by maxisbey · Pull Request #3080 · modelcontextprotocol/python-sdk

X Title: Gate the test matrix and retry setup-uv's flaky manifest fetch by maxisbey · Pull Request #3080 · modelcontextprotocol/python-sdk

Description: Two paired CI fixes: make the required all-green check actually gate the test matrix, and stop a known setup-uv flake from becoming merge-blocking once it does. Motivation and Context The gate. The test matrix job in shared.yml has had job-level continue-on-error: true since #708 (May 2025). That flag makes the checks reusable workflow report success to all-green no matter what the matrix does, so the only required status check passes even when every test leg fails. Concrete example: in run 28935462867 the test (3.10, locked, ubuntu-latest) job failed, yet the run concluded success and all-green passed. Scanning the last 200 CI runs (8 days): 14 green runs had all 20 test legs failing. Failures are still visible as red ✗ on the PR checks list — which is why nothing broken has actually landed — but the gate itself is decorative, and RELEASE.md had to document the manual workaround ("a green workflow run does not mean the tests passed"). The flag made sense in #708 as insulation for the then-new, then-flaky Windows matrix (the old hang issues), but it silences every leg, not just Windows, and the Windows instability it guarded against was fixed (most recently #3070). Removing it needs a companion: job-level continue-on-error was also suppressing fail-fast cancellation, so fail-fast: false is added to keep today's run-all-legs behavior (identical compute; without it, the first failing leg would cancel the other 19 and destroy per-leg signal). The flake. With the gate re-armed, a known setup-uv flake would become merge-blocking: the action fetches its version manifest from raw.githubusercontent.com in a single request with a hardcoded 5-second timeout and no retry (astral-sh/setup-uv#869; the timeout was introduced by astral-sh/setup-uv#883, first shipped in v8.2.0 — the version pinned here). The fetch happens even with an exact version: pin because the download URL only exists in the manifest, and it is unfixed through v8.3.2 (the fetch code is byte-identical), so bumping the pin doesn't help. It hit this repo 3 times in the last ~19 days (1, 2, 3) — roughly 0.06% of jobs, which across the 23 gating setup-uv executions per run would flake-block a run every couple of days. Each setup-uv step in shared.yml therefore becomes a pair: the first attempt has step-level continue-on-error: true, and an identical retry step runs on steps.setup-uv.outcome == 'failure' (outcome is the pre-continue-on-error result; failure() would never fire here). The retry step has no continue-on-error, so deterministic errors — bad version, real checksum mismatch — fail both attempts and the job goes red; nothing is masked. Same pattern as e.g. apache/shenyu's setup-java retry. The steps are duplicated inline on purpose; a composite action felt like more machinery than a temporary workaround deserves — these steps should be deleted once upstream adds a retry. Notes: A flake absorbed by the retry no longer shows a red ✗ on the PR — look for a non-skipped "Install uv (retry)" step in the run. The failed first attempt also leaves a harmless "Error retrieving cache key from state." warning from its post step; expected, not a cache bug. The setup-uv steps in conformance.yml, deploy-docs.yml, docs-preview.yml, and publish-pypi.yml stay single-shot on purpose: they don't feed all-green, no flakes were observed there (300 conformance runs scanned), and the release/preview workflows are better left churn-free here. The pattern is trivially copyable if one of them starts flaking. Side effect of the re-armed gate: release-build in publish-pypi.yml has needs: [checks], so releases become genuinely test-gated too, and the RELEASE.md caveat about checking individual jobs is now stale (removed here). How Has This Been Tested? actionlint, zizmor, and the repo pre-commit hooks pass with no findings (zizmor output identical to the current file). The retry wiring (continue-on-error + outcome semantics) was exercised with act: retry fires on first-attempt failure, is skipped on success, and a double failure fails the job. Live validation on this PR (temporary commits, since dropped): one commit pointed the first setup-uv attempt at an unreachable manifest-file URL to force the flake — the retry fired on all 20 legs and the jobs stayed green; one commit added a deliberately failing test — all-green went red, confirming the re-armed gate blocks. Breaking Changes None for SDK users. For contributors: PRs with failing tests now fail all-green instead of showing a green run with red legs. Types of changes Bug fix (non-breaking change which fixes an issue) Checklist I have read the MCP Documentation My code follows the repository's style guidelines New and existing tests pass locally I have added appropriate error handling I have added or updated documentation as needed Additional context The upstream root cause is fixable in setup-uv itself (a retry in its fetch wrapper, or skipping the manifest for exact version pins); happy to take that to astral-sh/setup-uv#869 separately. When upstream ships a retry, the duplicated steps here should be deleted — the inline comments say so. AI Disclaimer

Open Graph Description: Two paired CI fixes: make the required all-green check actually gate the test matrix, and stop a known setup-uv flake from becoming merge-blocking once it does. Motivation and Context The gate. The...

X Description: Two paired CI fixes: make the required all-green check actually gate the test matrix, and stop a known setup-uv flake from becoming merge-blocking once it does. Motivation and Context The gate. The...

Opengraph URL: https://github.com/modelcontextprotocol/python-sdk/pull/3080

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width