René's URL Explorer Experiment

Title: ci(conformance): pin harness to 0.2.0-alpha.3 with expected-failures baseline by maxisbey · Pull Request #2877 · modelcontextprotocol/python-sdk · GitHub

Open Graph Title: ci(conformance): pin harness to 0.2.0-alpha.3 with expected-failures baseline by maxisbey · Pull Request #2877 · modelcontextprotocol/python-sdk

X Title: ci(conformance): pin harness to 0.2.0-alpha.3 with expected-failures baseline by maxisbey · Pull Request #2877 · modelcontextprotocol/python-sdk

Description: Modernizes the conformance CI to match the typescript-sdk pattern so the job actually gates and the failure set burns down per SEP. Supersedes #1921 (the composite-action approach) — pinning the npm package directly is what typescript-sdk settled on. Motivation and Context The conformance jobs were pinned to 0.1.10/0.1.13 (drifted between server and client), ran with continue-on-error: true so they never gated, and had no baseline file — failures were silently ignored. The conformance harness has since gained --expected-failures (#113) and the 0.2.0-alpha line with version-aware scenario selection. Changes Pin the harness via a single workflow-level CONFORMANCE_VERSION: "0.2.0-alpha.3" env var (one place to bump). Add .github/actions/conformance/expected-failures.yml — the path the conformance repo's known-sdks.ts already expects. Client baseline is 16 scenarios grouped by SEP (same set as typescript-sdk's baseline); server active suite is fully green so server: []. Drop continue-on-error: true from both jobs. The runner now exits 0 only when failures match the baseline, 1 on regressions or stale entries. Harden run-server.sh with a port-conflict guard, dead-process check in the readiness loop, and curl --max-time, mirroring typescript-sdk's wrapper. The server --suite draft step (2026-07-28 scenarios) is a follow-up PR; the baseline file already has a server: placeholder for it. How Has This Been Tested? Run locally against the conformance harness at 0.2.0-alpha.3: server --suite active --expected-failures ... → 30/30 scenarios pass, 42 assertions, exit 0. client --suite all --expected-failures ... → 40 scenarios: 24 pass, 16 expected-fail, 0 unexpected, 0 stale, exit 0. Stale-entry guard: appending a passing scenario (initialize) to the baseline → exit 1 with Stale baseline entries (now passing - remove from baseline): initialize. bash -n, YAML parse, and pre-commit all clean. Breaking Changes None for SDK users. For contributors: the conformance jobs now fail on regressions instead of silently passing. run-server.sh requires CONFORMANCE_VERSION to be set when run locally (the error message points at the workflow pin). Types of changes Bug fix (non-breaking change which fixes an issue) New feature (non-breaking change which adds functionality) Breaking change (fix or feature that would cause existing functionality to change) Documentation update Checklist I have read the MCP Documentation My code follows the repository's style guidelines New and existing tests pass locally I have added appropriate error handling I have added or updated documentation as needed Additional context The conformance jobs are not branch-protection-required checks, so an npm-registry blip turns them red but does not block merges. Same posture as typescript-sdk; can revisit once it has run cleanly for a while. Backporting run-server.sh to v1.x will need the env: CONFORMANCE_VERSION block backported in the same change (the script now reads it via ${CONFORMANCE_VERSION:?}). Follow-up on the conformance repo: add a python-sdk (main) entry to known-sdks.ts alongside the existing python-sdk-v1. AI Disclaimer

Open Graph Description: Modernizes the conformance CI to match the typescript-sdk pattern so the job actually gates and the failure set burns down per SEP. Supersedes #1921 (the composite-action approach) — pinning the np...

X Description: Modernizes the conformance CI to match the typescript-sdk pattern so the job actually gates and the failure set burns down per SEP. Supersedes #1921 (the composite-action approach) — pinning the np...

Opengraph URL: https://github.com/modelcontextprotocol/python-sdk/pull/2877

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width