Title: Buffer per-request StreamableHTTP streams to avoid serial-router head-of-line block by maxisbey · Pull Request #2934 · modelcontextprotocol/python-sdk · GitHub
Open Graph Title: Buffer per-request StreamableHTTP streams to avoid serial-router head-of-line block by maxisbey · Pull Request #2934 · modelcontextprotocol/python-sdk
X Title: Buffer per-request StreamableHTTP streams to avoid serial-router head-of-line block by maxisbey · Pull Request #2934 · modelcontextprotocol/python-sdk
Description: Two related fixes to the stateful StreamableHTTPServerTransport so concurrent requests can't wedge each other and event-store ordering doesn't depend on scheduler timing. Buffer the per-request _request_streams (head-of-line block). Each POST registers a buffer-0 _request_streams[id] channel, start_soons the EventSourceResponse, and pushes the request to the server. The single serial message_router then forwards each response with await _request_streams[id][0].send(...) — which on a buffer-0 stream parks until that request's sse_writer (started lazily, two start_soon hops deep) has reached its first receive(). While the router is parked on one request, every other in-flight response on the session is head-of-line blocked behind it. This gives the three _request_streams[EventMessage] sites a small bounded buffer (REQUEST_STREAM_BUFFER_SIZE = 16) so the router can deposit and move on. The downstream sse_stream dict streams stay at 0 — they're not on the router's send target. Store the priming event before request dispatch (event-store ordering). With an event_store configured, message_router calls store_event(msg) before its send(), while the priming event was stored inside the lazily-started sse_writer — so its event-store position raced the first routed message. Splitting the helper into _mint_priming_event (store + return wire dict) and _run_sse_writer (forward onto the wire) lets the POST handler await _mint_priming_event(...) before writer.send(session_message). The server can't emit anything for a request it hasn't received, so the priming row now precedes every router store for that stream by data dependency rather than scheduler timing — the same shape the TypeScript SDK uses. Motivation and Context Fixes #1764. The natural race is hard to reproduce standalone (multiple parties report 0 hits across thousands of loopback iterations) but is observed consistently in production by several reporters on the issue. The unit-level head-of-line block is straightforward to demonstrate though: register two request streams, give B a consumer, leave A consumer-less, write responses A then B — on main B never arrives until A is closed; with this change B arrives immediately and A's response is buffered. The priming-order fix addresses a review finding that the buffer change widened a pre-existing window where a routed message could be stored before the priming event. How Has This Been Tested? New tests/server/test_streamable_http_router.py drives the routing layer directly: one test for the head-of-line block, one asserting [priming, m₁, …, m₅] event-store order with no sse_writer ever scheduled. tests/shared/test_streamable_http.py priming-helper unit tests updated for _mint_priming_event. Existing reconnection / sse-polling / replay tests pass unchanged. Breaking Changes None. EventStore ABC is unchanged. Intentional behaviour changes: The standalone GET stream now buffers up to 16 server-initiated messages before backpressure engages on a slow GET client (was 0). A request whose consumer stalls past 16 messages (e.g. a long-running tool emitting many progress notifications to a stalled SSE client) can still wedge the router — this moves the threshold from 0 to 16, it doesn't remove the serial router. The routerless design in _streamable_http_modern.py is the structural fix on main. _maybe_send_priming_event is replaced by _mint_priming_event (returns the wire dict instead of sending) and _run_sse_writer. Both are private; no public surface change. The replay path's priming/live-tail ordering window is pre-existing and unchanged here; tracked separately. Types of changes Bug fix (non-breaking change which fixes an issue) New feature (non-breaking change which adds functionality) Breaking change (fix or feature that would cause existing functionality to change) Documentation update Checklist I have read the MCP Documentation My code follows the repository's style guidelines New and existing tests pass locally I have added appropriate error handling I have added or updated documentation as needed Additional context The buffer-size approach matches what reporters on #1764 have field-validated (sed-patching to 10). 16 covers the reported single-response list-request case by construction (one message per id) plus headroom for small notification workloads, while staying bounded for the session-lifetime GET stream. A separate change is needed for the stateless task-leak half of #1764 (#2145's request-scoped task-group approach); that's not bundled here. AI Disclaimer
Open Graph Description: Two related fixes to the stateful StreamableHTTPServerTransport so concurrent requests can't wedge each other and event-store ordering doesn't depend on scheduler timing. Buffer the per-req...
X Description: Two related fixes to the stateful StreamableHTTPServerTransport so concurrent requests can't wedge each other and event-store ordering doesn't depend on scheduler timing. Buffer the...
Opengraph URL: https://github.com/modelcontextprotocol/python-sdk/pull/2934
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c3e2e39d-d034-dfba-7e12-1e7a3358a0a8 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | E4F4:2CFBF7:1B65CA5:264846E:6AFD0AD2 |
| html-safe-nonce | b86df3ffdf0815546830fb713fc087af26b741c124c1b31eb5d908faa7ce0391 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNEY0OjJDRkJGNzoxQjY1Q0E1OjI2NDg0NkU6NkFGRDBBRDIiLCJ2aXNpdG9yX2lkIjoiMTA0OTk3NTI1OTE5MDMzMjExNCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 1d0b6de2c7dccbff00561234de11db3b34e31798d24e618adda3a4738c3d330c |
| hovercard-subject-tag | pull_request:3904579047 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/modelcontextprotocol/python-sdk/pull/2934/files |
| twitter:image | https://avatars.githubusercontent.com/u/224885523?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/224885523?s=400&v=4 |
| og:image:alt | Two related fixes to the stateful StreamableHTTPServerTransport so concurrent requests can't wedge each other and event-store ordering doesn't depend on scheduler timing. Buffer the per-req... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b29de2abb97b6307b16270785efeb58b51a117d5a9b659acf00b77aa00e7043b |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/modelcontextprotocol/python-sdk git https://github.com/modelcontextprotocol/python-sdk.git |
| octolytics-dimension-user_id | 182288589 |
| octolytics-dimension-user_login | modelcontextprotocol |
| octolytics-dimension-repository_id | 862584018 |
| octolytics-dimension-repository_nwo | modelcontextprotocol/python-sdk |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 862584018 |
| octolytics-dimension-repository_network_root_nwo | modelcontextprotocol/python-sdk |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c9e619def6d670dc18a45d3b02cea938c1c7e47 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width