René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Heavy sync init work (e.g. local ML models) starves stdio initialize/tool calls even with threadpool offload -- Windows GIL contention","articleBody":"### Environment\n- `mcp` 1.28.1 (Python SDK, `mcp.server.fastmcp.FastMCP`)\n- stdio transport, Windows 11\n- Blocking work: `sentence-transformers` / torch (local embedding model), but this\n  generalizes to any CPU/GIL-bound library load (onnxruntime, local LLM inference, etc.)\n\n### Summary\nRelated to #1839 / #1909, but a distinct failure mode that threadpool offload does not\nfix: **GIL-bound CPU work** (loading a torch model) contends with the anyio stdio\ntransport's Windows pipe-reader thread for the GIL, regardless of where it runs in-process.\nThis makes several reasonable-looking designs fail in ways that are easy to ship and hard\nto reproduce in a quick manual test.\n\n### Measurements\nSame model load (`sentence-transformers/all-MiniLM-L6-v2`), only the location changes:\n\n| Where the load runs                              | `initialize` | First tool call |\n|----------------------------------------------------|:---:|:---:|\n| Standalone script, no event loop                  | —      | ~8s |\n| Synchronously before `mcp.run()` (\"warm on start\") | **blocks 8–70s** | instant |\n| In a background thread, started before `mcp.run()` | delayed | **deadlocks** |\n| Lazily on first call, offloaded via `anyio.to_thread` / threadpool | instant | **~70s** |\n| In a **separate subprocess**, talked to over a pipe | ~1s | ~8–9s |\n\nOnly the subprocess isolation keeps both connection time and first-call time bounded.\nThe variance (8s → 70s) is driven by contention between the model-loading thread and\nanyio's Windows stdio reader thread — both fighting for the GIL.\n\n### Why this matters beyond my case\nIt's tempting to \"fix\" a slow first tool call by warming eagerly before `mcp.run()`.\nThat instead moves the stall onto the `initialize` handshake — which is worse, because a\nclient that doesn't get the handshake in time just drops the server with no visible\nerror. That's a silent failure (tools missing, no exception) that's *intermittent*\n(fine when warm/cached, broken on a cold start), so it's easy to ship and hard to catch\nin CI or a quick manual check.\n\n### Suggested fix / ask\nNot asking for an SDK-level fix necessarily — `run_in_threadpool` from #1909 is the right\nanswer for I/O-bound blocking calls. But it'd help other implementers to:\n1. Note in the docs/guidance for `FastMCP` that CPU/GIL-bound initialization (local model\n   loads, etc.) should NOT be threadpool-offloaded in-process — it should run in a\n   separate subprocess — since threadpool offload does not release the GIL contention\n   the way it does for I/O-bound blocking calls.\n2. Consider whether `initialize` should have an explicit, documented deadline/backpressure\n   behavior so implementers know exactly how much startup latency is safe.\n\n---\n**Disclaimer:** I'm not a Python/asyncio expert — this diagnosis, the measurements, and\nthis write-up were produced by Claude Code (Anthropic's coding agent) while it was\nbuilding a local memory/retrieval MCP server for me and debugging why it intermittently\nfailed to connect. I'm filing it because the finding looked substantive and reproducible,\nbut I likely can't answer deep follow-up questions about the internals myself.\n","author":{"url":"https://github.com/scotthibbs","@type":"Person","name":"scotthibbs"},"datePublished":"2026-07-12T18:08:58.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/3089/python-sdk/issues/3089"}