Title: `bm_fastapi` intermittently fails with `httpx.PoolTimeout` under load · Issue #508 · python/pyperformance · GitHub
Open Graph Title: `bm_fastapi` intermittently fails with `httpx.PoolTimeout` under load · Issue #508 · python/pyperformance
X Title: `bm_fastapi` intermittently fails with `httpx.PoolTimeout` under load · Issue #508 · python/pyperformance
Description: Summary The fastapi benchmark (bm_fastapi) intermittently fails with httpx.PoolTimeout when the machine running the benchmark is under heavy load. This is a separate problem from the proxy-related 503 (issue #504). Environment pyperforma...
Open Graph Description: Summary The fastapi benchmark (bm_fastapi) intermittently fails with httpx.PoolTimeout when the machine running the benchmark is under heavy load. This is a separate problem from the proxy-related ...
X Description: Summary The fastapi benchmark (bm_fastapi) intermittently fails with httpx.PoolTimeout when the machine running the benchmark is under heavy load. This is a separate problem from the proxy-related ...
Opengraph URL: https://github.com/python/pyperformance/issues/508
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`bm_fastapi` intermittently fails with `httpx.PoolTimeout` under load","articleBody":"## Summary\n\nThe `fastapi` benchmark (`bm_fastapi`) intermittently fails with\n`httpx.PoolTimeout` when the machine running the benchmark is under heavy load.\nThis is a separate problem from the proxy-related 503 (issue #504).\n\n## Environment\n\n- pyperformance 1.14.0\n- Python 3.10\n- fastapi==0.121.0, httpx==0.28.1, uvicorn==0.38.0\n- No proxy involved (`trust_env=False` / no `HTTP_PROXY`)\n\n## How to reproduce\n\nThe benchmark fires `CONCURRENCY = 150` concurrent requests against its own\nuvicorn server. Reproduce deterministically by adding an artificial per-request\ndelay (simulating a slow server) and running 150 concurrent requests through an\n`httpx.AsyncClient` with the default pool:\n\n```python\nimport asyncio, httpx\n\nCONCURRENCY = 150\n\nasync def bench(url):\n async with httpx.AsyncClient() as client: # default max_connections=100\n tasks = [client.get(f\"{url}/items/{i}\") for i in range(CONCURRENCY)]\n await asyncio.gather(*tasks)\n```\n\nWith a server that takes ~6 seconds per response:\n\n- default pool (100 connections) → `httpx.PoolTimeout` after ~5.1s\n- `limits=httpx.Limits(max_connections=150)` → succeeds (~7.1s)\n\n## Actual result\n\n```\nhttpx.PoolTimeout\n```\n\nThe benchmark dies:\n\n```\nRuntimeError: Benchmark died\nERROR: Benchmark fastapi failed: Benchmark died\n```\n\n## Root cause\n\n`CONCURRENCY = 150`, but the `httpx.AsyncClient` uses httpx's default\n`max_connections = 100`. The 50 requests that exceed the pool size wait in the\npool's queue for a connection to free up. httpx's default *pool* timeout is 5\nseconds (`httpx.Timeout(timeout=5.0)`), so when the in-flight requests take\nlonger than 5 seconds (which is exactly when the machine is under load), the 50\nqueued requests exceed the pool timeout and raise `httpx.PoolTimeout`.\n\n## Expected behavior\n\nThe benchmark should complete regardless of how loaded the machine is, because\nits own concurrency is a fixed constant: the connection pool should be sized to\nmatch it.\n\n## Suggested fix\n\nSize the connection pool to the benchmark's concurrency:\n\n```python\nasync with httpx.AsyncClient(\n limits=httpx.Limits(max_connections=CONCURRENCY),\n) as client:\n```\n\nThis removes the queuing entirely, so no request can exceed the pool timeout.\n\n## Scope\n\nOnly `bm_fastapi` is affected: it is the benchmark that combines a fixed\n`CONCURRENCY` (150) larger than httpx's default pool (100) with a\n`httpx.AsyncClient` created without explicit pool limits.\n","author":{"url":"https://github.com/sendaoYan","@type":"Person","name":"sendaoYan"},"datePublished":"2026-11-09T07:08:21Z","url":"https://github.com/python/pyperformance/issues/508"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:881d5f5c-8157-2c9e-c5b1-1c866f904dcb |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | B6AC:97D39:9C2025:CE7437:6B01E930 |
| html-safe-nonce | fdf4843748b0e66dc14884b0c9076649b763088dfcfdca0cb9f91c83e11646d4 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNkFDOjk3RDM5OjlDMjAyNTpDRTc0Mzc6NkIwMUU5MzAiLCJ2aXNpdG9yX2lkIjoiMTQzMDE4MjMyMTMzMTQ5NzI2NCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 8bf43c67dc24448b8a2e2d5133aa9f7fb66d7b70809535424d2dfd4f1cd503ab |
| hovercard-subject-tag | issue:6281603243 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/pyperformance/508/issue_layout |
| twitter:image | https://opengraph.githubassets.com/e58849be7b5521551e999678cc40238965305f51aa1d9aef2440296649d98b07/python/pyperformance/issues/508 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/e58849be7b5521551e999678cc40238965305f51aa1d9aef2440296649d98b07/python/pyperformance/issues/508 |
| og:image:alt | Summary The fastapi benchmark (bm_fastapi) intermittently fails with httpx.PoolTimeout when the machine running the benchmark is under heavy load. This is a separate problem from the proxy-related ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | sendaoYan |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/pyperformance git https://github.com/python/pyperformance.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 65949828 |
| octolytics-dimension-repository_nwo | python/pyperformance |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 65949828 |
| octolytics-dimension-repository_network_root_nwo | python/pyperformance |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width