Title: fix(batcher): wait for in-flight batches in flush() by tony1233200 · Pull Request #8340 · getsentry/sentry-python · GitHub
Open Graph Title: fix(batcher): wait for in-flight batches in flush() by tony1233200 · Pull Request #8340 · getsentry/sentry-python
X Title: fix(batcher): wait for in-flight batches in flush() by tony1233200 · Pull Request #8340 · getsentry/sentry-python
Description: Fixes #8300 Problem Client.flush() can return before a batch reaches the transport. The flusher thread removes the batch from the buffer under _lock and only then calls _capture_func, after releasing the lock. A flush() from another thread in that window finds an empty buffer and returns at once, while the batch is still on its way to the transport. Reproduction (from the issue, span streaming with a transport that sleeps 0.5s in the handoff): after flush(): [] # before: span arrives later after flush(): ['span'] # after The same window applies to logs and metrics (Batcher._flush) and to streamed spans (SpanBatcher._flush). It also affects shutdown: close() flushes and then sets transport = None, so a batch still being handed off can find no transport and be dropped silently. Result Batcher now tracks in-flight batches: _flush marks a batch in-flight atomically with taking it from the buffer (under _lock), and clears the mark once _capture_func returns or raises. flush() drains the buffer, then waits for the batches that were in flight when it started, and drains once more for anything that arrived while waiting. Batches taken after the flush started are new work and are not covered by that flush. The wait is bounded to the in-flight batches present at entry: each completing flush advances a counter, so a continuously producing app cannot keep a flush() waiting forever. A re-entrant flush() from inside batcher code (for example a transport calling flush() from within _capture_func) skips the wait, using the existing _active thread-local guard, so it cannot deadlock on its own handoff. SpanBatcher gets the same treatment: it has its own _flush and __init__, so the counters live in both classes. LogBatcher and MetricsBatcher use the base Batcher._flush and are covered by the base-class change. close() benefits without a code change: it flushes before killing the batchers and clearing the transport, so the in-flight batch is now delivered first. Compatibility No public API change. flush() may now take longer when a slow transport is mid-handoff — that is the documented contract ("Wait for the current events to be sent"). No behavior change when nothing is in flight: the snapshot shows zero in-flight batches and flush() returns without waiting. Validation Issue reproduction run before and after (span streaming, SlowTransport sleeping 0.5s in the handoff): before, after flush(): []; after, after flush(): ['span']. New regression test test_flush_waits_for_in_flight_span_batch in tests/test_client.py: fails on the base code, passes with the fix. It builds the client directly instead of using the sentry_init fixture, because that fixture installs _install_flush_completion_handshake, which would mask the race. Test runs (local venv, Python 3.12): pytest tests/test_client.py::test_flush_waits_for_in_flight_span_batch — 1 passed. pytest tests/test_client.py — 69 failed, 149 passed, 1 skipped. The 69 failures are identical to the base code (same test names, verified by diffing the failure lists): async-transport tests failing because aiohttp is not installed in this environment, so Client falls back to HttpTransport. No new failures. pytest tests/test_logs.py tests/test_metrics.py — 58 passed. ruff check and ruff format --check on the changed files: clean. mypy sentry_sdk/_batcher.py sentry_sdk/_span_batcher.py: 20 errors, all in untouched files (transport.py, utils.py, missing optional-dependency stubs in this environment); 0 errors in the changed files, and the same 20 errors on the base code. Not run: the full test suite and the Docker-based integration tests (no Docker in this environment); CI covers those. Note: open PR #7776 also touches Batcher._flush (moves serialization outside the lock). It does not address this race; the two changes touch the same function and may need a rebase if #7776 lands first.
Open Graph Description: Fixes #8300 Problem Client.flush() can return before a batch reaches the transport. The flusher thread removes the batch from the buffer under _lock and only then calls _capture_func, after releasi...
X Description: Fixes #8300 Problem Client.flush() can return before a batch reaches the transport. The flusher thread removes the batch from the buffer under _lock and only then calls _capture_func, after releasi...
Opengraph URL: https://github.com/getsentry/sentry-python/pull/8340
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:b30f7a00-de3f-9650-25af-5ac569fda5ac |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 89C8:16EED3:AB833F:DB2FD1:6B00F671 |
| html-safe-nonce | b71648982a895d67ec3c5472de39c5ed26aba2bbdd2539eb2fc34fed0a25abb7 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4OUM4OjE2RUVEMzpBQjgzM0Y6REIyRkQxOjZCMDBGNjcxIiwidmlzaXRvcl9pZCI6IjUyNDkxOTU2MzUyMzk4NzYyMDkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 79d49fff0affedc31ef4a197cc4a46970e1db2f45b484b3db9145831c79fbfc9 |
| hovercard-subject-tag | pull_request:5413136235 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/getsentry/sentry-python/pull/8340/files |
| twitter:image | https://avatars.githubusercontent.com/u/127579663?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/127579663?s=400&v=4 |
| og:image:alt | Fixes #8300 Problem Client.flush() can return before a batch reaches the transport. The flusher thread removes the batch from the buffer under _lock and only then calls _capture_func, after releasi... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 4ef498cbabcbe3c50befc7a5b743e7072ee770b4daf0e15e54801180d1fa91fc |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/getsentry/sentry-python git https://github.com/getsentry/sentry-python.git |
| octolytics-dimension-user_id | 1396951 |
| octolytics-dimension-user_login | getsentry |
| octolytics-dimension-repository_id | 137814779 |
| octolytics-dimension-repository_nwo | getsentry/sentry-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 137814779 |
| octolytics-dimension-repository_network_root_nwo | getsentry/sentry-python |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 9983dd5da06943f833c971dd5b6bc988e2858be5 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width