Title: `Monitor._thread_lock` leaked into forked children causing deadlocks · Issue #6148 · getsentry/sentry-python · GitHub
Open Graph Title: `Monitor._thread_lock` leaked into forked children causing deadlocks · Issue #6148 · getsentry/sentry-python
X Title: `Monitor._thread_lock` leaked into forked children causing deadlocks · Issue #6148 · getsentry/sentry-python
Description: How do you use Sentry? Sentry Saas (sentry.io) Version 2.57.0 Steps to Reproduce When os.fork() is called while thread in the parent is inside with self._thread_lock:, the child inherits the lock locked. The thread that originally acquir...
Open Graph Description: How do you use Sentry? Sentry Saas (sentry.io) Version 2.57.0 Steps to Reproduce When os.fork() is called while thread in the parent is inside with self._thread_lock:, the child inherits the lock l...
X Description: How do you use Sentry? Sentry Saas (sentry.io) Version 2.57.0 Steps to Reproduce When os.fork() is called while thread in the parent is inside with self._thread_lock:, the child inherits the lock l...
Opengraph URL: https://github.com/getsentry/sentry-python/issues/6148
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`Monitor._thread_lock` leaked into forked children causing deadlocks","articleBody":"### How do you use Sentry?\n\nSentry Saas (sentry.io)\n\n### Version\n\n2.57.0\n\n### Steps to Reproduce\n\n\nWhen `os.fork()` is called while thread in the parent is inside `with self._thread_lock:`, the child inherits the lock locked. The thread that originally acquired it does not exist in the child, so the lock can never be released, and every subsequent call to `Monitor._ensure_running` in the child blocks forever.\n\nWe hit this in production with Celery's `prefork` pool: when a worker dies (`WorkerLostError`), the master's pool supervisor calls `os.fork()` to spawn a replacement. If a Sentry code path is mid-`_ensure_running` on another master thread at that exact moment, the replacement worker is poisoned. Tasks dispatched to it hang and only die at the soft time limit.\n\nThe repro below makes this deterministic by using `os.register_at_fork` to acquire `_thread_lock` in the forking thread itself.\n\nSave as `sentry_monitor_lock.py` and run with `uv run sentry_monitor_lock.py`. Requires Redis at `redis://localhost:6379/0`.\n\n```python\n# /// script\n# requires-python = \">=3.13\"\n# dependencies = [\n# \"sentry-sdk==2.57.0\",\n# \"celery[redis]==5.6.2\",\n# ]\n# ///\n\"\"\"Repro: sentry_sdk Monitor._thread_lock leaked locked across fork().\"\"\"\nimport logging, os, pathlib, sys, tempfile, threading\nfrom datetime import datetime, timedelta, timezone\n\n# Wrap threading.Lock BEFORE importing sentry_sdk so we can capture\n# the Monitor._thread_lock instance at construction time.\nreal_Lock = threading.Lock\nsentry_monitor_locks = []\n\ndef tracking_sentry_monitor_lock(*a, **kw):\n lock = real_Lock(*a, **kw)\n if sys._getframe(1).f_code.co_filename.endswith(\"/sentry_sdk/monitor.py\"):\n sentry_monitor_locks.append(lock)\n return lock\n\nthreading.Lock = tracking_sentry_monitor_lock\n\nimport sentry_sdk\nfrom celery import Celery\nfrom sentry_sdk.integrations.celery import CeleryIntegration\n\nTEMP_DIR = pathlib.Path(tempfile.gettempdir()) / \"sentry_monitor_lock\"\nTEMP_DIR.mkdir(exist_ok=True, parents=True)\nKILL_FLAG = TEMP_DIR / \"flag\"\nKILL_FLAG.unlink(missing_ok=True)\n\nsentry_sdk.init(\n dsn=\"http://public@localhost/1\",\n integrations=[CeleryIntegration()],\n traces_sample_rate=1.0,\n)\n\nassert sentry_monitor_locks, \"Monitor lock not captured during init()\"\n\n# Acquire the lock in the forking thread itself; child inherits it\n# locked because no after_in_child release is registered.\ndef before_fork():\n if KILL_FLAG.exists():\n sentry_monitor_locks[0].acquire()\n\ndef after_in_parent():\n if KILL_FLAG.exists():\n KILL_FLAG.unlink(missing_ok=True)\n sentry_monitor_locks[0].release()\n\nos.register_at_fork(before=before_fork, after_in_parent=after_in_parent)\n\napp = Celery(\"sentry_monitor_lock\", broker=\"redis://localhost:6379/0\")\napp.conf.task_soft_time_limit = 5\napp.conf.beat_schedule = {\"noop\": {\"task\": \"sentry_monitor_lock.noop\", \"schedule\": 5}}\n\n@app.task\ndef kill_me():\n KILL_FLAG.touch()\n os._exit(9)\n\n@app.task\ndef noop():\n return \"ok\"\n\nkill_me.apply_async(eta=datetime.now(timezone.utc) + timedelta(seconds=30))\n\nif __name__ == \"__main__\":\n app.start([\n \"worker\", \"--beat\",\n f\"--schedule={TEMP_DIR}/schedule.db\",\n \"--pool=prefork\", \"--concurrency=1\",\n \"--prefetch-multiplier=1\", \"--loglevel=INFO\",\n ])\n```\n\n**Timeline when run:**\n\n1. `0 s` — celery starts, beat schedules `noop` every 5 s, master spawns initial worker. `noop` tasks succeed.\n2. `30 s` — `kill_me` runs in the worker, touches `KILL_FLAG`, then `os._exit(9)`s.\n3. `~30 s` — master's billiard supervisor sees `WorkerLostError` and calls `os.fork()` to spawn a replacement. The `before_fork` hook fires (gated on `KILL_FLAG`) and acquires `Monitor._thread_lock`. Child inherits the lock locked. Parent releases in `after_in_parent` and recovers.\n4. `30 s onward` — beat keeps dispatching `noop` to the replacement worker. Every task hangs.\n\n### Expected Result\n\nThe replacement worker should process `noop` tasks normally after the post-`kill_me` fork. `Monitor._thread_lock` state in the child must not depend on what threads in the parent were doing at the instant of fork.\n\n### Actual Result\n\n\nThe replacement worker is permanently wedged. Every `noop` it picks up hangs and dies on the soft time limit:\n\n```python\n[2026-04-27 12:59:05,848: INFO/Beat] Scheduler: Sending due task noop (sentry_monitor_lock.noop)\n[2026-04-27 12:59:05,857: INFO/MainProcess] Task sentry_monitor_lock.noop[bf207be7-1240-4bc2-b73e-ded5e6d37534] received\n[2026-04-27 12:59:07,096: WARNING/MainProcess] Soft time limit (5s) exceeded for sentry_monitor_lock.noop[fac600de-e6c7-4533-90da-6e28a22b4fd4]\n[2026-04-27 12:59:07,102: ERROR/MainProcess] Task handler raised error: SoftTimeLimitExceeded()\nbilliard.einfo.RemoteTraceback:\n\"\"\"\nTraceback (most recent call last):\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/billiard/pool.py\", line 362, in workloop\n result = (True, prepare_result(fun(*args, **kwargs)))\n ~~~^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/celery/app/trace.py\", line 694, in fast_trace_task\n R, I, T, Rstr = tasks[task].__trace__(\n ~~~~~~~~~~~~~~~~~~~~~^\n uuid, args, kwargs, request,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/utils.py\", line 1900, in runner\n return sentry_patched_function(*args, **kwargs)\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/integrations/celery/__init__.py\", line 332, in _inner\n with sentry_sdk.start_transaction(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n transaction,\n ^^^^^^^^^^^^\n ...\u003c8 lines>...\n },\n ^^\n ):\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/api.py\", line 408, in start_transaction\n return get_current_scope().start_transaction(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n transaction, instrumenter, custom_sampling_context, **kwargs\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/scope.py\", line 1134, in start_transaction\n transaction._set_initial_sampling_decision(sampling_context=sampling_context)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/tracing.py\", line 1216, in _set_initial_sampling_decision\n self.sample_rate /= 2**client.monitor.downsample_factor\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/monitor.py\", line 109, in downsample_factor\n self._ensure_running()\n ~~~~~~~~~~~~~~~~~~~~^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/monitor.py\", line 51, in _ensure_running\n with self._thread_lock:\n ^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/billiard/pool.py\", line 228, in soft_timeout_sighandler\n raise SoftTimeLimitExceeded()\nbilliard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()\n\"\"\"\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/billiard/pool.py\", line 362, in workloop\n result = (True, prepare_result(fun(*args, **kwargs)))\n ~~~^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/celery/app/trace.py\", line 694, in fast_trace_task\n R, I, T, Rstr = tasks[task].__trace__(\n ~~~~~~~~~~~~~~~~~~~~~^\n uuid, args, kwargs, request,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/utils.py\", line 1900, in runner\n return sentry_patched_function(*args, **kwargs)\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/integrations/celery/__init__.py\", line 332, in _inner\n with sentry_sdk.start_transaction(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n transaction,\n ^^^^^^^^^^^^\n ...\u003c8 lines>...\n },\n ^^\n ):\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/api.py\", line 408, in start_transaction\n return get_current_scope().start_transaction(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n transaction, instrumenter, custom_sampling_context, **kwargs\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/scope.py\", line 1134, in start_transaction\n transaction._set_initial_sampling_decision(sampling_context=sampling_context)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/tracing.py\", line 1216, in _set_initial_sampling_decision\n self.sample_rate /= 2**client.monitor.downsample_factor\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/monitor.py\", line 109, in downsample_factor\n self._ensure_running()\n ~~~~~~~~~~~~~~~~~~~~^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/sentry_sdk/monitor.py\", line 51, in _ensure_running\n with self._thread_lock:\n ^^^^^^^^^^^^^^^^^\n File \"/Users/lv/.cache/uv/environments-v2/sentry-monitor-lock-647cb2fd2bb3ef20/lib/python3.14/site-packages/billiard/pool.py\", line 228, in soft_timeout_sighandler\n raise SoftTimeLimitExceeded()\nbilliard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()\n```","author":{"url":"https://github.com/vokracko","@type":"Person","name":"vokracko"},"datePublished":"2026-04-27T11:00:41Z","url":"https://github.com/getsentry/sentry-python/issues/6148"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:e241e9c9-a2cd-7e75-3da7-1775173fffd2 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | ABE4:68AAF:3B6B46:528B6A:6B008CDD |
| html-safe-nonce | 0df3653d51d37f86b1a2fb3b17b1bb8e5629cda3f4c5b1d1f13aff2f9ead83d3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQkU0OjY4QUFGOjNCNkI0Njo1MjhCNkE6NkIwMDhDREQiLCJ2aXNpdG9yX2lkIjoiODAzNTU0ODYyMDQ3ODUxNjQ0NSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | efa59d478f1e783a3fa0234af03c806873170189319c7511b91203d3a811b84a |
| hovercard-subject-tag | issue:4335185519 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/getsentry/sentry-python/6148/issue_layout |
| twitter:image | https://opengraph.githubassets.com/0ff74f5773e562840e9f44a043747d97e3c54099e21568b51dbfb307c4ef031c/getsentry/sentry-python/issues/6148 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/0ff74f5773e562840e9f44a043747d97e3c54099e21568b51dbfb307c4ef031c/getsentry/sentry-python/issues/6148 |
| og:image:alt | How do you use Sentry? Sentry Saas (sentry.io) Version 2.57.0 Steps to Reproduce When os.fork() is called while thread in the parent is inside with self._thread_lock:, the child inherits the lock l... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | vokracko |
| hostname | github.com |
| expected-hostname | github.com |
| None | c710a5cb5c47be60abfe6e8410436d409906979dbed614ef09f29c60f5535b08 |
| turbo-cache-control | no-preview |
| go-import | github.com/getsentry/sentry-python git https://github.com/getsentry/sentry-python.git |
| octolytics-dimension-user_id | 1396951 |
| octolytics-dimension-user_login | getsentry |
| octolytics-dimension-repository_id | 137814779 |
| octolytics-dimension-repository_nwo | getsentry/sentry-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 137814779 |
| octolytics-dimension-repository_network_root_nwo | getsentry/sentry-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | be616a2a6ec1c609d6e002896dff5352d5c5f28b |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width