René's URL Explorer Experiment


Title: Django dev server hangs at startup due to deadlock between `LogtailHandler.flush()` and flusher/uploader · Issue #42 · logtail/logtail-python · GitHub

Open Graph Title: Django dev server hangs at startup due to deadlock between `LogtailHandler.flush()` and flusher/uploader · Issue #42 · logtail/logtail-python

X Title: Django dev server hangs at startup due to deadlock between `LogtailHandler.flush()` and flusher/uploader · Issue #42 · logtail/logtail-python

Description: This seems really similar to #35, although that issue was closed and I’m experiencing this today. I’m not clear on whether that issue was actually solved since there was no feedback at the end of the discussion. If the root logger in a D...

Open Graph Description: This seems really similar to #35, although that issue was closed and I’m experiencing this today. I’m not clear on whether that issue was actually solved since there was no feedback at the end of t...

X Description: This seems really similar to #35, although that issue was closed and I’m experiencing this today. I’m not clear on whether that issue was actually solved since there was no feedback at the end of t...

Opengraph URL: https://github.com/logtail/logtail-python/issues/42

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Django dev server hangs at startup due to deadlock between `LogtailHandler.flush()` and flusher/uploader","articleBody":"This seems really similar to #35, although that issue was closed and I’m experiencing this today. I’m not clear on whether that issue was actually *solved* since there was no feedback at the end of the discussion.\n\nIf the root logger in a Django project is handled by `LogtailHandler`, the Django dev server (`python manage.py runserver`) deadlocks when starting unless `--noreload` is specified as an option.\n\n\n## To Reproduce\n\nUsing:\n- logtail-python v0.4.0\n- django v6.0.7\n- CPython 3.13.15 (but looking at CPython’s source, I think this would happen in 3.14 and 3.15)\n\n\n\nGiven a settings file with logging configured:\n\n```py\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n    'handlers': {\n        'logtail': {\n            'class': 'logtail.LogtailHandler',\n            'source_token': LOGTAIL_TOKEN,\n            'host': LOGTAIL_HOST,\n        },\n    },\n    'loggers': {\n        '': {\n            'level': 'INFO',\n            'handlers': ['logtail'],\n        }\n    },\n}\n```\n\nRunning the the dev server outputs the following and then hangs:\n\n```\n> python manage.py runserver 0.0.0.0:8000\nWatching for file changes with StatReloader\nPerforming system checks...\n\nSystem check identified no issues (0 silenced).\n```\n\nThe server doesn’t fully initialize and doesn’t respond to HTTP requests. If everything went fine, you normally see the above followed by something like:\n\n```\nSeptember 05, 2026 - 02:54:50\nDjango version 6.0.7, using settings 'config.settings.dev'\nStarting development server at http://0.0.0.0:8000/\nQuit the server with CONTROL-C.\n\nWARNING: This is a development server. Do not use it in a production setting. Use a production WSGI or ASGI server instead.\nFor more information on production servers see: https://docs.djangoproject.com/en/6.0/howto/deployment/\n```\n\n## Simpler Reproduction (No Django)\n\nThat said, the actual problem here is just a deadlock when reconfiguring loggers while there are logs left to be flushed. You can reproduce it more simply without Django using the following script:\n\n```py\nimport logging\nimport logging.config\nfrom os import getenv\n\n\nlogger = logging.getLogger('example_logger')\n\n\nLOGGING = {\n    'version': 1,\n    'disable_existing_loggers': False,\n    'handlers': {\n        'logtail': {\n            'class': 'logtail.LogtailHandler',\n            'source_token': getenv('LOGTAIL_TOKEN'),\n            'host': getenv('LOGTAIL_HOST'),\n        },\n    },\n    'loggers': {\n        '': {\n            'level': 'INFO',\n            'handlers': ['logtail'],\n        }\n    },\n}\n\nlogging.config.dictConfig(LOGGING)\n\nprint('=== Logs configured. ===')\n\nlogger.info('Hello!')\n\nprint('=== Logged one message. ===')\n\nlogging.config.dictConfig(LOGGING)\n\nprint('=== Logging reconfigured. ===')\n\nlogger.info('Goodbye!')\n\nprint('=== Logged second message. ===')\n```\n\nThat *should* print:\n\n```\n=== Logs configured. ===\n=== Logged one message. ===\n=== Logged reconfigured. ===\n=== Logged second message. ===\n```\n\nBut instead it just prints the following and then hangs:\n\n```\n=== Logs configured. ===\n=== Logged one message. ===\n```\n\n\n## Cause\n\nThe previous issue (#35) identified the hang as an infinite loop in `FlushWorker.flush()` (which is called by `LogtailHandler.flush()`): https://github.com/logtail/logtail-python/blob/7a7f3f77dc8f69dfc187cf28595e3d584f21c5e7/logtail/flusher.py#L90-L94\n\nIn my case, the infinite loop here is actually because the log flushing/HTTP thread is deadlocked (so `self._clean` never gets set to `True`). Django’s main thread (where the loggers are being reconfigured and `flush()` is called) is holding a lock that the flushing thread needs in order to finish flushing.\n\nSpecifically, the following happens:\n1. `logging.dictConfig()` (or any other config method) [takes the `logging._lock` lock](https://github.com/python/cpython/blob/4061bc4c35f7c26f25264666d4ba083b93d2f6f9/Lib/logging/config.py#L534-L544).\n2. The configuration method de-registers all the handlers (they are still attached to individual loggers in `some_logger.handlers`, so they still work; they just can’t be looked up by name).\n3. The configuration method calls `flush()` on all the handlers.\n4. `LogtailHandler.flush()` calls `FlushWorker.flush()`, which waits on another thread to do the actual sending of logs.\n5. On the thread that is buffering and sending logs, `FlushWorker.step()` buffers up the logs and then sends them via the uploader.\n6. The uploader calls `requests.Session.post()`, which tries to log a message about the HTTP request. That logging call deadlocks.\n\n    Various methods in `logging.Logger` and `logging.Handler` instances try to take the `logging._lock` lock, which is already held by the main thread that is reconfiguring loggers, and won’t be released until flushing is done, which can’t be done until after these logging calls in the HTTP library.\n\n    (The main method I’m observing causing the deadlock here is `Logger.isEnabledFor()`, which [takes the lock if it has not yet cached information about what level it is set at](https://github.com/python/cpython/blob/v3.13.15/Lib/logging/__init__.py#L1769-L1786). So if all the loggers down in requests and urllib3 have already been used, you might avoid the deadlock. But there are other places the lock is used that might also cause problems.)\n\nIn the context of the Django dev server, the following happens when reloading is turned on (the default behavior):\n1. Django loads settings (which configures loggers) before starting the `runserver` command.\n2. The `runserver` command logs `\"Watching for file changes with StatReloader\"`.\n3. The `runserver` command (re)starts the actual WSGI app (which reloads settings, which reconfigures loggers, which hits the above issue).\n\nIf you don’t have reloading enabled (e.g. you call `python manage.py runserver --noreload`), it doesn’t try to log a message before starting the Django app, and avoids the deadlock (since there is nothing to send when the handlers flush).\n\n\n## Fixes\n\nThis is a little thorny! As far as I can tell, this *should* work fine if the logging happens on the thread where loggers are being reconfigured (since the lock is an `RLock`, it can be reacquired recursively by the same thread). So the main thing to do here is make sure the actual sending of logs happens on the same thread where `flush()` is called.\n\nMy naive thought here is that the whole handler/flush worker needs a bit of re-architecting. Instead of using a queue to buffer up log records, the handler could probably just have a list that is protected by a lock. When the buffer is full, it could be sliced off and dropped onto a queue as a single bundle for a sending thread to handle. Calls to `flush()` would just bypass the queue and call the sending method directly. Another thread could serve as a heartbeat to flush the queue on a schedule (this could be the same as the thread that is reading off the queue, but is easier to conceptualize as two threads). \n\nThere might be easier ways to do this that fit better with the current code. I spent more time on identifying the deadlock than thinking through an ideal solution so far. 😉 ","author":{"url":"https://github.com/Mr0grog","@type":"Person","name":"Mr0grog"},"datePublished":"2026-09-05T05:16:46Z","url":"https://github.com/logtail/logtail-python/issues/42"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:45b4190e-297e-7ea3-90eb-43d7ccbc2e2f
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idD4DA:13F309:FCDBE:14D12A:6AFBAF3D
html-safe-nonce0163a13381f4df916f39402c351cb909e684387ce51333d28ced48a27dd9a71c
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENERBOjEzRjMwOTpGQ0RCRToxNEQxMkE6NkFGQkFGM0QiLCJ2aXNpdG9yX2lkIjoiNDA2MjUxMDg3NzMyMjc1MTgwNSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac7a24f6413e613ca5948f81cbe039e518dcf914138fee4214409c264d0819e0c6
hovercard-subject-tagissue:5355788754
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/logtail/logtail-python/42/issue_layout
twitter:imagehttps://opengraph.githubassets.com/74d508b12e4054638338c631fbe10f7119a888f41c28903e7fb8650ad133f6c7/logtail/logtail-python/issues/42
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/74d508b12e4054638338c631fbe10f7119a888f41c28903e7fb8650ad133f6c7/logtail/logtail-python/issues/42
og:image:altThis seems really similar to #35, although that issue was closed and I’m experiencing this today. I’m not clear on whether that issue was actually solved since there was no feedback at the end of t...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameMr0grog
hostnamegithub.com
expected-hostnamegithub.com
Nonee25fbd830d3e6ad0e8935c105cf4fdc70c593dffcd27bb45b790ae4fece5ea55
turbo-cache-controlno-preview
go-importgithub.com/logtail/logtail-python git https://github.com/logtail/logtail-python.git
octolytics-dimension-user_id78922349
octolytics-dimension-user_loginlogtail
octolytics-dimension-repository_id351343852
octolytics-dimension-repository_nwologtail/logtail-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id351343852
octolytics-dimension-repository_network_root_nwologtail/logtail-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release08c8f17f9d27054a108b8b6254e405de0151f660
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/logtail/logtail-python/issues/42#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Flogtail%2Flogtail-python%2Fissues%2F42
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Flogtail%2Flogtail-python%2Fissues%2F42
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=logtail%2Flogtail-python
Reloadhttps://github.com/logtail/logtail-python/issues/42
Reloadhttps://github.com/logtail/logtail-python/issues/42
Reloadhttps://github.com/logtail/logtail-python/issues/42
logtailhttps://github.com/logtail
logtail-pythonhttps://github.com/logtail/logtail-python
Codehttps://github.com/logtail/logtail-python
Issues1 (1)https://github.com/logtail/logtail-python/issues
Pull requestshttps://github.com/logtail/logtail-python/pulls
Actionshttps://github.com/logtail/logtail-python/actions
Security and qualityhttps://github.com/logtail/logtail-python/security
Insightshttps://github.com/logtail/logtail-python/pulse
https://github.com/logtail/logtail-python
New issuehttps://github.com/login?return_to=https://github.com/logtail/logtail-python/issues/42
New issuehttps://github.com/login?return_to=https://github.com/logtail/logtail-python/issues/42
Django dev server hangs at startup due to deadlock between LogtailHandler.flush() and flusher/uploaderhttps://github.com/logtail/logtail-python/issues/42#top
https://github.com/PetrHeinz
https://github.com/Mr0grog
Mr0groghttps://github.com/Mr0grog
on Sep 5, 2026https://github.com/logtail/logtail-python/issues/42#issue-5355788754
#35https://github.com/logtail/logtail-python/issues/35
#35https://github.com/logtail/logtail-python/issues/35
logtail-python/logtail/flusher.pyhttps://github.com/logtail/logtail-python/blob/7a7f3f77dc8f69dfc187cf28595e3d584f21c5e7/logtail/flusher.py#L90-L94
7a7f3f7https://github.com/logtail/logtail-python/commit/7a7f3f77dc8f69dfc187cf28595e3d584f21c5e7
takes the logging._lock lockhttps://github.com/python/cpython/blob/4061bc4c35f7c26f25264666d4ba083b93d2f6f9/Lib/logging/config.py#L534-L544
takes the lock if it has not yet cached information about what level it is set athttps://github.com/python/cpython/blob/v3.13.15/Lib/logging/__init__.py#L1769-L1786
linear-codehttps://github.com/apps/linear-code
PetrHeinzhttps://github.com/PetrHeinz
on Sep 5, 2026https://github.com/logtail/logtail-python/issues/42#event-30608207020
https://github.com/PetrHeinz
PetrHeinzhttps://github.com/PetrHeinz
on Sep 25, 2026https://github.com/logtail/logtail-python/issues/42#issuecomment-5833547849
@Mr0groghttps://github.com/Mr0grog
0.4.2https://github.com/logtail/logtail-python/releases/tag/v0.4.2
#54https://github.com/logtail/logtail-python/pull/54
0.4.3https://github.com/logtail/logtail-python/releases/tag/v0.4.3
PetrHeinzhttps://github.com/PetrHeinz
completedhttps://github.com/logtail/logtail-python/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Sep 25, 2026https://github.com/logtail/logtail-python/issues/42#event-31840064697
Sign up for freehttps://github.com/signup?return_to=https://github.com/logtail/logtail-python/issues/42
Sign in to commenthttps://github.com/login?return_to=https://github.com/logtail/logtail-python/issues/42
PetrHeinzhttps://github.com/PetrHeinz
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.