René's URL Explorer Experiment

Title: DownloadManager: add redact_url= kwarg to hide auth-bearing URLs in logs by bitcoin3us · Pull Request #136 · MicroPythonOS/MicroPythonOS · GitHub

Open Graph Title: DownloadManager: add redact_url= kwarg to hide auth-bearing URLs in logs by bitcoin3us · Pull Request #136 · MicroPythonOS/MicroPythonOS

X Title: DownloadManager: add redact_url= kwarg to hide auth-bearing URLs in logs by bitcoin3us · Pull Request #136 · MicroPythonOS/MicroPythonOS

Description: Summary DownloadManager.download_url prints the full request URL three times per download (start, finished, exception path) plus the entire response-headers dict. Great for the typical callers fetching public URLs (app icons, OS updates, weather), but it silently leaks any auth secret embedded in the URL's path or query string. Two real cases motivating this PR: Lightning Piggy's on-chain wallet (LightningPiggyApp PR #25) queries https://btc1.trezor.io/api/v2/xpub/zpub6q…?details=txs&tokens=derived to fetch balance + transactions for a watch-only xpub. The xpub itself sits in the URL path. A leaked xpub exposes the wallet's entire past + future address derivation tree to whoever reads the log — non-recoverable confidentiality damage even though funds custody is unaffected. The app already scrubs xpubs from its own print() calls, but DownloadManager's prints sit below that and re-leak on every poll. Any future caller using API-key-in-URL or token-in-URL auth. Common pattern (LNBits, BlueWallet's LNDHub, some HTTPS RPC endpoints) — the leak surface is wider than just Lightning Piggy. Behaviour Adds a redact_url=False kwarg (default preserves current behaviour). When True: URL logged as scheme://host[:port]/...REDACTED... instead of full. The host is intentionally kept so failure triage (DNS / connectivity / wrong endpoint) is still possible. Response-headers dump is suppressed entirely (replaced with ). Response headers often carry set-cookie, cf-ray, and other correlatable session tokens. Exception messages have the URL substring scrubbed before printing (aiohttp's ClientConnectorError typically embeds the URL). Default False is deliberate — most callers fetch public URLs and want the full log line for diagnostics. Callers explicitly opt in: await DownloadManager.download_url(url, redact_url=True) Tests tests/test_download_manager.py — added 3 new test classes (10 tests): TestSafeUrl (6 tests) for the new _safe_url helper — path-with-query strip, port handling, naked-host pass-through, malformed-URL placeholder, belt-and-braces "secret substrings never appear in output" guard. TestRedactUrlKwarg (2 tests) confirming the kwarg flows through the call surface (sync wrapper + async impl + mock) and defaults to False. TestRedactedExceptionPath (2 tests, commit 64963f9) — end-to-end coverage of the URL-bearing exception scrubbing path. Installs a fake aiohttp into sys.modules so the function-local import aiohttp resolves to a stub whose ClientSession.get() raises RuntimeError(message) with the full URL in the message (mimics aiohttp's ClientConnectorError without needing the network), then captures print(...) and asserts the secret-bearing substring is scrubbed under redact_url=True and preserved under redact_url=False. All 22 existing tests continue to pass (default behaviour unchanged): $ bash tests/unittest.sh tests/test_download_manager.py Ran 32 tests OK Follow-up Lightning Piggy's on-chain wallet (PR #25) will follow up on its side to pass redact_url=True once this lands. That's a one-line change in onchain_wallet.py:fetch_balance_and_payments(). Test plan All 32 tests pass on macOS desktop (bash tests/unittest.sh tests/test_download_manager.py) Existing tests (default behaviour) unchanged URL-bearing exception path covered by unit tests (commit 64963f9): fake-aiohttp injection deterministically exercises the except Exception as e: ... url in err_str branch and verifies both the scrubbed (redact_url=True) and preserved (redact_url=False) print output Hardware end-to-end smoke test: an LP onchain wallet call with redact_url=True shows https://btc1.trezor.io/...REDACTED... in serial logs. Gated on LightningPiggy #25 follow-up adding the kwarg — will tick once that lands and the integrated build is exercised on device 🤖 Generated with Claude Code

Open Graph Description: Summary DownloadManager.download_url prints the full request URL three times per download (start, finished, exception path) plus the entire response-headers dict. Great for the typical callers fetc...

X Description: Summary DownloadManager.download_url prints the full request URL three times per download (start, finished, exception path) plus the entire response-headers dict. Great for the typical callers fetc...

Opengraph URL: https://github.com/MicroPythonOS/MicroPythonOS/pull/136

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width