René's URL Explorer Experiment

Title: Retry transient full-scan upload failures (502/503/504/408, dropped connections) by mtorp · Pull Request #232 · SocketDev/socket-python-cli · GitHub

Open Graph Title: Retry transient full-scan upload failures (502/503/504/408, dropped connections) by mtorp · Pull Request #232 · SocketDev/socket-python-cli

X Title: Retry transient full-scan upload failures (502/503/504/408, dropped connections) by mtorp · Pull Request #232 · SocketDev/socket-python-cli

Description: Problem The full-scan upload (POST /v0/orgs//full-scans) makes exactly one attempt. When it fails, the entire run dies — for --reach runs that means a completed multi-hour tier-1 reachability analysis produces no scan and no report. Uploads can fail transiently at the gateway/connection level — an HTTP 502/503/504/408, a dropped or reset connection, or a client-side timeout — without the server having created the scan. A retried upload almost always succeeds. Change Core.create_full_scan now retries the fullscans.post call: Up to 3 total attempts, waiting ~10s before attempt 2 and ~30s before attempt 3 (plus up to 2s jitter so CI fleets don't retry in lock-step), with a log.warning before each retry. Retries transient failures only, as classified by the SDK: requires socketdev>=3.3.0 (SocketDev/socket-sdk-python#93), where do_request records the HTTP status code on every exception it raises and APIFailure.is_transient_error() owns the classification — true for HTTP 408/502/503/504, dropped/reset connections, and client-side timeouts. The CLI encodes no knowledge of the SDK exception hierarchy and parses no message text, so SDK restructuring can't silently break the classification. Never retried: deterministic errors (400/401/403/404/429), responses that came back with an error payload, and unexpected errors wrapped in a bare APIFailure. Note: uv.lock is regenerated once socketdev 3.3.0 is released to PyPI (blocked on SocketDev/socket-sdk-python#93); CI is expected to fail on dependency resolution until then. Safety No duplicate scans: in these failure modes the server never finished reading the request body, so no scan was created — a retry cannot duplicate one. In the rare case where a gateway timeout races a request the server later completes, the extra scan is benign and superseded by the retry (as if the CLI had run twice). Files are rebuilt per attempt: fullscans.post(use_lazy_loading=True) rebuilds its LazyFileLoaders from the plain paths on every call, so re-invoking it is safe (the loaders from a failed attempt are exhausted/closed and must not be reused). Temp .br facts files survive retries: the retry loop lives inside the existing try/finally, so the brotli-compressed .socket.facts.json.br siblings are only cleaned up after the last attempt. Testing New tests/unit/test_full_scan_retry.py (16 tests): retry on 502 then success, exhausting attempts re-raises, retry on catch-all 408/503/504, retry on connection-level errors, no retry on 400 / dedicated 4xx classes / error payloads, .br temp-file lifecycle across retries (success and exhaustion), and that the retry decision delegates to APIFailure.is_transient_error() (the classification truth table itself is tested in the SDK, next to the code that raises). Full suite against socketdev 3.3.0: 364 passed, 2 pre-existing skips.

Open Graph Description: Problem The full-scan upload (POST /v0/orgs/<org>/full-scans) makes exactly one attempt. When it fails, the entire run dies — for --reach runs that means a completed multi-hour tier-1 reachab...

X Description: Problem The full-scan upload (POST /v0/orgs/&lt;org&gt;/full-scans) makes exactly one attempt. When it fails, the entire run dies — for --reach runs that means a completed multi-hour tier-1...

Opengraph URL: https://github.com/SocketDev/socket-python-cli/pull/232

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width