René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG] Intermittent EOF errors calling S3 ListObjects from GitHub Actions due to Go HTTP keep-alive / HTTP2 interaction","articleBody":"SUMMARY\n\nStackQL intermittently fails when listing S3 bucket objects only in GitHub Actions, while the same credentials and queries work reliably from a local laptop.\n\nExample failure:\n\nGet \"https://s3.ap-southeast-2.amazonaws.com/stackql-trial-bucket-02?max-keys=1000\n\": EOF\n\nThis is not an AWS service defect and not a credentials issue.\nThe root cause is Go’s default HTTP client connection reuse behavior (keep-alive / HTTP2) interacting poorly with the GitHub Actions network environment when using raw HTTP + SigV4 signing (no AWS SDK).\n\nENVIRONMENT\n\nStackQL using any-sdk HTTP client\n\nAWS SigV4 signing only (no AWS SDK HTTP client usage)\n\nSame AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY locally and in GitHub Actions\n\nRegion explicitly provided to SigV4 signer\n\nFails only in GitHub Actions\n\nError manifests as immediate EOF while reading response\n\nWHY THIS HAPPENS\n\nGitHub Actions runners aggressively reuse TCP connections\n\nNAT, proxying, and idle connection reaping are common\n\nServer-side connection close is legal and frequent\n\nGo net/http default transport enables:\n\nHTTP keep-alives\n\nHTTP/2\n\nIdle connection reuse\n\nA reused but half-dead connection results in:\n\nConnection closed by server\n\nClient receives EOF before response body\n\nAWS S3 is behaving within specification\n\nEarly connection close is permitted\n\nThis is a client-side transport robustness issue\n\nThis does not reproduce locally\n\nLocal networks are more tolerant of connection reuse\n\nCI networking is not\n\nRetrying is not a viable solution\n\nStackQL fans out across tens or hundreds of endpoints\n\nRetrying on EOF causes request amplification and latency explosion\n\nThis is a transport correctness problem, not a transient API failure\n\nEVIDENCE\n\nSame credentials, same query, same endpoint succeed locally\n\nFailures occur only in GitHub Actions\n\nDisabling HTTP/2 via GODEBUG reduces failures\n\nEliminating connection reuse eliminates failures entirely\n\nCORRECT FIX (VENDOR-AGNOSTIC)\n\nExplicitly control the Go HTTP transport used by any-sdk / StackQL and disable connection reuse for AWS endpoints.\n\nProposed transport configuration:\n\ntransport := \u0026http.Transport{\n    DisableKeepAlives: true,\n    ForceAttemptHTTP2: false,\n}\n\nclient := \u0026http.Client{\n    Transport: transport,\n}\n\n\nEffects:\n\nOne request per TCP connection\n\nNo reuse of half-closed sockets\n\nDeterministic behavior in CI\n\nNo AWS SDK dependency\n\nNo retries required\n\nSCOPE\n\nApply to AWS providers (or make configurable per provider)\n\nDoes not require AWS environment variables\n\nPreserves StackQL vendor-independence\n\nAcceptable performance tradeoff for control-plane style queries\n\nNON-SOLUTIONS (INTENTIONALLY AVOIDED)\n\nAdding retries on EOF\n\nIntroducing AWS SDK HTTP client\n\nRequiring AWS_REGION / AWS_DEFAULT_REGION\n\nCI-specific bash hacks\n\nTreating this as an AWS outage or service bug\n\nACTION ITEMS\n\nAdd explicit HTTP transport ownership in any-sdk\n\nDisable keep-alives and HTTP/2 for AWS providers\n\nDocument rationale (CI + NAT behavior)\n\nAdd CI regression coverage if possible","author":{"url":"https://github.com/general-kroll-4-life","@type":"Person","name":"general-kroll-4-life"},"datePublished":"2025-12-21T23:45:59.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/599/stackql/issues/599"}