René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Enabling zstd bundles for GHES seems to have broken proxy support on Enterprise Cloud with self-hosted runners","articleBody":"It looks like a recent change to enable zstd bundles on GHES has broken CodeQL scanning for anyone on Enterprise Cloud with self-hosted runners that are behind a proxy that is configured by the normal `https_proxy` type system variables.\r\n\r\nI did some investigation to understand why, and it appears that in the commit https://github.com/github/codeql-action/commit/33f2dc57a419743d4e5f95dab4045a64d0d76809 the code path used now always uses the streaming version as the feature flag that previously prevented it has been removed.\r\n\r\nThe main problem seems to be that the change from v3.27.0 to v3.27.1 has switched http libraries and broken the previous proxy support.\r\n\r\nSpecifically, in the working v3.27.0 version, the feature flag checked here:\r\nhttps://github.com/github/codeql-action/blob/v3.27.0/lib/tools-download.js#L59\r\nwasn't triggered and therefore it used this code path:\r\nhttps://github.com/github/codeql-action/blob/v3.27.0/lib/tools-download.js#L73\r\nwhich downloads via the `tool-cache`:\r\nhttps://github.com/actions/toolkit/blob/main/packages/tool-cache/src/tool-cache.ts\r\nand this respects the system proxy variables as you can see in:\r\nhttps://github.com/actions/toolkit/blob/main/packages/http-client/src/proxy.ts\r\n\r\nUnfortunately, now the feature flag is removed in v3.27.1:\r\nhttps://github.com/github/codeql-action/blob/v3.27.1/lib/tools-download.js#L62\r\nit always uses the streaming method, but this does not use the [toolkit http-client](https://github.com/actions/toolkit/tree/main/packages/http-client) that includes system proxy support but instead appears to use the `follow_directs` library here:\r\nhttps://github.com/github/codeql-action/blob/v3.27.1/lib/tools-download.js#L62\r\nwhich appears to be a small wrapper over the basic node fetch functionality and does not support proxies as far as I can make out.\r\n\r\nThis ultimately has meant that we've had to revert to v3.27.0 functionality to get CodeQL scanning in our Enterprise Cloud self-hosted runners working again.\r\n\r\nAs a short term fix, is it possible this feature flag can be brought back until proxy support has been added to the new streaming method?","author":{"url":"https://github.com/vyadh","@type":"Person","name":"vyadh"},"datePublished":"2024-11-12T09:53:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/2593/codeql-action/issues/2593"}