René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Retries in WebCmdlet does not respect Retry-After header in 429 status","articleBody":"### Prerequisites\n\n- [X] Write a descriptive title.\n- [X] Make sure you are able to repro it on the [latest released version](https://github.com/PowerShell/PowerShell/releases)\n- [X] Search the existing issues.\n- [X] Refer to the [FAQ](https://github.com/PowerShell/PowerShell/blob/master/docs/FAQ.md).\n- [X] Refer to [Differences between Windows PowerShell 5.1 and PowerShell](https://docs.microsoft.com/powershell/scripting/whats-new/differences-from-windows-powershell).\n\n### Steps to reproduce\n\nThe improvements of the PowerShell 7.4.0 (Preview) notes that WebCmdlets (`Invoke-WebRequest` and `Invoke-RestMethod`) follow the retry interval specified in the Retry-After header for status code 429 (Too Many Requests), but it does not actually follow it.\r\n\r\nI tested this by running a server on the local machine that always returns status 429 and Retry-After: 2.\r\n\r\nWhen I run the `Invoke-WebRequest` with `-MaximumRetryCount 2`, it retries with the default retry interval of 5 seconds instead of the 2 seconds specified in Retry-After header value.\r\n\r\n```PowerShell\r\nPS C:\\\u003e Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCount 2 -Verbose\r\nVERBOSE: HTTP/1.1 GET with 0-byte payload\r\nVERBOSE: Retrying after interval of 5 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: Retrying after interval of 5 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: received 5-byte response of content type text/plain\r\n\r\nPS C:\\\u003e $Error[0].Exception.Response.StatusCode.Value__\r\n429\r\n\r\nPS C:\\\u003e $Error[0].Exception.Response.Headers\r\n\r\nKey         Value\r\n---         -----\r\nRetry-After {2}\r\nServer      {Microsoft-HTTPAPI/2.0}\r\nDate        {Sat, 06 May 2023 17:25:42 GMT}\r\n```\r\n\r\nI try the same thing with `curl 8.0.1`, the retry is performed correctly at 2 second intervals.\r\n\r\n```cmd\r\nC:\\\u003e curl.exe --retry 2 \"http://localhost:8080/test\"\r\nRetryWarning: Problem : HTTP error. Will retry in 2 seconds. 2 retries left.\r\nRetryWarning: Problem : HTTP error. Will retry in 2 seconds. 1 retries left.\r\n```\r\n\r\nSince this change was implemented in PR #18717, I checked the source code and found an mistake in the code. It executes the retry logic at 409 (Conflict) instead of 429.\r\n\r\nhttps://github.com/CarloToso/PowerShell/blob/07e041fe10d4c753dfccfa46008bdedebdfd3027/src/Microsoft.PowerShell.Commands.Utility/commands/utility/WebCmdlet/Common/WebRequestPSCmdlet.Common.cs#L1340-L1343\r\n\n\n### Expected behavior\n\n```console\nPS C:\\\u003e Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCount 2 -Verbose\r\nVERBOSE: HTTP/1.1 GET with 0-byte payload\r\nVERBOSE: Retrying after interval of 2 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: Retrying after interval of 2 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: received 5-byte response of content type text/plain\n```\n\n\n### Actual behavior\n\n```console\nPS C:\\\u003e Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCount 2 -Verbose\r\nVERBOSE: HTTP/1.1 GET with 0-byte payload\r\nVERBOSE: Retrying after interval of 5 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: Retrying after interval of 5 seconds. Status code for previous attempt: TooManyRequests\r\nVERBOSE: received 5-byte response of content type text/plain\n```\n\n\n### Error details\n\n```console\nPS C:\\\u003e Get-Error\r\n\r\nException             :\r\n    Type       : Microsoft.PowerShell.Commands.HttpResponseException\r\n    Response   : StatusCode: 429, ReasonPhrase: 'Too Many Requests', Version: 1.1, Content: System.Net.Http.HttpConnect\r\nionResponseContent, Headers:\r\n                 {\r\n                 Retry-After: 2\r\n                 Server: Microsoft-HTTPAPI/2.0\r\n                 Date: Sat, 06 May 2023 17:25:42 GMT\r\n                 Content-Length: 5\r\n                 Content-Type: text/plain\r\n                 }, Trailing Headers:\r\n                 {\r\n                 }\r\n    StatusCode : TooManyRequests\r\n    TargetSite :\r\n        Name          : ThrowTerminatingError\r\n        DeclaringType : System.Management.Automation.MshCommandRuntime, System.Management.Automation, Version=7.4.0.3,\r\nCulture=neutral, PublicKeyToken=31bf3856ad364e35\r\n        MemberType    : Method\r\n        Module        : System.Management.Automation.dll\r\n    Message    : Response status code does not indicate success: 429 (Too Many Requests).\r\n    Source     : System.Management.Automation\r\n    HResult    : -2146233088\r\n    StackTrace :\r\n   at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord)\r\nTargetObject          : Method: GET, RequestUri: 'http://localhost:8080/test', Version: 1.1, Content: \u003cnull\u003e, Headers:\r\n                        {\r\n                        User-Agent: Mozilla/5.0\r\n                        User-Agent: (Windows NT 10.0; Microsoft Windows 10.0.22621; ja-JP)\r\n                        User-Agent: PowerShell/7.4.0\r\n                        Accept-Encoding: gzip\r\n                        Accept-Encoding: deflate\r\n                        Accept-Encoding: br\r\n                        }\r\nCategoryInfo          : InvalidOperation: (Method: GET, Reques…ept-Encoding: br\r\n                        }:HttpRequestMessage) [Invoke-WebRequest], HttpResponseException\r\nFullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand\r\nErrorDetails          : Retry\r\nInvocationInfo        :\r\n    MyCommand        : Invoke-WebRequest\r\n    ScriptLineNumber : 1\r\n    OffsetInLine     : 1\r\n    HistoryId        : 34\r\n    Line             : Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCount 2 -Verbose\r\n    Statement        : Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCount 2 -Verbose\r\n    PositionMessage  : At line:1 char:1\r\n                       + Invoke-Webrequest -Uri \"http://localhost:8080/test\" -MaximumRetryCoun …\r\n                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    InvocationName   : Invoke-Webrequest\r\n    CommandOrigin    : Internal\r\nScriptStackTrace      : at \u003cScriptBlock\u003e, \u003cNo file\u003e: line 1\n```\n\n\n### Environment data\n\n```powershell\nPS C:\\\u003e $PSVersionTable\r\n\r\nName                           Value\r\n----                           -----\r\nPSVersion                      7.4.0-preview.3\r\nPSEdition                      Core\r\nGitCommitId                    7.4.0-preview.3\r\nOS                             Microsoft Windows 10.0.22621\r\nPlatform                       Win32NT\r\nPSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}\r\nPSRemotingProtocolVersion      2.3\r\nSerializationVersion           1.1.0.1\r\nWSManStackVersion              3.0\n```\n\n\n### Visuals\n\n_No response_","author":{"url":"https://github.com/mkht","@type":"Person","name":"mkht"},"datePublished":"2023-05-06T17:45:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/19621/PowerShell/issues/19621"}