René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Invoke-RestMethod -FollowRelLink does not work when links contain commas","articleBody":"### Prerequisites\r\n\r\n- [X] Write a descriptive title.\r\n- [X] Make sure you are able to repro it on the [latest released version](https://github.com/PowerShell/PowerShell/releases)\r\n- [X] Search the existing issues.\r\n- [X] Refer to the [FAQ](https://github.com/PowerShell/PowerShell/blob/master/docs/FAQ.md).\r\n- [X] Refer to [Differences between Windows PowerShell 5.1 and PowerShell](https://docs.microsoft.com/powershell/scripting/whats-new/differences-from-windows-powershell).\r\n\r\n### Steps to reproduce\r\n\r\nHere is a basic sample with PowerShell based on what the [current code seems to do](https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/WebCmdlet/Common/WebRequestPSCmdlet.Common.cs#L1865):\r\n\r\n```powershell\r\n[string[]]$Links = \"\u003chttps://some.api.com/api/item?fields=id,description,name\u0026offset=0\u0026limit=1000\u003e;rel=`\"first`\",\u003chttps://some.api.com/api/item?fields=id,short_description,name\u0026offset=1000\u0026limit=1000\u003e;rel=`\"next`\",\u003chttps://some.api.com/api/item?fields=id,short_description,name\u0026offset=4000\u0026limit=1000\u003e;rel=`\"last`\"\"\r\n\r\n$Links[0].Split(',')\r\n```\r\n\r\n**Result**\r\n\r\n```\r\n\u003chttps://some.api.com/api/item?fields=id\r\ndescription\r\nname\u0026offset=0\u0026limit=1000\u003e;rel=\"first\"\r\n\u003chttps://some.api.com/api/item?fields=id\r\nshort_description\r\nname\u0026offset=1000\u0026limit=1000\u003e;rel=\"next\"\r\n\u003chttps://some.api.com/api/item?fields=id\r\nshort_description\r\nname\u0026offset=4000\u0026limit=1000\u003e;rel=\"last\"\r\n```\r\n\r\n### Expected behavior\r\n\r\nInvoke-RestMethod can follow RelLink when a comma does appear within a header url. Although I feel the REST API provider (I discovered this with ServiceNow) should provide urlencoded links, I believe this cannot be relied upon and is an easy fix by using a `MatchCollection` instead.\r\n\r\n**Current** -  [ParseLinkHeader](https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/WebCmdlet/Common/WebRequestPSCmdlet.Common.cs#L1845-L1881)\r\n\r\n**Suggested**\r\n\r\n```csharp\r\ninternal void ParseLinkHeader(HttpResponseMessage response, System.Uri requestUri)\r\n{\r\n    if (_relationLink == null)\r\n    {\r\n        // Must ignore the case of relation links. See RFC 8288 (https://tools.ietf.org/html/rfc8288)\r\n        _relationLink = new Dictionary\u003cstring, string\u003e(StringComparer.OrdinalIgnoreCase);\r\n    }\r\n    else\r\n    {\r\n        _relationLink.Clear();\r\n    }\r\n\r\n    // we only support the URL in angle brackets and `rel`, other attributes are ignored\r\n    // user can still parse it themselves via the Headers property\r\n    const string pattern = \"\u003c(?\u003curl\u003e.*?)\u003e;\\\\s*rel=(?\u003cquoted\u003e\\\")?(?\u003crel\u003e(?(quoted).*?|[^,;]*))(?(quoted)\\\")\";\r\n    IEnumerable\u003cstring\u003e links;\r\n    if (response.Headers.TryGetValues(\"Link\", out links))\r\n    {\r\n        foreach (string linkHeader in links)\r\n        {\r\n            MatchCollection matchCollection = Regex.Matches(link, pattern)\r\n            foreach (Match match in matchCollection)\r\n            {\r\n                if (match.Success)\r\n                {\r\n                    string url = match.Groups[\"url\"].Value;\r\n                    string rel = match.Groups[\"rel\"].Value;\r\n                    if (url != string.Empty \u0026\u0026 rel != string.Empty \u0026\u0026 !_relationLink.ContainsKey(rel))\r\n                    {\r\n                        Uri absoluteUri = new(requestUri, url);\r\n                        _relationLink.Add(rel, absoluteUri.AbsoluteUri);\r\n                    }\r\n                }\r\n            }\r\n        }\r\n    }\r\n}\r\n```\r\n\r\n\r\n### Actual behavior\r\n\r\nSee Steps To Reproduce. Splitting the links returned by a comma breaks the links before they can be parsed by the regex.\r\n\r\n### Error details\r\n\r\nN/A\r\n\r\n### Environment data\r\n\r\n7+\r\n\r\n### Visuals\r\n\r\n_No response_","author":{"url":"https://github.com/ashscodes","@type":"Person","name":"ashscodes"},"datePublished":"2022-06-15T08:47:31.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/17528/PowerShell/issues/17528"}