René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Quoting in subexpressions in string literals is quite confused","articleBody":"### Prerequisites\r\n\r\n- [X] Write a descriptive title.\r\n- [X] Make sure you are able to repro it on the [latest released version](https://github.com/PowerShell/PowerShell/releases)\r\n- [X] Search the existing issues.\r\n- [X] Refer to the [FAQ](https://github.com/PowerShell/PowerShell/blob/master/docs/FAQ.md).\r\n- [X] Refer to [Differences between Windows PowerShell 5.1 and PowerShell](https://docs.microsoft.com/powershell/scripting/whats-new/differences-from-windows-powershell).\r\n\r\n### Steps to reproduce\r\n\r\nI was attempting to write a function to escape a string to correctly print a string for generated powershell code.\r\nThe rules for escaping double-quotes inside a double-quoted string are obscure at best, and quite broken depending on your point of view.\r\n\r\nThe coloring in the powershell console appears to mostly match the code behavior, but tools like VSCode do not match powershell behavior.\r\n\r\nHere is a block of code demonstrating a number of issues\r\n#1: Escaping isn't required, but sometimes it is allowed\r\n#2: To have double-quotes inside an embedded string requires double backticks, yet only a single backtick is necessary for things like newline.\r\n#3: Parans inside a nested string confuse powershell\r\n#3: Doubling nested double-quotes seems to escape them, but there is a lot of wiggle room for how many double quotes is needed.\r\n```\r\n##########################################################################################\r\nWrite-Output \"Begin Sample 1\"\r\n# Inside a \"$()\" subexpression it turns out you don't need to escape quotes\r\n$b = \"'$($a.replace(\"'\", \"''\"))'\"\r\n\r\n# Oddly, PowerShell accepts the back-tick double-quote (`\") though...\r\n\"...$(\"hello\")...\"     # OK - as expected, escaping inner quotes isn't necessary\r\n\r\n# Yet, escaping is accepted by powershell, but confuses VSCode coloring.\r\n\"...$(`\"hello\")...\"    # OK, VSCode miscolors \"hello\" as not a string and open \"(\" as mismatched\r\n\"...$(`\"hello`\")...\"   # OK, VSCode miscolors \"hello\" as not a string\r\n\r\n# Escaping here even produces a newline character\r\n\"...$(`\"hello`n\")...\"  # Treats `n as newline.\r\n\r\nWrite-Output \"End of test 1\"\r\n\r\n##########################################################################################\r\nWrite-Output \"Begin Sample 2\"\r\n# To have embedded double-quotes inside the subexpression it is necessary to use Double-BackTick\r\n# Weirdly, doubling the back-ticks appears to behave like a single back-tick outside of a subexpression by escaping the double-quote.\r\nWrite-Output \"$(`\"hello``\"...``\" \")\" # Prints hello\"...\". Also confuses VSCode.\r\n\r\n# Write-Output \"$(\"hello`\"...`\" \")\" # Fails: You must provide a value expression following the '..' operator.\r\n\r\nWrite-Output $(\"hello`\"...`\" \") # Prints hello\"...\"\r\n\r\n\r\n##########################################################################################\r\nWrite-Output \"Begin Sample 3\"\r\n# Here, Powershell interprets the closing paran, inside quotes, as being not quoted.\r\n# this does not compile - even though the error message appears to correctly color the \"hello)\" portion as a string.\r\n# Also, back-ticks don't seem to be able to escape nested parans.\r\nWrite-Output \"$( Write-Output \"hello)\" )\"  # Does not run: The string is missing the terminator: \".\r\nWrite-Output \"$( Write-Output \"(hello)\" )\" # Prints  (hello)\r\n\r\n##########################################################################################\r\nWrite-Output \"Begin Sample 4\"\r\n# Funky handling of multiple double-quotes.\r\n# 0 to 2 double quotes: Not output\r\n# 3 to 6 double quotes: Single double-quote\r\n# 7 to 10 double quotes: Two double-quotes\r\n# 11 to 14 double quotes: Three double-quotes\r\n# For 2, 5, 10 and 14 the closing \")\" doesn't print either.\r\nWrite-Output \"$( Write-Output \" 2 hell(o\"\" 5+5)\")\"             # Prints:  2 hell(o 5+5      \r\nWrite-Output \"$( Write-Output \" 3 hell(o\"\"\" 5+5)\")\"            # Prints:  3 hell(o\" 5+5)    \r\nWrite-Output \"$( Write-Output \" 4 hell(o\"\"\"\" 5+5)\")\"           # Prints:  4 hell(o\" 5+5)    \r\nWrite-Output \"$( Write-Output \" 5 hell(o\"\"\"\"\" 5+5)\")\"          # Prints:  5 hell(o\" 5+5     \r\nWrite-Output \"$( Write-Output \" 7 hell(o\"\"\"\"\"\"\" 5+5)\")\"        # Prints:  7 hell(o\"\" 5+5)   \r\nWrite-Output \"$( Write-Output \"10 hell(o\"\"\"\"\"\"\"\"\"\" 5+5)\")\"     # Prints: 10 hell(o\"\" 5+5    \r\nWrite-Output \"$( Write-Output \"11 hell(o\"\"\"\"\"\"\"\"\"\"\" 5+5)\")\"    # Prints: 11 hell(o\"\"\" 5+5)  -- Also, this confuses VSCode coloring.\r\nWrite-Output \"$( Write-Output \"12 hell(o\"\"\"\"\"\"\"\"\"\"\"\" 5+5)\")\"   # Prints: 12 hell(o\"\"\" 5+5)  \r\nWrite-Output \"$( Write-Output \"14 hell(o\"\"\"\"\"\"\"\"\"\"\"\"\"\" 5+5)\")\" # Prints: 14 hell(o\"\"\" 5+5   \r\n\r\n# More samples of bad handling of nested parans.\r\n# In both cases there are no errors, but the text after the close \")\" is dropped.\r\nWrite-Output \"$(Write-Output \"he(llo\" )after )\"                        # Prints: he(llo\r\nWrite-Output \"$(Write-Output \"he(llo\" ) after$(Write-Output more)\"string)\"     # Prints: he(llo -- And VSCode color is confused: \" Extra quote to fix VSCode coloring\r\n\r\nWrite-Output \"End\"\r\n\r\n```\r\n\r\n### Expected behavior\r\n\r\n```console\r\nInside a \"$()\" subexpression, I would expect to have to escape double quotes\r\n\"$(`\"hello`\")\"\r\nBut, there appears to be some special case so that isn't necessary.\r\n\r\nThe problem is that this special case only works as expected in trivial cases like the above.\r\n```\r\n\r\n\r\n### Actual behavior\r\n\r\n```console\r\nit is difficult to determine how to properly escape a string in a nested expression.\r\nSometimes backticks are not needed, but accepted.\r\nSometimes they are required.\r\n\r\nAlso, parans inside a nested string are interpreted as not inside a string - they interfere with paran outside of the subexpression.\r\n```\r\n\r\n\r\n### Error details\r\n\r\n```console\r\nVarious, including:\r\nYou must provide a value expression following the '..' operator.\r\nThe string is missing the terminator: \".\r\n```\r\n\r\n\r\n### Environment data\r\n\r\n```powershell\r\nName                           Value\r\n----                           -----\r\nPSVersion                      7.2.6\r\nPSEdition                      Core\r\nGitCommitId                    7.2.6\r\nOS                             Microsoft Windows 10.0.19044\r\nPlatform                       Win32NT\r\nPSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}\r\nPSRemotingProtocolVersion      2.3\r\nSerializationVersion           1.1.0.1\r\nWSManStackVersion              3.0\r\n```\r\n\r\n\r\n### Visuals\r\n\r\n![image](https://user-images.githubusercontent.com/2498088/184441272-2aba7b65-7155-4b7a-a306-9861acd12129.png)\r\n\r\n![image](https://user-images.githubusercontent.com/2498088/184441302-f169c4bc-efdb-47c0-9e22-116951b6ccd5.png)\r\n\r\n![image](https://user-images.githubusercontent.com/2498088/184441371-93f631af-db96-409e-8d24-550a98dc520e.png)\r\n\r\nVSCode\r\n![image](https://user-images.githubusercontent.com/2498088/184682127-bd93136e-71ba-4da0-ac17-56591c2c8ff0.png)\r\n\r\n![image](https://user-images.githubusercontent.com/2498088/184682187-c758f12c-1176-4e7e-b37d-1e1a07e7a879.png)\r\n![image](https://user-images.githubusercontent.com/2498088/184682432-1a25e09d-c362-4475-b0b5-36e76ba5ad74.png)\r\n\r\n\r\n\r\n","author":{"url":"https://github.com/jporkka","@type":"Person","name":"jporkka"},"datePublished":"2022-08-12T20:38:47.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":12},"url":"https://github.com/17887/PowerShell/issues/17887"}