René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Make console windows fully UTF-8 by default on Windows, in line with the behavior on Unix-like platforms - character encoding, code page","articleBody":"PowerShell Core now commendably defaults to UTF-8 encoding, including when sending strings _to_ _external_ programs, as reflected in `$OutputEncoding`'s default value.\r\n\r\nHowever, because the console-window shortcut file / taskbar entry still defaults to the OEM code page implied by the legacy system locale (e.g. `437` on US-English systems), it misinterprets strings _from_ external programs; e.g., with Node.js installed:\r\n\r\n```powershell\r\nPSCoreOnWin\u003e $captured = '€' | node -pe \"require('fs').readFileSync(0).toString().trim()\"; $captured\r\nΓé¼    # !! node's UTF-8 output was misinterpreted.\r\n```\r\n\r\nThis currently requires the following _workaround_ (in addition to requiring the console window to use a TrueType font (true by default on Windows 10)):\r\n\r\n```powershell\r\n[console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding\r\n```\r\n\r\n\u003csup\u003ePrepend `$OutputEncoding = ` to make a _Windows PowerShell_ console fully UTF-8-aware.\u003c/sup\u003e\r\n\r\nThe above implicitly switches to the UTF-8 code page (`65001`), as then reflected in `chcp`.\r\n\r\nThis obscure workaround shouldn't be necessary, and I think it would make sense for PowerShell to automatically set `[console]::InputEncoding` and `[console]::OutputEncoding` to (BOM-less) UTF-8 on startup.\r\n\r\n_Update_: When this issue was originally created, there was no mechanism for presetting code page `65001` (UTF-8) _system-wide_, which necessitated the awkward workaround. In recent versions of Windows 10 it _is_ now possible to switch to code page `65001` _as the system locale_ and therefore system-wide, although as of Windows 10 version 1909 that feature is still in _beta_ - see [this SO answer](https://stackoverflow.com/a/57134096/45375).\r\n\r\n  * _Caveat_: In addition to defaulting the _OEM_ code page to `65001` in all console windows (including `cmd.exe` windows), this invariably also makes _Windows PowerShell_'s  _ANSI_-encoding-default cmdlets default to UTF-8, notably `Get-Content` and `Set-Content`, which can be problematic from a backward-compatibility perspective. \r\nAdditionally, there is a _bug_ - [see below](https://github.com/PowerShell/PowerShell/issues/7233#issuecomment-640243647).\r\n\r\nThe change, which can also be made programmatically ([see below](https://github.com/PowerShell/PowerShell/issues/7233#issuecomment-640243647)), requires administrative privileges and a reboot.\r\n\r\nEnvironment data\r\n----------------\r\n\r\n```powershell\r\nPowerShell Core 7.1.0-preview.3 on Windows 10\r\n```\r\n","author":{"url":"https://github.com/mklement0","@type":"Person","name":"mklement0"},"datePublished":"2018-07-05T18:22:35.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":26},"url":"https://github.com/7233/PowerShell/issues/7233"}