René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Table format has been broken when output column contains east asian fullwidth characters","articleBody":"Steps to reproduce\r\n------------------\r\n\r\n```powershell\r\n# make directories contains east asian fullwidth characters.\r\n# in this example, I use Japanese.\r\nmkdir .\\Test\r\ncd .\\Test\r\nmkdir Hello\r\nmkdir HelloWorld\r\nmkdir Helloワールド\r\nmkdir ハロー\r\nmkdir ハローワールド\r\n\r\n# invoke Get-ChildItem (output table format by default)\r\nGet-ChildItem\r\n```\r\n\r\nExpected behavior\r\n-----------------\r\n\r\nTable layout is not broken.\r\n\r\ne.g. on Windows PowerShell 5.1\r\n\r\n![error02](https://user-images.githubusercontent.com/720127/36887317-00949454-1e34-11e8-8916-df60343f4ab3.png)\r\n\r\nActual behavior\r\n---------------\r\n\r\nTable layout has been broken.  \r\n\r\n![error01](https://user-images.githubusercontent.com/720127/36887330-0c15b8b2-1e34-11e8-91a5-3d1ebf733075.png)\r\n\r\nIn this case, the last column(`Name`) is broken.  \r\nBut the column other than the last column will also be broken.  \r\n\r\n![error03](https://user-images.githubusercontent.com/720127/36887340-17f63008-1e34-11e8-8d07-ec9b0c6f4c88.png)\r\n\r\nEnvironment data\r\n----------------\r\n\r\n```powershell\r\n\u003e $PSVersionTable\r\nName                           Value\r\n----                           -----\r\nPSVersion                      6.0.1\r\nPSEdition                      Core\r\nGitCommitId                    v6.0.1\r\nOS                             Microsoft Windows 10.0.16299\r\nPlatform                       Win32NT\r\nPSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}\r\nPSRemotingProtocolVersion      2.3\r\nSerializationVersion           1.1.0.1\r\nWSManStackVersion              3.0\r\n```\r\n\r\nThis problem occurs on all platforms.\r\n\r\nCause\r\n----------------\r\n\r\nThe root cause is [ConsoleControl.LengthInBufferCells()](https://github.com/PowerShell/PowerShell/blob/v6.0.1/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleControl.cs#L2763) method returns invalid value when `str` parameter contains east asian fullwidth characters.  \r\n\r\nhttps://github.com/PowerShell/PowerShell/blob/67c9ae47f81a38ee3fcc1e16b187b79fb9cbe148/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleControl.cs#L2763-L2786\r\n\r\nWhen [TableWriter](https://github.com/PowerShell/PowerShell/blob/v6.0.1/src/System.Management.Automation/FormatAndOutput/common/TableWriter.cs) creates table row line, `ConsoleControl.LengthInBufferCells()` method is called to get row character width.\r\n\r\n`ConsoleControl.LengthInBufferCells()` method is expected to return a value according to the width of the character.  \r\nGenerally, east asian fullwidth characters(e.g. Japanese) are expected to return 2 or more per a character, but current version of `ConsoleControl.LengthInBufferCells()` simply returns character length.\r\n\r\n```csharp\r\n// length expected 6(3 fullwidth characters), but actual returns 3.\r\nConsoleControl.LengthInBufferCells(\"ハロー\", 0, false); \r\n```\r\n\r\nSo, the padding in the last column(`Name`) has be greater than expected and then Table layout has been broken.\r\n\r\n```powershell\r\n# expected 120 characters width\r\n\"d-----       2018/03/02     15:52                Hello                                                                 \"\r\n# actual 123 characters width\r\n\"d-----       2018/03/02     15:52                ハロー                                                                   \"            \r\n```\r\n\r\nReference 1\r\n----------------\r\n\r\nThe past version of `ConsoleControl.LengthInBufferCells()` method calls [LengthInBufferCellsFE()](https://github.com/PowerShell/PowerShell/blob/v6.0.0-alpha.9/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleControl.cs#L2875) method internally.  (FE means Far East?)\r\n\r\nhttps://github.com/PowerShell/PowerShell/blob/f46284414e9dda57eff3cdba7e61570a9a83e27e/src/Microsoft.PowerShell.ConsoleHost/host/msh/ConsoleControl.cs#L2875-L3036\r\n\r\nThis `LengthInBufferCellsFE()` method handles the length of east asian fullwidth characters correctly, but it was removed at f604b0e3f063c7a89db025f11b320e5c9797c6e0 .\r\n\r\nReference 2\r\n----------------\r\n\r\nTechnical report about East Asian Width.\r\n\r\n* [East Asian Width](http://unicode.org/reports/tr11/)\r\n","author":{"url":"https://github.com/stknohg","@type":"Person","name":"stknohg"},"datePublished":"2018-03-02T07:19:23.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":7},"url":"https://github.com/6290/PowerShell/issues/6290"}