Title: Feature Request: Add OutputType parameter to Import-Csv · Issue #8862 · PowerShell/PowerShell · GitHub
Open Graph Title: Feature Request: Add OutputType parameter to Import-Csv · Issue #8862 · PowerShell/PowerShell
X Title: Feature Request: Add OutputType parameter to Import-Csv · Issue #8862 · PowerShell/PowerShell
Description: Summary of the new feature/enhancement By using a concrete type, instead of PSObject, both import speed and memory usage can be improved. As it is today, Import-Csv is almost useless for larger datasets, since the overhead of our NotePro...
Open Graph Description: Summary of the new feature/enhancement By using a concrete type, instead of PSObject, both import speed and memory usage can be improved. As it is today, Import-Csv is almost useless for larger dat...
X Description: Summary of the new feature/enhancement By using a concrete type, instead of PSObject, both import speed and memory usage can be improved. As it is today, Import-Csv is almost useless for larger dat...
Opengraph URL: https://github.com/PowerShell/PowerShell/issues/8862
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Feature Request: Add OutputType parameter to Import-Csv ","articleBody":"# Summary of the new feature/enhancement\r\n\r\nBy using a concrete type, instead of `PSObject`, both import speed and memory usage can be improved. \r\n\r\nAs it is today, Import-Csv is almost useless for larger datasets, since the overhead of our NoteProperties is 48 bytes, not counting the name and the value. When the imported values are integers, that is a blowup-factor of ~20.\r\n\r\nThese numbers are from my prototype:\r\n\r\n| Destination | Time | Memory used |\r\n| ----- | :-----: | :------: |\r\n| PSObject | 01:34 | 19Gb |\r\n| class with int props | 01:02 | 490Mb |\r\n| class with string props | 00:10 | 4.2Gb |\r\n\r\nBy keeping it as strings, the import speed is vastly improved. By converting to integers, the speed is still improved, and the memory requirements are vastly improved.\r\n\r\n# Proposed technical implementation details (optional)\r\n\r\nSee https://github.com/PowerShell/PowerShell/pull/8860.\r\n\r\nThe gist is to generate expression trees, that sets the properties or call the constructor, on an instance of the provided type.\r\n\r\nThe use of the constructor allows for custom type conversion, where there are no language conversions from string to the property type.\r\n\r\nThe type needs to have members that match the names of the columns in the CSV. \r\nMaybe we should provide a way of providing alternate headers to map to existing objects?\r\n\r\nData.csv\r\n```csv\r\nText, Integer,Date\r\nHi,42,2016-12-24\r\nBye,4711,2016-12-25\r\n```\r\n\r\n```powershell\r\nclass MyCsv {\r\n [string] $Text\r\n [int] $Integer\r\n [DateTime] $Date\r\n}\r\n$d = Import-Csv -OutputType ([MyCsv]) -Path .\\data.csv\r\n```\r\nI also implemented ctor calls, that takes precedence.\r\n\r\n```powershell\r\nclass MyCsv2 {\r\n MyCsv2([string] $text, [int] $integer, $date){\r\n $this.Name = $this.text\r\n $this.Number = $integer * 100\r\n $this.When = $dateTime\r\n }\r\n [string] $Name\r\n [long] $Number\r\n [DateTime] $When\r\n}\r\n$e = Import-Csv -OutputType ([MyCsv2]) -Path .\\data.csv\r\n```\r\n\r\nI would like to see a discussion about the feature set, error handling, names for parameters etc.","author":{"url":"https://github.com/powercode","@type":"Person","name":"powercode"},"datePublished":"2019-02-10T23:19:13Z","url":"https://github.com/PowerShell/PowerShell/issues/8862"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:c86bdf4c-5c5f-92a0-1915-204a0766cd26 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | A12E:126CB9:27E56AE:379E012:6AFD51DF |
| html-safe-nonce | 0fb885fd7d2643fb028a452904f07a9c57f4e03a31d4647ba42261ad39569a90 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMTJFOjEyNkNCOToyN0U1NkFFOjM3OUUwMTI6NkFGRDUxREYiLCJ2aXNpdG9yX2lkIjoiNTc3MjgzODczNTE2MjcyNDgzMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 22a752f90f8c827d29fece68ba617399772f893bfea905fa70594645acee63ca |
| hovercard-subject-tag | issue:408595733 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/PowerShell/PowerShell/8862/issue_layout |
| twitter:image | https://opengraph.githubassets.com/198cce626f34251b26850b432108cbb4ba06e07eeadfa029c3e04db3a9e8eed3/PowerShell/PowerShell/issues/8862 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/198cce626f34251b26850b432108cbb4ba06e07eeadfa029c3e04db3a9e8eed3/PowerShell/PowerShell/issues/8862 |
| og:image:alt | Summary of the new feature/enhancement By using a concrete type, instead of PSObject, both import speed and memory usage can be improved. As it is today, Import-Csv is almost useless for larger dat... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | powercode |
| hostname | github.com |
| expected-hostname | github.com |
| None | 66653c2e55a6fff2930be1bdf07ee4f8aa137e3956d056c822563be6f878755e |
| turbo-cache-control | no-preview |
| go-import | github.com/PowerShell/PowerShell git https://github.com/PowerShell/PowerShell.git |
| octolytics-dimension-user_id | 11524380 |
| octolytics-dimension-user_login | PowerShell |
| octolytics-dimension-repository_id | 49609581 |
| octolytics-dimension-repository_nwo | PowerShell/PowerShell |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 49609581 |
| octolytics-dimension-repository_network_root_nwo | PowerShell/PowerShell |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | b6f01f7cb9f35f8c09949773241a3d49dd04b340 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width