Title: `+=` copies array many times · Issue #23899 · PowerShell/PowerShell · GitHub
Open Graph Title: `+=` copies array many times · Issue #23899 · PowerShell/PowerShell
X Title: `+=` copies array many times · Issue #23899 · PowerShell/PowerShell
Description: Summary of the new feature / enhancement I was looking into += as the canonical example of inefficient powershell code for my presentation at psconfeu, and noticed that doing += on 10k items allocates 1.5 GB of memory and forces many gar...
Open Graph Description: Summary of the new feature / enhancement I was looking into += as the canonical example of inefficient powershell code for my presentation at psconfeu, and noticed that doing += on 10k items alloca...
X Description: Summary of the new feature / enhancement I was looking into += as the canonical example of inefficient powershell code for my presentation at psconfeu, and noticed that doing += on 10k items alloca...
Opengraph URL: https://github.com/PowerShell/PowerShell/issues/23899
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`+=` copies array many times","articleBody":"### Summary of the new feature / enhancement\n\nI was looking into `+=` as the canonical example of inefficient powershell code for my presentation at psconfeu, and noticed that doing `+=` on 10k items allocates 1.5 GB of memory and forces many garbage collections. This is way more allocations than the common wisdom of \"it copies array once for every item\" suggests.\r\n\r\n```powershell\r\n$start = [GC]::GetTotalAllocatedBytes($true)\r\n\r\n$numbers = 1..10000 \r\n$newNumbers = @()\r\nforeach ($number in $numbers) {\r\n $newNumbers += $number\r\n}\r\n\r\n$end = [GC]::GetTotalAllocatedBytes($true)\r\n\r\n\"$([System.Math]::Round(($end-$start) / 1GB, 2)) GB\"\r\n```\r\n\r\nThe relevant code in powershell:\r\n\r\nhttps://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs#L3519-L3536 \r\n\r\nThis code handles the array as enumerable of unknown size, instead of special casing it to avoid growing the list over and over as it resizes from the initial capacity of 4 items. So for example for the final items in my 10k array it grows the list 12 times per item.\r\n\r\nThis can be confirmed by profiling the code in VS allocation profiler, where many of the allocations are attributed to arrays that have sizes which are multiplies of 2.\r\n\r\n \n\n### Proposed technical implementation details (optional)\n\nas suggested by @SeeminglyScience \r\n\r\n> a simple optimization could be just adding a sizeHint param to AddObject\r\n\r\nOr \r\n\r\n> the binder, to emit different code gen based on the type","author":{"url":"https://github.com/nohwnd","@type":"Person","name":"nohwnd"},"datePublished":"2024-06-06T21:53:59Z","url":"https://github.com/PowerShell/PowerShell/issues/23899"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:200bf6fb-0736-fed2-6ff0-680973fb4f81 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | A804:354CE1:22EBBE:305AD8:6AFDA069 |
| html-safe-nonce | d3ca8cb38ab8c1eb477a8b77de84233444deda1084fddc921d0a41ab249c3037 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBODA0OjM1NENFMToyMkVCQkU6MzA1QUQ4OjZBRkRBMDY5IiwidmlzaXRvcl9pZCI6IjE3Mzk4NDU0MjYxMDM1NTgyNDkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 1d5c8ecb7422a8603920dd8f117c6dabdf8c4e40eb91b5a309d40e79f4a40598 |
| hovercard-subject-tag | issue:2339211352 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/PowerShell/PowerShell/23899/issue_layout |
| twitter:image | https://opengraph.githubassets.com/fe2f206bd5c6c10bbe55a3b10398d317ad61e6944c480a89a12ba454003ce2a5/PowerShell/PowerShell/issues/23899 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/fe2f206bd5c6c10bbe55a3b10398d317ad61e6944c480a89a12ba454003ce2a5/PowerShell/PowerShell/issues/23899 |
| og:image:alt | Summary of the new feature / enhancement I was looking into += as the canonical example of inefficient powershell code for my presentation at psconfeu, and noticed that doing += on 10k items alloca... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | nohwnd |
| hostname | github.com |
| expected-hostname | github.com |
| None | bc86424ef34a73b386237872800e922d149d65ba907f63d6f150a2dcb8433bea |
| turbo-cache-control | no-preview |
| go-import | github.com/PowerShell/PowerShell git https://github.com/PowerShell/PowerShell.git |
| octolytics-dimension-user_id | 11524380 |
| octolytics-dimension-user_login | PowerShell |
| octolytics-dimension-repository_id | 49609581 |
| octolytics-dimension-repository_nwo | PowerShell/PowerShell |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 49609581 |
| octolytics-dimension-repository_network_root_nwo | PowerShell/PowerShell |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 927a6c5ab1c32b38830f1b9d3050c0b9e9aa92e1 |
| ui-target | canary-2 |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width