Title: ConvertFrom-JSON high memory consumption · Issue #7698 · PowerShell/PowerShell · GitHub
Open Graph Title: ConvertFrom-JSON high memory consumption · Issue #7698 · PowerShell/PowerShell
X Title: ConvertFrom-JSON high memory consumption · Issue #7698 · PowerShell/PowerShell
Description: Steps to reproduce $JSON = @" { "configGlossary:installationAt": "Philadelphia, PA", "configGlossary:adminEmail": "ksm@pobox.com", "configGlossary:poweredBy": "Cofax", "configGlossary:poweredByIcon": "\/images\/cofax.gif", "configGlossar...
Open Graph Description: Steps to reproduce $JSON = @" { "configGlossary:installationAt": "Philadelphia, PA", "configGlossary:adminEmail": "ksm@pobox.com", "configGlossary:poweredBy": "Cofax", "configGlossary:poweredByIcon...
X Description: Steps to reproduce $JSON = @" { "configGlossary:installationAt": "Philadelphia, PA", "configGlossary:adminEmail": "ksm@pobox.com", "configGlossary:...
Opengraph URL: https://github.com/PowerShell/PowerShell/issues/7698
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"ConvertFrom-JSON high memory consumption","articleBody":"Steps to reproduce\r\n------------------\r\n\r\n```powershell\r\n$JSON = @\"\r\n{\r\n \"configGlossary:installationAt\": \"Philadelphia, PA\",\r\n \"configGlossary:adminEmail\": \"ksm@pobox.com\",\r\n \"configGlossary:poweredBy\": \"Cofax\",\r\n \"configGlossary:poweredByIcon\": \"\\/images\\/cofax.gif\",\r\n \"configGlossary:staticPath\": \"\\/content\\/static\",\r\n \"templateProcessorClass\": \"org.cofax.WysiwygTemplate\",\r\n \"templateLoaderClass\": \"org.cofax.FilesTemplateLoader\",\r\n \"templatePath\": \"templates\",\r\n \"templateOverridePath\": \"\",\r\n \"defaultListTemplate\": \"listTemplate.htm\",\r\n \"defaultFileTemplate\": \"articleTemplate.htm\",\r\n \"useJSP\": false,\r\n \"jspListTemplate\": \"listTemplate.jsp\",\r\n \"jspFileTemplate\": \"articleTemplate.jsp\",\r\n \"cachePackageTagsTrack\": 200,\r\n \"cachePackageTagsStore\": 200,\r\n \"cachePackageTagsRefresh\": 60,\r\n \"cacheTemplatesTrack\": 100,\r\n \"cacheTemplatesStore\": 50,\r\n \"cacheTemplatesRefresh\": 15,\r\n \"cachePagesTrack\": 200,\r\n \"cachePagesStore\": 100,\r\n \"cachePagesRefresh\": 10,\r\n \"cachePagesDirtyRead\": 10,\r\n \"searchEngineListTemplate\": \"forSearchEnginesList.htm\",\r\n \"searchEngineFileTemplate\": \"forSearchEngines.htm\",\r\n \"searchEngineRobotsDb\": \"WEB-INF\\/robots.db\",\r\n \"useDataStore\": true,\r\n \"dataStoreClass\": \"org.cofax.SqlDataStore\",\r\n \"redirectionClass\": \"org.cofax.SqlRedirection\",\r\n \"dataStoreName\": \"cofax\",\r\n \"dataStoreDriver\": \"com.microsoft.jdbc.sqlserver.SQLServerDriver\",\r\n \"dataStoreUrl\": \"jdbc:microsoft:sqlserver:\\/\\/LOCALHOST:1433;DatabaseName=goon\",\r\n \"dataStoreUser\": \"sa\",\r\n \"dataStorePassword\": \"dataStoreTestQuery\",\r\n \"dataStoreTestQuery\": \"SET NOCOUNT ON;select test='test';\",\r\n \"dataStoreLogFile\": \"\\/usr\\/local\\/tomcat\\/logs\\/datastore.log\",\r\n \"dataStoreInitConns\": 10,\r\n \"dataStoreMaxConns\": 100,\r\n \"dataStoreConnUsageLimit\": 100,\r\n \"dataStoreLogLevel\": \"debug\",\r\n \"maxUrlLength\": 500\r\n}\r\n\"@\r\n\r\n$Count = 39999\r\n$jsonarr = \"[\" + (($json + \",\") * $Count) + $json + \"]\"\r\n[system.gc]::Collect()\r\nwrite-host \"\"\r\nwrite-host \"\"\r\nwrite-host \"JSON -- Objects: $($Count + 1)\"\r\nwrite-host \"JSON -- Length: $($jsonarr.length / 1024 / 1024 ) Megabytes\"\r\n\r\n$BeforeMemory = $((get-process -pid $PID).WorkingSet)\r\n[void] (convertfrom-json $jsonarr)\r\n$AfterMemory = $((get-process -pid $PID).WorkingSet)\r\n\r\n$MemoryDiff = $AfterMemory - $BeforeMemory\r\n\r\nwrite-host \"Memory Usage -- Before: $($BeforeMemory / 1024 / 1024) Megabytes\"\r\nwrite-host \"Memory Usage -- After: $($AfterMemory / 1024 / 1024) Megabytes\"\r\nwrite-host \"Memory Usage -- Difference: $($MemoryDiff / 1024 / 1024) Megabytes\"\r\nwrite-host \"Memory Usage -- Multiple: $($MemoryDiff / $jsonarr.length)\"\r\n\r\n```\r\noutput:\r\n```\r\nJSON -- Objects: 40000\r\nJSON -- Length: 67.4819955825806 Megabytes\r\nMemory Usage -- Before: 210.8359375 Megabytes\r\nMemory Usage -- After: 1240.90625 Megabytes\r\nMemory Usage -- Difference: 1030.0703125 Megabytes\r\nMemory Usage -- Multiple: 15.2643724241892\r\n```\r\nActual behavior\r\n-----------------\r\nPowershell Core ends up with 1 GB of memory used and peaks to 1.6 GB of memory utilization during convertfrom-json\r\n\r\nExpected behavior\r\n-----------------\r\n\r\nPowershell Core shouldn't need 1.6 gigabytes of memory to deserialize a 65MB JSON file.\r\n\r\nEnvironment data\r\n----------------\r\n\r\n\u003c!-- provide the output of $PSVersionTable --\u003e\r\n\r\n```powershell\r\nName Value\r\n---- -----\r\nPSVersion 6.1.0-preview.3\r\nPSEdition Core\r\nGitCommitId v6.1.0-preview.3\r\nOS Microsoft Windows 10.0.17134\r\nPlatform Win32NT\r\nPSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}\r\nPSRemotingProtocolVersion 2.3\r\nSerializationVersion 1.1.0.1\r\nWSManStackVersion 3.0\r\n```\r\n","author":{"url":"https://github.com/strawgate","@type":"Person","name":"strawgate"},"datePublished":"2018-09-03T17:39:17.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":22},"url":"https://github.com/7698/PowerShell/issues/7698"}
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:2739fc9d-f464-4a02-6f64-b2f07e3a6369 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | B61C:17402E:41006F4:5C765F1:6A55077E |
| html-safe-nonce | bf5cee0f890d79e979c8a8eda89f90ef388683f8a7be768cfcc47dbfa880f0db |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNjFDOjE3NDAyRTo0MTAwNkY0OjVDNzY1RjE6NkE1NTA3N0UiLCJ2aXNpdG9yX2lkIjoiNzE4MDgxNTAxMjkxMDY2MzU1MCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | baa887620a8b87372cdee544a92aa9c4e2977232c016f2684edd5583ed440a84 |
| hovercard-subject-tag | issue:356571845 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/PowerShell/PowerShell/7698/issue_layout |
| twitter:image | https://opengraph.githubassets.com/82a955face7188497e30f62165b0ddacdde41b6e541237af7eb0b00ab919dc8c/PowerShell/PowerShell/issues/7698 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/82a955face7188497e30f62165b0ddacdde41b6e541237af7eb0b00ab919dc8c/PowerShell/PowerShell/issues/7698 |
| og:image:alt | Steps to reproduce $JSON = @" { "configGlossary:installationAt": "Philadelphia, PA", "configGlossary:adminEmail": "ksm@pobox.com", "configGlossary:poweredBy": "Cofax", "configGlossary:poweredByIcon... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | strawgate |
| hostname | github.com |
| expected-hostname | github.com |
| None | eae2aab99907699a22d2c8449aa683ca74454fca87211b98f7c7b722086b2839 |
| turbo-cache-control | no-preview |
| go-import | github.com/PowerShell/PowerShell git https://github.com/PowerShell/PowerShell.git |
| octolytics-dimension-user_id | 11524380 |
| octolytics-dimension-user_login | PowerShell |
| octolytics-dimension-repository_id | 49609581 |
| octolytics-dimension-repository_nwo | PowerShell/PowerShell |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 49609581 |
| octolytics-dimension-repository_network_root_nwo | PowerShell/PowerShell |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 6a35ab858be185cd86d12b93a43bccf34e8d8536 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width