Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython · GitHub
Open Graph Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython
X Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython
Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the same contents to different files sequentia...
Open Graph Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa...
X Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa...
Opengraph URL: https://github.com/python/cpython/issues/129711
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"json.dump(x,f) is much slower than f.write(json.dumps(x))","articleBody":"# Bug report\n\n### Bug description:\n\nExperimentally I measured a huge performance improvement when I switched my code from\n```\njson.dump(x, f, **)\n```\nto\n```\nf.write(json.dumps(x, **))\n```\n\n### Method\nI essentially wrote the same contents to different files sequentially and measured the total amount of time taken. The json contents had 1, 300, and 400 entries per level, and 1, 5, and 6 levels of depth. There's quite a level of variance here but this wasn't what I was trying to measure in the first place. I discovered this by chance, so forgive the lack of precision. I also don't have the source code anymore because I wasn't originally planning to report this discovery.\n\n### Results\n| File Size | Consecutive Files | dump µs | dumps µs |\n|----------:|-----------------:|-------:|------------:|\n| 74 | 1 | 508 | 581 |\n| 74 | 2 | 520 | 541 |\n| 74 | 4 | 1153 | 1151 |\n| 74 | 8 | 1930 | 1750 |\n| 39184 | 1 | 6363 | 1086 |\n| 39184 | 2 | 11261 | 1821 |\n| 39184 | 4 | 38126 | 3521 |\n| 39184 | 8 | 80411 | 6466 |\n| 468218 | 1 | 82821 | 11921 |\n| 468218 | 2 | 150234 | 38017 |\n| 468218 | 4 | 302357 | 42137 |\n| 468218 | 8 | 573450 | 78545 |\n\n### Conclusion\nA cursory investigation into the cpython code suggests that the slow part is the sequential writing of the `iterencode` yield. The chunks are quite small.\n\n### CPython versions tested on:\n\n3.10\n\n### Operating systems tested on:\n\nmacOS\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-130076\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/wjmelements","@type":"Person","name":"wjmelements"},"datePublished":"2025-02-06T03:22:41Z","url":"https://github.com/python/cpython/issues/129711"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:ad1cb933-3d91-6e52-bdd4-67e10996703c |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | A7BE:26ECEB:4087921:51FAFA3:6B020984 |
| html-safe-nonce | e9524e39c2c55a7ee447be205d703cb1dd24794627b8b92f603f9c50654e0d06 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBN0JFOjI2RUNFQjo0MDg3OTIxOjUxRkFGQTM6NkIwMjA5ODQiLCJ2aXNpdG9yX2lkIjoiNDMwNjg3ODExNzI1ODIyMzQxIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 6e3774cf1c103e9ead64138cb19a97ac0f50c50196a567365a9284883eb3686b |
| hovercard-subject-tag | issue:2834432837 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/129711/issue_layout |
| twitter:image | https://opengraph.githubassets.com/5b33e306221d31913f806b525ba2114bf76ab4b0567ca7348ea34b6485873258/python/cpython/issues/129711 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/5b33e306221d31913f806b525ba2114bf76ab4b0567ca7348ea34b6485873258/python/cpython/issues/129711 |
| og:image:alt | Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | wjmelements |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width