René's URL Explorer Experiment


Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython · GitHub

Open Graph Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython

X Title: json.dump(x,f) is much slower than f.write(json.dumps(x)) · Issue #129711 · python/cpython

Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the same contents to different files sequentia...

Open Graph Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa...

X Description: Bug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa...

Opengraph URL: https://github.com/python/cpython/issues/129711

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"json.dump(x,f) is much slower than f.write(json.dumps(x))","articleBody":"# Bug report\n\n### Bug description:\n\nExperimentally I measured a huge performance improvement when I switched my code from\n```\njson.dump(x, f, **)\n```\nto\n```\nf.write(json.dumps(x, **))\n```\n\n### Method\nI essentially wrote the same contents to different files sequentially and measured the total amount of time taken. The json contents had 1, 300, and 400 entries per level, and 1, 5, and 6 levels of depth. There's quite a level of variance here but this wasn't what I was trying to measure in the first place. I discovered this by chance, so forgive the lack of precision. I also don't have the source code anymore because I wasn't originally planning to report this discovery.\n\n### Results\n| File Size | Consecutive Files |  dump µs | dumps µs |\n|----------:|-----------------:|-------:|------------:|\n|        74 |                1 |    508 |        581 |\n|        74 |                2 |    520 |        541 |\n|        74 |                4 |   1153 |        1151 |\n|        74 |                8 |   1930 |        1750 |\n|     39184 |                1 |   6363 |        1086 |\n|     39184 |                2 |  11261 |        1821 |\n|     39184 |                4 |  38126 |        3521 |\n|     39184 |                8 |  80411 |       6466 |\n|    468218 |                1 |  82821 |       11921 |\n|    468218 |                2 | 150234 |       38017 |\n|    468218 |                4 | 302357 |       42137 |\n|    468218 |                8 | 573450 |       78545 |\n\n### Conclusion\nA cursory investigation into the cpython code suggests that the slow part is the sequential writing of the `iterencode` yield. The chunks are quite small.\n\n### CPython versions tested on:\n\n3.10\n\n### Operating systems tested on:\n\nmacOS\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-130076\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/wjmelements","@type":"Person","name":"wjmelements"},"datePublished":"2025-02-06T03:22:41Z","url":"https://github.com/python/cpython/issues/129711"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:ad1cb933-3d91-6e52-bdd4-67e10996703c
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idA7BE:26ECEB:4087921:51FAFA3:6B020984
html-safe-noncee9524e39c2c55a7ee447be205d703cb1dd24794627b8b92f603f9c50654e0d06
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBN0JFOjI2RUNFQjo0MDg3OTIxOjUxRkFGQTM6NkIwMjA5ODQiLCJ2aXNpdG9yX2lkIjoiNDMwNjg3ODExNzI1ODIyMzQxIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac6e3774cf1c103e9ead64138cb19a97ac0f50c50196a567365a9284883eb3686b
hovercard-subject-tagissue:2834432837
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/129711/issue_layout
twitter:imagehttps://opengraph.githubassets.com/5b33e306221d31913f806b525ba2114bf76ab4b0567ca7348ea34b6485873258/python/cpython/issues/129711
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/5b33e306221d31913f806b525ba2114bf76ab4b0567ca7348ea34b6485873258/python/cpython/issues/129711
og:image:altBug report Bug description: Experimentally I measured a huge performance improvement when I switched my code from json.dump(x, f, **) to f.write(json.dumps(x, **)) Method I essentially wrote the sa...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamewjmelements
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/cpython/issues/129711#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F129711
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F129711
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/issues/129711
Reloadhttps://github.com/python/cpython/issues/129711
Reloadhttps://github.com/python/cpython/issues/129711
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
https://github.com/python/cpython
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/129711
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/129711
json.dump(x,f) is much slower than f.write(json.dumps(x))https://github.com/python/cpython/issues/129711#top
extension-modulesC modules in the Modules dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22extension-modules%22
performancePerformance or resource usagehttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22performance%22
stdlibStandard Library Python modules in the Lib/ directoryhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22stdlib%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
https://github.com/wjmelements
wjmelementshttps://github.com/wjmelements
on Feb 6, 2025https://github.com/python/cpython/issues/129711#issue-2834432837
Draft: gh-129711: Reduce number of yield statements in json encoder #130076https://github.com/python/cpython/pull/130076
Nexthttps://github.com/python/cpython/issues/129711?timeline_page=1
wjmelementshttps://github.com/wjmelements
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
on Feb 6, 2025https://github.com/python/cpython/issues/129711#event-16203639467
tomasr8https://github.com/tomasr8
JSON issueshttps://github.com/orgs/python/projects/6
on Feb 6, 2025https://github.com/python/cpython/issues/129711#event-16206309863
https://github.com/pochmann3
pochmann3https://github.com/pochmann3
on Feb 6, 2025https://github.com/python/cpython/issues/129711#issuecomment-2639419940
Oldhttps://github.com/python/cpython/issues?q=is%3Aissue%20label%3Aperformance%20json.dump
#56343https://github.com/python/cpython/issues/56343
https://github.com/pochmann3
pochmann3https://github.com/pochmann3
on Feb 6, 2025https://github.com/python/cpython/issues/129711#issuecomment-2639436546
Attempt This Online!https://ato.pxeger.com/run?1=PVC7TgMxEBStP4B6O9vInHxCSCFSPoCKgjJKAcRONpIf8u2JQ1G-JE2a8FF8DX4cbOeZ2dnxnL_jF-2Dv1yuI9n7xc_NrU3BAaEzSIAuhkTzi7HB0BhhBZxzNlMYFByG4JnNOIbulRL63fOLkGzKyFpv4A56rTWzIQECekhvfmfEk9ZyySBP1U0bVlyr6iNsTRGuK82LfbcdXRSTAiu5arDtPhOSEf_0ICZZ2HLxoVlTtm7hRTFVUH-gwI_u3aRVr2VLV8UxJydh-ZGWj11vT-AGgGPZO3HZ6plb-mvrFw
eendebakpthttps://github.com/eendebakpt
performancePerformance or resource usagehttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22performance%22
on Feb 7, 2025https://github.com/python/cpython/issues/129711#event-16223502992
picnixzhttps://github.com/picnixz
stdlibStandard Library Python modules in the Lib/ directoryhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22stdlib%22
extension-modulesC modules in the Modules dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22extension-modules%22
on Feb 7, 2025https://github.com/python/cpython/issues/129711#event-16224160382
https://github.com/pganssle
pgansslehttps://github.com/pganssle
on Feb 7, 2025https://github.com/python/cpython/issues/129711#issuecomment-2643293045
CPython uses the Python version of make_encoderhttps://github.com/python/cpython/blob/a93a5a39fb3c4d15082d2464ef10161aa9f2c46e/Lib/json/encoder.py#L251-L260
_json.chttps://github.com/python/cpython/blob/main/Modules/_json.c
@pochmann3https://github.com/pochmann3
https://github.com/pochmann3
pochmann3https://github.com/pochmann3
on Feb 9, 2025https://github.com/python/cpython/issues/129711#issuecomment-2646537732
https://github.com/eendebakpt
eendebakpthttps://github.com/eendebakpt
on Feb 13, 2025https://github.com/python/cpython/issues/129711#issuecomment-2656121637
@Ghost4https://github.com/Ghost4
https://devguide.python.org/https://devguide.python.org/
eendebakpthttps://github.com/eendebakpt
on Jun 15, 2026https://github.com/python/cpython/issues/129711#event-26778639424
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/baf0b9322f297a3e13c1f690572fd51e6f24d16c
pythongh-129711https://github.com/python/cpython/issues/129711
baf0b93https://github.com/eendebakpt/cpython/commit/baf0b9322f297a3e13c1f690572fd51e6f24d16c
eendebakpthttps://github.com/eendebakpt
on Aug 11, 2026https://github.com/python/cpython/issues/129711#event-29306353417
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/7eb7a4d0a00a377102cb2bc4abcc7e811b8f67a3
pythongh-129711https://github.com/python/cpython/issues/129711
7eb7a4dhttps://github.com/eendebakpt/cpython/commit/7eb7a4d0a00a377102cb2bc4abcc7e811b8f67a3
pythongh-129711: Factor out circular-marker and dict-key helpers in _…https://github.com/eendebakpt/cpython/commit/72e86c445f27a5e8c8dac2fd2c666a91c2f5f860
pythongh-129711https://github.com/python/cpython/issues/129711
72e86c4https://github.com/eendebakpt/cpython/commit/72e86c445f27a5e8c8dac2fd2c666a91c2f5f860
eendebakpthttps://github.com/eendebakpt
on Aug 12, 2026https://github.com/python/cpython/issues/129711#event-29361468631
pythongh-129711: Add tests for the streaming JSON encoderhttps://github.com/eendebakpt/cpython/commit/58a11f5ef4a837560a295333abbb8c8ba991a9fb
pythongh-129711https://github.com/python/cpython/issues/129711
58a11f5https://github.com/eendebakpt/cpython/commit/58a11f5ef4a837560a295333abbb8c8ba991a9fb
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/0a537634a6403dae7a8e95712e6a05a726030cb9
pythongh-129711https://github.com/python/cpython/issues/129711
0a53763https://github.com/eendebakpt/cpython/commit/0a537634a6403dae7a8e95712e6a05a726030cb9
pythongh-129711: Add tests for the streaming JSON encoderhttps://github.com/eendebakpt/cpython/commit/f293cd32cea4016fc54030db8ad9137d93a8b7c6
pythongh-129711https://github.com/python/cpython/issues/129711
f293cd3https://github.com/eendebakpt/cpython/commit/f293cd32cea4016fc54030db8ad9137d93a8b7c6
pythongh-129711: Factor out circular-marker and dict-key helpers in _…https://github.com/eendebakpt/cpython/commit/40ed0133e8096069171943ef9abe1ac199782d72
pythongh-129711https://github.com/python/cpython/issues/129711
40ed013https://github.com/eendebakpt/cpython/commit/40ed0133e8096069171943ef9abe1ac199782d72
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/d0ad68e90dd6df6a0319abb7ee87ee44e1626623
pythongh-129711https://github.com/python/cpython/issues/129711
d0ad68ehttps://github.com/eendebakpt/cpython/commit/d0ad68e90dd6df6a0319abb7ee87ee44e1626623
pythongh-129711: Add tests for the streaming JSON encoderhttps://github.com/eendebakpt/cpython/commit/2541a179819ad7970989b85bdee3dc2154d3c133
pythongh-129711https://github.com/python/cpython/issues/129711
2541a17https://github.com/eendebakpt/cpython/commit/2541a179819ad7970989b85bdee3dc2154d3c133
pythongh-129711: Refactor _json encoder helpers for reusehttps://github.com/eendebakpt/cpython/commit/4aafd9ebae72e92cadfb4334bc99e0b7180a9a97
pythongh-129711https://github.com/python/cpython/issues/129711
4aafd9ehttps://github.com/eendebakpt/cpython/commit/4aafd9ebae72e92cadfb4334bc99e0b7180a9a97
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/c676a7c6e9ccfcb12a100c497eeff7523d106d1d
pythongh-129711https://github.com/python/cpython/issues/129711
c676a7chttps://github.com/eendebakpt/cpython/commit/c676a7c6e9ccfcb12a100c497eeff7523d106d1d
pythongh-129711: Add tests for the streaming JSON encoderhttps://github.com/eendebakpt/cpython/commit/1a3d569642479b0b24aa7f36a07a51939625b785
pythongh-129711https://github.com/python/cpython/issues/129711
1a3d569https://github.com/eendebakpt/cpython/commit/1a3d569642479b0b24aa7f36a07a51939625b785
pythongh-129711: Add a streaming C implementation of the JSON encoderhttps://github.com/eendebakpt/cpython/commit/0e443423eef57cbca3a715a627418eee2d684b6c
pythongh-129711https://github.com/python/cpython/issues/129711
0e44342https://github.com/eendebakpt/cpython/commit/0e443423eef57cbca3a715a627418eee2d684b6c
pythongh-129711: Add tests for the streaming JSON encoderhttps://github.com/eendebakpt/cpython/commit/4ce308e4be170c6b6f857cd913fb94e2cf965700
pythongh-129711https://github.com/python/cpython/issues/129711
4ce308ehttps://github.com/eendebakpt/cpython/commit/4ce308e4be170c6b6f857cd913fb94e2cf965700
eendebakpthttps://github.com/eendebakpt
on Aug 22, 2026https://github.com/python/cpython/issues/129711#event-29854492751
pythongh-129711: Add a no-escape fast path to the _json str escapershttps://github.com/eendebakpt/cpython/commit/190399667b0d1ff6d1c0f96615b6616623ee86c3
pythongh-129711https://github.com/python/cpython/issues/129711
1903996https://github.com/eendebakpt/cpython/commit/190399667b0d1ff6d1c0f96615b6616623ee86c3
pythongh-129711: Add a no-escape fast path to the _json str escapershttps://github.com/eendebakpt/cpython/commit/c1311048f6b7eea785703de228b36db2ff6c57e7
pythongh-129711https://github.com/python/cpython/issues/129711
c131104https://github.com/eendebakpt/cpython/commit/c1311048f6b7eea785703de228b36db2ff6c57e7
pythongh-129711: Add a no-escape fast path to the _json str escapershttps://github.com/eendebakpt/cpython/commit/9a9172b60a90d2af8904691cdd38f49c114c657a
pythongh-129711https://github.com/python/cpython/issues/129711
9a9172bhttps://github.com/eendebakpt/cpython/commit/9a9172b60a90d2af8904691cdd38f49c114c657a
pythongh-129711: Add a no-escape fast path to the _json str escapershttps://github.com/eendebakpt/cpython/commit/e32789e915a4a6e156eac73d6512c01f3587619d
pythongh-129711https://github.com/python/cpython/issues/129711
e32789ehttps://github.com/eendebakpt/cpython/commit/e32789e915a4a6e156eac73d6512c01f3587619d
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/cpython/issues/129711
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/cpython/issues/129711
extension-modulesC modules in the Modules dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22extension-modules%22
performancePerformance or resource usagehttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22performance%22
stdlibStandard Library Python modules in the Lib/ directoryhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22stdlib%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
JSON issueshttps://github.com/orgs/python/projects/6
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.