Title: email: generating messages got slower after gh-87720, since folding walks the parse tree repeatedly · Issue #160675 · python/cpython · GitHub
Open Graph Title: email: generating messages got slower after gh-87720, since folding walks the parse tree repeatedly · Issue #160675 · python/cpython
X Title: email: generating messages got slower after gh-87720, since folding walks the parse tree repeatedly · Issue #160675 · python/cpython
Description: Bug report Bug description: The performance regression suite I run against CPython builds picked up a slowdown in email header folding after #134271 (gh-87720). The fix itself is important, since it stops encoded words with specials from...
Open Graph Description: Bug report Bug description: The performance regression suite I run against CPython builds picked up a slowdown in email header folding after #134271 (gh-87720). The fix itself is important, since i...
X Description: Bug report Bug description: The performance regression suite I run against CPython builds picked up a slowdown in email header folding after #134271 (gh-87720). The fix itself is important, since i...
Opengraph URL: https://github.com/python/cpython/issues/160675
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"email: generating messages got slower after gh-87720, since folding walks the parse tree repeatedly","articleBody":"# Bug report\n\n### Bug description:\n\nThe performance regression suite I run against CPython builds picked up a slowdown in `email` header folding after #134271 (gh-87720). The fix itself is important, since it stops encoded words with specials from losing their quoting when refolded, so the goal here is only to make the new check cheaper, with exactly the same output.\n\nThe new check calls `contains_token_type('vtext')` for most parts while folding, and that walks the whole subtree of the part every time. Parts that don't fit are then split and their subparts are walked again, so headers without any `vtext`, which is the common case, have their tree walked several times. For a 50-recipient `To` header that's about 3,000 calls over 1,500 tokens.\n\nIt shows up whenever new messages are generated, since every header gets folded:\n\n```python\nimport email.policy\nfrom email.message import EmailMessage\n\nPOLICY = email.policy.SMTP.clone(max_line_length=78)\n\nmessage = EmailMessage(policy=POLICY)\nmessage[\"From\"] = \"Orders \u003corders@example.com>\"\nmessage[\"To\"] = \", \".join(f\"Customer {i:02d} \u003ccustomer-{i:02d}@example.net>\" for i in range(50))\nmessage[\"Subject\"] = \"Your order has been processed\"\nmessage.set_content(\"Your order was processed successfully.\\n\")\nmessage.as_bytes(policy=POLICY)\n```\n\n`as_bytes()` time per message, before and after the change:\n\n| recipients | before | after |\n|---:|---:|---:|\n| 1 | 0.071 ms | 0.092 ms (+29%) |\n| 10 | 0.223 ms | 0.345 ms (+55%) |\n| 50 | 0.855 ms | 1.400 ms (+64%) |\n\nTo check this isn't just a microbenchmark, I also ran a more realistic workload: 2,000 generated messages with a mix of 1 to 50 recipients, `Received`, `DKIM-Signature` and `List-Unsubscribe` headers, about 40% of them replies with `References`, and plain-text or HTML bodies. Generating and serializing them got about **17% slower** overall (+0.22 ms per message). Re-serializing parsed messages is barely affected (about +2%), since unchanged headers keep their original lines.\n\nThis only affects messages using the new policies (`EmailMessage`, `policy.default`, `policy.SMTP`, also through `smtplib.send_message()`). Messages using `compat32` go through the old folding code and aren't affected.\n\nThis isn't specific to any platform. The numbers above are local (arm64), and I validated the same on AWS Lambda, where a 50-recipient serialization was about 64% slower on 3.12 (arm64, PGO+LTO). The change is now in 3.12, 3.13, 3.14 and main.\n\nI think the repeated walks can be avoided without changing the result. A part can only contain `vtext` if the whole tree does, so the tree can be checked once per header and the per-part check skipped when there is none. When there is, the per-part check can walk the tree iteratively instead of with a recursive method call per token. With that, the realistic workload goes from +17% to about +6%, the 50-recipient case from +64% to about +18%, and headers that do contain encoded words stay as fast as they are now.\n\nThe folded output stays byte-for-byte identical to the current code (I compared the benchmark messages, the realistic workload and a few hundred foldings with encoded words, specials, quoted strings and comments), and `test_email` passes on main, 3.12 and 3.13. Happy to send a PR. cc @bitdancer\n\n### CPython versions tested on:\n\n3.12, 3.13, CPython main branch\n\n### Operating systems tested on:\n\nLinux\n","author":{"url":"https://github.com/leandrodamascena","@type":"Person","name":"leandrodamascena"},"datePublished":"2026-10-25T18:09:57Z","url":"https://github.com/python/cpython/issues/160675"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:93184531-2bee-b580-45dc-e2dcbc004488 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | E15A:136CDA:1DA7485:272BCD7:6B022F4B |
| html-safe-nonce | 07fe0a0eedc283f1db570774948be013c04984f1440de93b674fd604ed8e29fd |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFMTVBOjEzNkNEQToxREE3NDg1OjI3MkJDRDc6NkIwMjJGNEIiLCJ2aXNpdG9yX2lkIjoiMjM2MzU3OTg3ODQxMjAwNTE5NSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 63e64de7f886003519814a514b14fcf257ee2545d7080fc0bfc6fdbe832f260b |
| hovercard-subject-tag | issue:6027641020 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/160675/issue_layout |
| twitter:image | https://opengraph.githubassets.com/16b7fe98cab495fccab0ed2ea39bb562b0443f2bd8fade3523b789af023a8b3c/python/cpython/issues/160675 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/16b7fe98cab495fccab0ed2ea39bb562b0443f2bd8fade3523b789af023a8b3c/python/cpython/issues/160675 |
| og:image:alt | Bug report Bug description: The performance regression suite I run against CPython builds picked up a slowdown in email header folding after #134271 (gh-87720). The fix itself is important, since i... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | leandrodamascena |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width