Title: gh-150942: Speed up re.split result building by snurf198 · Pull Request #155940 · python/cpython · GitHub
Open Graph Title: gh-150942: Speed up re.split result building by snurf198 · Pull Request #155940 · python/cpython
X Title: gh-150942: Speed up re.split result building by snurf198 · Pull Request #155940 · python/cpython
Description: @hugovk @corona10 Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF, removing a reference-count round-trip per appended item (and a per-append lock on the free-threaded build). This is the same change gh-150943 made to re.findall and re.sub/subn, applied to the one result-building loop it left behind. The error: label only drops list, and _PyList_AppendTakeRef steals the reference on failure too, so the error path is unchanged. Microbenchmarks Default build Benchmark main this PR speedup re_split 4.54 ms 4.31 ms 1.05× re_split_groups 5.52 ms 5.23 ms 1.06× re_split_ws 4.18 ms 4.03 ms 1.04× re_split_bytes 4.08 ms 3.95 ms 1.03× str_split_control 1.37 ms 1.37 ms 1.00× geomean 1.03× Free-threading build Benchmark main this PR speedup re_split 6.48 ms 5.84 ms 1.11× re_split_groups 8.05 ms 6.81 ms 1.18× re_split_ws 5.74 ms 5.26 ms 1.09× re_split_bytes 6.13 ms 5.39 ms 1.14× str_split_control not significant geomean 1.08× Benchmark script """Microbenchmark re.split (result-list building).""" import re import pyperf WORDS = "the quick brown fox jumps over the lazy dog " * 2000 CSVISH = "field1,field2,field3,field4,field5,field6,field7,field8\n" * 3000 CSVISH_BYTES = CSVISH.encode() split_re = re.compile(r"[,\n]") group_re = re.compile(r"([,\n])") ws_re = re.compile(r"\s+") bytes_re = re.compile(rb"[,\n]") nomatch_re = re.compile(r"[;|]") def b_split(): return split_re.split(CSVISH) def b_split_groups(): return group_re.split(CSVISH) def b_split_ws(): return ws_re.split(WORDS) def b_split_bytes(): return bytes_re.split(CSVISH_BYTES) def b_split_nomatch(): return nomatch_re.split(CSVISH) def b_str_split(): return CSVISH.split(",") if __name__ == "__main__": runner = pyperf.Runner() runner.bench_func("re_split", b_split) runner.bench_func("re_split_groups", b_split_groups) runner.bench_func("re_split_ws", b_split_ws) runner.bench_func("re_split_bytes", b_split_bytes) runner.bench_func("re_split_nomatch", b_split_nomatch) runner.bench_func("str_split_control", b_str_split) Issue: gh-150942
Open Graph Description: @hugovk @corona10 Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF, removing a reference-count round-trip per appended item (and a pe...
X Description: @hugovk @corona10 Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF, removing a reference-count round-trip per appended item (and a pe...
Opengraph URL: https://github.com/python/cpython/pull/155940
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:b93bc70e-8528-c972-9d84-9d696e85cd59 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | E9F0:3A4875:3863BDB:4B0DBD8:6B0216D5 |
| html-safe-nonce | c4e66801be8e902d1e55f9a67209babc7929978bf91da16d80e058e6b37e0f32 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFOUYwOjNBNDg3NTozODYzQkRCOjRCMERCRDg6NkIwMjE2RDUiLCJ2aXNpdG9yX2lkIjoiNjYyMDAyMzY5Mjg4Mzc5MzYyMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 9b678b050ea37ca2005ca0a74f40b7775bad36068c0905688bdd1806422df0d5 |
| hovercard-subject-tag | pull_request:4293927233 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/155940/files |
| twitter:image | https://avatars.githubusercontent.com/u/55132820?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/55132820?s=400&v=4 |
| og:image:alt | @hugovk @corona10 Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF, removing a reference-count round-trip per appended item (and a pe... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width