Title: gh-150942: Optimize stringlib split/splitlines with _PyList_AppendTakeRef by danielKim614 · Pull Request #155922 · python/cpython · GitHub
Open Graph Title: gh-150942: Optimize stringlib split/splitlines with _PyList_AppendTakeRef by danielKim614 · Pull Request #155922 · python/cpython
X Title: gh-150942: Optimize stringlib split/splitlines with _PyList_AppendTakeRef by danielKim614 · Pull Request #155922 · python/cpython
Description: Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF in the stringlib SPLIT_ADD (past the 12-slot prealloc) and SPLIT_APPEND (used by splitlines() for every line) macros, removing a reference-count round-trip per appended item (and a per-append lock on the free-threaded build). This covers split/rsplit/splitlines for str, bytes and bytearray. The result list is freshly allocated and stays local to the split function until it is returned. Microbenchmarks Plain release build, macOS arm64 Interleaved A/B runs (8 rounds alternating baseline/patched binaries), medians Benchmark workload main this PR speedup str_split_10k_pieces ("abc " * 10_000).split(" ") 165 us 152 us 1.09× str_split_whitespace_10k ("word " * 10_000).split() 184 us 168 us 1.10× str_splitlines_10k ("line\n" * 10_000).splitlines() 182 us 165 us 1.10× bytes_split_10k_pieces (b"abc " * 10_000).split(b" ") 148 us 139 us 1.06× geomean 1.09× test_str and test_bytes pass, including -R 3:3 refleak runs. Benchmark script """Microbenchmark str/bytes split / rsplit / splitlines (result-list building).""" import pyperf # Many small pieces (>12): exercises the append path after the 12-slot prealloc. import pyperf # Many small pieces (>12): exercises the append path after the 12-slot prealloc. STR_MANY = "abc " * 10_000 # .split(" ") -> 10001 pieces STR_WS = "word " * 10_000 # .split() whitespace variant BYTES_MANY = b"abc " * 10_000 # bytes.split LINES = "line\n" * 10_000 # .splitlines(): SPLIT_APPEND for every line # Controls STR_FEW = "a b c d e f g h i j" # 10 pieces: stays in the SET_ITEM prealloc path STR_BIG = ("x" * 4096 + " ") * 250 # large pieces: copy-dominated def bench_method(loops, obj, name, *args): method = getattr(obj, name) range_it = range(loops) t0 = pyperf.perf_counter() for _ in range_it: method(*args) return pyperf.perf_counter() - t0 runner = pyperf.Runner() runner.bench_time_func("str_split_10k_pieces", bench_method, STR_MANY, "split", " ") runner.bench_time_func("str_split_whitespace_10k", bench_method, STR_WS, "split") runner.bench_time_func("str_splitlines_10k", bench_method, LINES, "splitlines") runner.bench_time_func("bytes_split_10k_pieces", bench_method, BYTES_MANY, "split", b" ") runner.bench_time_func("str_split_10_pieces_ctl", bench_method, STR_FEW, "split", " ") runner.bench_time_func("str_split_large_pieces_ctl", bench_method, STR_BIG, "split", " ") Issue: gh-150942
Open Graph Description: Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF in the stringlib SPLIT_ADD (past the 12-slot prealloc) and SPLIT_APPEND (used by spl...
X Description: Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF in the stringlib SPLIT_ADD (past the 12-slot prealloc) and SPLIT_APPEND (used by spl...
Opengraph URL: https://github.com/python/cpython/pull/155922
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c1e7d700-300d-7acf-0085-677cef195a29 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 99C6:26ECEB:4246A49:542AD81:6B021268 |
| html-safe-nonce | de5221ff9a0ec90a1ea02f5ac43edf49e0525c971b43c08fd8d1ea4b18005fe8 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5OUM2OjI2RUNFQjo0MjQ2QTQ5OjU0MkFEODE6NkIwMjEyNjgiLCJ2aXNpdG9yX2lkIjoiNzY4ODU5NTAzNzY4NDM3MjA3MiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | df05ba73b198903a02436f754e4befa5bf2e0d02d5a2a672ad77d20466738c83 |
| hovercard-subject-tag | pull_request:4292950226 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/155922/files |
| twitter:image | https://avatars.githubusercontent.com/u/78344167?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/78344167?s=400&v=4 |
| og:image:alt | Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF in the stringlib SPLIT_ADD (past the 12-slot prealloc) and SPLIT_APPEND (used by spl... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width