Title: lib: use utf8 fast path for streaming TextDecoder by ChALkeR · Pull Request #61549 · nodejs/node · GitHub
Open Graph Title: lib: use utf8 fast path for streaming TextDecoder by ChALkeR · Pull Request #61549 · nodejs/node
X Title: lib: use utf8 fast path for streaming TextDecoder by ChALkeR · Pull Request #61549 · nodejs/node
Description: Tracking: #61041 A continuation of #61409 This is based on the logic in https://github.com/ExodusOSS/bytes Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants. Previously, ICU and string_decoder were used on with-intl and without-intl variants correspondingly for streaming UTF-8 implementation Instead, just do minor quick slices on JS side and use the single stateless native decoding API to be fast on large chunks This: Improves streaming UTF-8 TextDecoder performance Brings streaming fatal UTF-8 TextDecoder support to without-intl builds Unifies the logic, using a single codepath everywhere for UTF-8 Benchmarks Previously: #61131 Non-chunked, 25.5.0 Test Size Throughput Mean Time Latin lipsum (ASCII) 84.902 KiB 36.69 GiB/s 0.002 ms Arabic lipsum 79.771 KiB 2.03 GiB/s 0.037 ms Chinese lipsum 68.203 KiB 4.10 GiB/s 0.016 ms Arabic + 2 * ASCII 249.577 KiB 2.72 GiB/s 0.092 ms Non-ASCII char + ASCII 84.905 KiB 3.84 GiB/s 0.024 ms Non-chunked, PR Test Size Throughput Mean Time Latin lipsum (ASCII) 84.902 KiB 36.19 GiB/s 0.002 ms Arabic lipsum 79.771 KiB 2.02 GiB/s 0.038 ms Chinese lipsum 68.203 KiB 4.00 GiB/s 0.016 ms Arabic + 2 * ASCII 249.577 KiB 3.32 GiB/s 0.075 ms Non-ASCII char + ASCII 84.905 KiB 4.71 GiB/s 0.022 ms (should not be affected) Chunked (1000 byte chunks), 25.5.0 Test Size Throughput Mean Time Latin lipsum (ASCII) 84.902 KiB 1.31 GiB/s 0.062 ms Arabic lipsum 79.771 KiB 0.90 GiB/s 0.084 ms Chinese lipsum 68.203 KiB 0.99 GiB/s 0.066 ms Arabic + 2 * ASCII 249.577 KiB 1.14 GiB/s 0.209 ms Non-ASCII char + ASCII 84.905 KiB 1.31 GiB/s 0.062 ms Chunked (1000 byte chunks), PR Test Size Throughput Mean Time Latin lipsum (ASCII) 84.902 KiB 7.45 GiB/s 0.011 ms Arabic lipsum 79.771 KiB 1.24 GiB/s 0.062 ms Chinese lipsum 68.203 KiB 1.54 GiB/s 0.042 ms Arabic + 2 * ASCII 249.577 KiB 2.67 GiB/s 0.091 ms Non-ASCII char + ASCII 84.905 KiB 7.26 GiB/s 0.011 ms The benchmark creates about ~70-256 chunks on each test The improvement is significant
Open Graph Description: Tracking: #61041 A continuation of #61409 This is based on the logic in https://github.com/ExodusOSS/bytes Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants. Previ...
X Description: Tracking: #61041 A continuation of #61409 This is based on the logic in https://github.com/ExodusOSS/bytes Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants. Previ...
Opengraph URL: https://github.com/nodejs/node/pull/61549
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:0cf63d1e-7ce9-77c4-473f-23f7da734041 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D564:36296E:8753EA:C29287:6A4CE6E2 |
| html-safe-nonce | f911e4f30106a6e5939f474e946adf459c0f80e0bfb66135009e3fbb85cc2f8d |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENTY0OjM2Mjk2RTo4NzUzRUE6QzI5Mjg3OjZBNENFNkUyIiwidmlzaXRvcl9pZCI6Ijg2MjI5OTI1NzA1MTM2ODAwOTgiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 9d79546c695511f52c5ab8edd2f40d1d2793d75ad2820c7a8460bafe00a8a8db |
| hovercard-subject-tag | pull_request:3215682222 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/nodejs/node/pull/61549/files |
| twitter:image | https://avatars.githubusercontent.com/u/291301?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/291301?s=400&v=4 |
| og:image:alt | Tracking: #61041 A continuation of #61409 This is based on the logic in https://github.com/ExodusOSS/bytes Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants. Previ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 299b43bca6e02ad35197ffeba30d2466846d5fb02ab96fbced5b5e6cec589fb8 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/nodejs/node git https://github.com/nodejs/node.git |
| octolytics-dimension-user_id | 9950313 |
| octolytics-dimension-user_login | nodejs |
| octolytics-dimension-repository_id | 27193779 |
| octolytics-dimension-repository_nwo | nodejs/node |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 27193779 |
| octolytics-dimension-repository_network_root_nwo | nodejs/node |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | c5a57f04eeb310f57c73fd6d751d957e2ca27ed2 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width