Title: Pin the hash seed in the microbenchmark harness by codspeed[bot] · Pull Request #9126 · RustPython/RustPython · GitHub
Open Graph Title: Pin the hash seed in the microbenchmark harness by codspeed[bot] · Pull Request #9126 · RustPython/RustPython
X Title: Pin the hash seed in the microbenchmark harness by codspeed[bot] · Pull Request #9126 · RustPython/RustPython
Description: Requested in #9010. Why benches/microbenchmarks.rs builds the interpreter from Settings::default(), which leaves hash_seed = None. Each benchmark process then picks a random str hash secret. Depending on the seed, the names in the module globals dict collide more or less, so dict, loop, string and compare microbenchmarks move by 10–17% between runs even when the code is the same. Examples: loop_append.py: 2.3 ms ↔ 2.7 ms (#9010, a libc-only bump) cmp.py: 62.5 ms ↔ 69.6 ms (#9010, #9113) loop_string.py / strings.py: about 843 ↔ 995 µs and 40.7 ↔ 47.6 ms (#8957, #8978, #8979, #9103) Change Set settings.hash_seed = Some(0) in bench_rustpython_code. The hash secret is set once per process by the first interpreter, and every RustPython interpreter in this harness uses the same settings, so all microbenchmarks now hash with the same secret on every run. Benchmark code and CPython comparisons are not affected. The first CodSpeed run on this PR can move the affected benchmarks once, since the fixed seed may give different collisions than the random seed in the base run. After that, results should only change when the code does. benches/execution.rs uses the same Settings::default() pattern and could be pinned the same way if needed. Comment @codspeedbot pin the hash seed in benches/execution.rs too and I'll do it.
Open Graph Description: Requested in #9010. Why benches/microbenchmarks.rs builds the interpreter from Settings::default(), which leaves hash_seed = None. Each benchmark process then picks a random str hash secret. Depend...
X Description: Requested in #9010. Why benches/microbenchmarks.rs builds the interpreter from Settings::default(), which leaves hash_seed = None. Each benchmark process then picks a random str hash secret. Depend...
Opengraph URL: https://github.com/RustPython/RustPython/pull/9126
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:46b4a1f0-a67c-2a24-b83b-3d6b161d5b88 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D8D4:B53AF:33D18FE:44FD5FD:6B01FA1A |
| html-safe-nonce | 6736fa1bdcf45fa1230c66a7b5bd3819a1bfe58c5ba23f03e473ee113723f4a6 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEOEQ0OkI1M0FGOjMzRDE4RkU6NDRGRDVGRDo2QjAxRkExQSIsInZpc2l0b3JfaWQiOiI5MTIyNjA3MDI0OTM5NTk2MzE0IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 933d426e80952caaec62a8143428f7777b74956e546bdfe9d051193c4569ef42 |
| hovercard-subject-tag | pull_request:4862230721 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9126/files |
| twitter:image | https://avatars.githubusercontent.com/in/257293?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/in/257293?s=400&v=4 |
| og:image:alt | Requested in #9010. Why benches/microbenchmarks.rs builds the interpreter from Settings::default(), which leaves hash_seed = None. Each benchmark process then picks a random str hash secret. Depend... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width