René's URL Explorer Experiment


Title: Refactor benchmarking suite to use Criterion and add microbenchmarks by orf · Pull Request #2367 · RustPython/RustPython · GitHub

Open Graph Title: Refactor benchmarking suite to use Criterion and add microbenchmarks by orf · Pull Request #2367 · RustPython/RustPython

X Title: Refactor benchmarking suite to use Criterion and add microbenchmarks by orf · Pull Request #2367 · RustPython/RustPython

Description: This PR refactors the benchmark suite to use Criterion (https://docs.rs/criterion/0.3.3/criterion/). I've completely re-written the benchmarks, which I don't think where running correctly before. Now every file inside benchmarks/ is both parsed and executed by rustpython + cpython and the results compared. Adding a new benchmark is as simple as adding a file to that directory, and any exceptions while executing the code are correctly displayed to the console rather than just panicking. I've also added an extra benchmark that runs pystone three times with a different step count for both cpython and rustpython and compares the results. I added the ability to run microbenchmarks quickly and easily. For example to benchmark context manager exceptions you could add: from contextlib import contextmanager @contextmanager def try_catch(*args, **kwargs): try: yield except RuntimeError: pass # --- with try_catch(): raise RuntimeError() Only the part after the # --- will be benchmarked. Here are the results: We could then also use this github action to run them as part of the CI steps in merge requests.

Open Graph Description: This PR refactors the benchmark suite to use Criterion (https://docs.rs/criterion/0.3.3/criterion/). I've completely re-written the benchmarks, which I don't think where running correctly b...

X Description: This PR refactors the benchmark suite to use Criterion (https://docs.rs/criterion/0.3.3/criterion/). I've completely re-written the benchmarks, which I don't think where running cor...

Opengraph URL: https://github.com/RustPython/RustPython/pull/2367

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:d57ccf4c-b6df-7341-80c7-5a8b5fb38035
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-id8BAC:311DC9:D0EDBC:11A15EE:6A59440B
html-safe-nonceae46a426fbb6e0dd7e9566974f21d2d851e4400bd66d01f07f65c105a6a54aed
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4QkFDOjMxMURDOTpEMEVEQkM6MTFBMTVFRTo2QTU5NDQwQiIsInZpc2l0b3JfaWQiOiIxNDk4MTgxMDEyMzQ5ODYyOTIzIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmacb3b32f1f844fc61401f8dcdc0763f72f9099ce322d31bfdfdfa752a36760e562
hovercard-subject-tagpull_request:536366069
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/2367/files
twitter:imagehttps://avatars.githubusercontent.com/u/1027207?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/1027207?s=400&v=4
og:image:altThis PR refactors the benchmark suite to use Criterion (https://docs.rs/criterion/0.3.3/criterion/). I've completely re-written the benchmarks, which I don't think where running correctly b...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None9f801e05b35c85b2d72eae6534d2b1d4c02ba8732fd046450bab31e2f52450ce
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git
octolytics-dimension-user_id39710557
octolytics-dimension-user_loginRustPython
octolytics-dimension-repository_id135201145
octolytics-dimension-repository_nwoRustPython/RustPython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id135201145
octolytics-dimension-repository_network_root_nwoRustPython/RustPython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release49918fc503387d6d65b6bc6ce6f711c14d17bfbd
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/RustPython/RustPython/pull/2367/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F2367%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F2367%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=RustPython%2FRustPython
Reloadhttps://github.com/RustPython/RustPython/pull/2367/files
Reloadhttps://github.com/RustPython/RustPython/pull/2367/files
Reloadhttps://github.com/RustPython/RustPython/pull/2367/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
RustPython https://github.com/RustPython
RustPythonhttps://github.com/RustPython/RustPython
Notifications https://github.com/login?return_to=%2FRustPython%2FRustPython
Fork 1.5k https://github.com/login?return_to=%2FRustPython%2FRustPython
Star 22.2k https://github.com/login?return_to=%2FRustPython%2FRustPython
Code https://github.com/RustPython/RustPython
Issues 287 https://github.com/RustPython/RustPython/issues
Pull requests 104 https://github.com/RustPython/RustPython/pulls
Discussions https://github.com/RustPython/RustPython/discussions
Actions https://github.com/RustPython/RustPython/actions
Projects https://github.com/RustPython/RustPython/projects
Models https://github.com/RustPython/RustPython/models
Wiki https://github.com/RustPython/RustPython/wiki
Security and quality 0 https://github.com/RustPython/RustPython/security
Insights https://github.com/RustPython/RustPython/pulse
Code https://github.com/RustPython/RustPython
Issues https://github.com/RustPython/RustPython/issues
Pull requests https://github.com/RustPython/RustPython/pulls
Discussions https://github.com/RustPython/RustPython/discussions
Actions https://github.com/RustPython/RustPython/actions
Projects https://github.com/RustPython/RustPython/projects
Models https://github.com/RustPython/RustPython/models
Wiki https://github.com/RustPython/RustPython/wiki
Security and quality https://github.com/RustPython/RustPython/security
Insights https://github.com/RustPython/RustPython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
coolreader18https://github.com/coolreader18
RustPython:masterhttps://github.com/RustPython/RustPython/tree/master
Conversation 11 https://github.com/RustPython/RustPython/pull/2367
Commits 16 https://github.com/RustPython/RustPython/pull/2367/commits
Checks 0 https://github.com/RustPython/RustPython/pull/2367/checks
Files changed https://github.com/RustPython/RustPython/pull/2367/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
Refactor benchmarking suite to use Criterion and add microbenchmarks https://github.com/RustPython/RustPython/pull/2367/files#top
Show all changes 16 commits https://github.com/RustPython/RustPython/pull/2367/files
5788e7b Refactor benchmarking suite to use Criterion orf Dec 10, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/5788e7bf8e1e0019e47f24ad6521c5836f3b146e
3a85967 Remove dependency orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/3a859672dea50a9a1e84e6ceb5d3fcdb14a45fe3
a788df4 Reduce to 30k orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/a788df4eaf71557d7becd02a28cba70e6b8e2d6b
6a841c0 Remove pystone second run orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/6a841c07f1786badcfc25f0d3e26d725b3f75afe
184f9aa Switch to 500 orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/184f9aa59c60cc1ff05bba424373f46b3a79309b
9f04f37 Fix strings orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/9f04f377cd1561b6f4a65837e8e8042c1598c90e
a605b93 Improve benchmarks orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/a605b93d0fff15dbfd5c0182f9fa2d0900e3a310
1e4fc5c fmt orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/1e4fc5c70f879b59a8d6cb02adb0a13a0a4f8be8
e745805 Exclude benchmarks from flake8 orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/e7458059fd41abe1ff66b71645c05783119eb37a
c447ffd Fix loading orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/c447ffd7591cf93a10237cfc0821593d62d77408
bfd05ee Switch benchmarkid orf Dec 11, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/bfd05eee24491856111330b9c5e2ca397b2edaef
dde8af1 Add microbenchmarks orf Dec 13, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/dde8af14f4912d3a96f941b75e8a48ec9e0de6b5
eb22ba1 Add microbenchmarks orf Dec 13, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/eb22ba12c61414771decf6c4f1215b1ec9209fb5
7818fd2 Fix micro benchmarks orf Dec 13, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/7818fd20833b7c3de8fdf7abe9681492a81f55ad
d36e150 Ignore benches orf Dec 13, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/d36e1500ebbdfc69ec5af0a14fdb2e9210f81bc9
b221dbb Allow running pystone standalone coolreader18 Dec 13, 2020 https://github.com/RustPython/RustPython/pull/2367/commits/b221dbb5d65f1bf7321a22a0111c345c623fccc1
Clear filters https://github.com/RustPython/RustPython/pull/2367/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
ci.yaml https://github.com/RustPython/RustPython/pull/2367/files#diff-944291df2c9c06359d37cc8833d182d705c9e8c3108e7cfe132d61a06e9133dd
.gitignore https://github.com/RustPython/RustPython/pull/2367/files#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947
Cargo.lock https://github.com/RustPython/RustPython/pull/2367/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87e
Cargo.toml https://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
README.md https://github.com/RustPython/RustPython/pull/2367/files#diff-a78b465f9dbab9f55bce48f68761be8cef21f8dc2501840b833524cd9c8f7551
mandelbrot.py https://github.com/RustPython/RustPython/pull/2367/files#diff-fdb630c8593335f78ec913db0fcff2744835ea474cf7f4aef733ee1544016e0a
nbody.py https://github.com/RustPython/RustPython/pull/2367/files#diff-81ec59aa356ffa6cf3a40b0fda7c256c626eff1f1c16350472dfdb3b696a1af7
pystone.py https://github.com/RustPython/RustPython/pull/2367/files#diff-a8416d8bbb23097497113e260faf940561833843b143f0187eef0f65ea7aec32
strings.py https://github.com/RustPython/RustPython/pull/2367/files#diff-51846fcbe7910ef90b55a52753b516e689911e1598e06e82f33e43763c3b9534
execution.rs https://github.com/RustPython/RustPython/pull/2367/files#diff-45dec9160d1a6d1dc8ee22dd72626ff449d4d6c519ccb2f3bc4521633cb952f4
microbenchmarks.rs https://github.com/RustPython/RustPython/pull/2367/files#diff-64eea763ace57eff71d3255cd4f1f05d365af6edcf556ff1b39e67704e5cd665
addition.py https://github.com/RustPython/RustPython/pull/2367/files#diff-f0b7991a303527af7bc13c29a053270cf516540277adbfe5a24bc04b4e544684
call_kwargs.py https://github.com/RustPython/RustPython/pull/2367/files#diff-0c4dcfaeb56260776ee8a0e3b75bb065d28aa2e8d872cac6392a50d11b29bf1d
call_simple.py https://github.com/RustPython/RustPython/pull/2367/files#diff-b6dce082457acffa1b617a99a93fba7d1e990df74df99de6b4f03c88604d3088
complex_class.py https://github.com/RustPython/RustPython/pull/2367/files#diff-4971035252f32c4abc205f3455f2a6082c282754b8e26eab32cfa125ea352995
comprehension_dict.py https://github.com/RustPython/RustPython/pull/2367/files#diff-1f7c9d4e8373d1e596e61cecb6790074e5e16325ec84aee9bb6b92c44fa08927
comprehension_list.py https://github.com/RustPython/RustPython/pull/2367/files#diff-a975f0a3eec0a2b6e07401b21014bb672f9e92c1e45b708ace4e93e75aae3f73
comprehension_set.py https://github.com/RustPython/RustPython/pull/2367/files#diff-bf9591235a353db84c2421075ca244dbb02a580b66c9a101c0594038054fd9c1
construct_object.py https://github.com/RustPython/RustPython/pull/2367/files#diff-be8636ad8271f64161ce9897fb92891c2f35059c90897f0e60e3c4f63ce54bd3
define_class.py https://github.com/RustPython/RustPython/pull/2367/files#diff-19d0dab85db55424da6e8243e77108a1914d76ca324b2a3463c80c861c2cdfbf
define_function.py https://github.com/RustPython/RustPython/pull/2367/files#diff-b7d08f60c6348ea512519fd7bead730f24334140c465a2c7b533f4e1334c4cd9
exception_context.py https://github.com/RustPython/RustPython/pull/2367/files#diff-24a31bf773e74ce94d8732773ebcef8c094d5273879613e4ea8ef28db3847c99
exception_nested.py https://github.com/RustPython/RustPython/pull/2367/files#diff-9004493253f73522972de6b7b01060ce83a912d8eb283019bee8bc440cc09f89
exception_simple.py https://github.com/RustPython/RustPython/pull/2367/files#diff-bd3e356d78e2cf9d2c3282bc2fccdb94780935aef447123e40012cc93c3f333f
loop_append.py https://github.com/RustPython/RustPython/pull/2367/files#diff-0378fd74f0ed8e4b71b3ebdaaadb03dcc6dd23e50fd343d67bf2977bf6603e27
loop_string.py https://github.com/RustPython/RustPython/pull/2367/files#diff-8f092a5200f5229d0e60055260f27bf9b6c6f7e8ee11430a52b2300b5db9d152
test_benchmarks.py https://github.com/RustPython/RustPython/pull/2367/files#diff-3cb37300a8d77c989b1cfe6121c21fd7f85d467dad1b84807bb0088e71135573
README.md https://github.com/RustPython/RustPython/pull/2367/files#diff-576c27794bae284efc278336a16d0649aaf599973a64d4b65dc67bc99bb7f000
bench.rs https://github.com/RustPython/RustPython/pull/2367/files#diff-05b9c77043c9f8c8d27da4f1fbf165490a9d77f47aaf03a7a2c4b31561558a82
minidom.py https://github.com/RustPython/RustPython/pull/2367/files#diff-d68fb8c28dfed97e9eb022e8b45699a3fa6a587219533349aca62f6558930185
.github/workflows/ci.yamlhttps://github.com/RustPython/RustPython/pull/2367/files#diff-944291df2c9c06359d37cc8833d182d705c9e8c3108e7cfe132d61a06e9133dd
View file https://github.com/RustPython/RustPython/blob/b221dbb5d65f1bf7321a22a0111c345c623fccc1/.github/workflows/ci.yaml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/2367/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/2367/files#diff-944291df2c9c06359d37cc8833d182d705c9e8c3108e7cfe132d61a06e9133dd
https://github.com/RustPython/RustPython/pull/2367/files#diff-944291df2c9c06359d37cc8833d182d705c9e8c3108e7cfe132d61a06e9133dd
.gitignorehttps://github.com/RustPython/RustPython/pull/2367/files#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947
View file https://github.com/RustPython/RustPython/blob/b221dbb5d65f1bf7321a22a0111c345c623fccc1/.gitignore
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/2367/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/2367/files#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947
Cargo.lockhttps://github.com/RustPython/RustPython/pull/2367/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87e
View file https://github.com/RustPython/RustPython/blob/b221dbb5d65f1bf7321a22a0111c345c623fccc1/Cargo.lock
Open in desktop https://desktop.github.com
how customized files appear on GitHubhttps://docs.github.com/github/administering-a-repository/customizing-how-changed-files-appear-on-github
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
Cargo.tomlhttps://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
View file https://github.com/RustPython/RustPython/blob/b221dbb5d65f1bf7321a22a0111c345c623fccc1/Cargo.toml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/2367/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/2367/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
Please reload this pagehttps://github.com/RustPython/RustPython/pull/2367/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.