René's URL Explorer Experiment


Title: gh-153804: `_remote_debugging`: Tachyon Oracle by maurycy · Pull Request #153806 · python/cpython · GitHub

Open Graph Title: gh-153804: `_remote_debugging`: Tachyon Oracle by maurycy · Pull Request #153806 · python/cpython

X Title: gh-153804: `_remote_debugging`: Tachyon Oracle by maurycy · Pull Request #153806 · python/cpython

Description: Please read #153804 for raison d'être. The ultimate idea is that instead of "I generated a small script with Claude, therefore we're better than we were" we repeatedly and rigorously measure and ensure that our changes to the profiler do not make things worse or, even sometimes, improve the reported data. The PR introduces a new script suited for Tachyon, named Tools/inspection/oracle_external_inspection.py, mostly reusing the snippets from Tools/inspection/benchmark_external_inspection.py. For each snippet it reports: [130] 2026-07-16T10:56:25.380112000+0200 maurycy@gimel /Users/maurycy/work/cpython (tachyon-oracle e6a6696?) % sudo ./python.exe Tools/inspection/oracle_external_inspection.py --snippet flat_alternating 3.16.0a0 (heads/tachyon-oracle:e6a6696eec2, Jul 15 2026, 16:39:58) [Clang 21.0.0 (clang-2100.1.1.101)] cases=flat_alternating runs=1 duration=3.0 rate_khz=100.0 warmup=0.7 poisson_sampling=False flat_alternating (get_stack_trace) ref_keys=6 mode samples µs errors% impossible impossible% tvd_excess tvd_floor blocking-nocache 291259 8.72 0.00 0 0.00 ref - blocking-cache 299140 8.45 0.00 0 0.00 0.031 0.002 live-cache 251090 6.89 16.30 1857 0.74 0.090 0.002 live-nocache 251006 6.51 16.33 2422 0.96 0.094 0.002 I was able to nicely reproduce #151424 with 63.35±2.19 impossible% rate, that matches that finding: [130] 2026-06-21T21:36:22.319345000+0200 maurycy@gimel /Users/maurycy/work/cpython (maurycy/bye-abba 981433d?) % sudo ./python.exe Tools/inspection/oracle_external_inspection.py --snippet flat_alternating --rate 1 --duration 3 --runs 8 3.16.0a0 (heads/main:6679ac07d88, Jul 16 2026, 11:19:41) [Clang 21.0.0 (clang-2100.1.1.101)] cases=flat_alternating runs=8 duration=3.0 rate_khz=1.0 warmup=0.7 poisson_sampling=False flat_alternating (get_stack_trace) ref_keys=6 mode samples µs errors% impossible impossible% tvd_excess tvd_floor blocking-nocache 24008 19.34±0.30 0.00±0.00 0 0.00±0.00 ref - blocking-cache 24008 18.51±0.33 0.00±0.00 0 0.00±0.00 0.002±0.007 0.017 live-cache 21042 7.70±0.28 12.35±0.61 13333 63.35±2.19 0.319±0.035 0.025 live-nocache 20501 9.31±1.30 14.61±1.14 60 0.29±0.10 0.028±0.004 0.018 Please see Tools/inspection/oracle_external_inspection.py --help for all the flags. Not pasting it here for the brevity. Please see Tools/inspection/oracle_external_inspection.py for a run with all the snippets. The first two columns are relatively obvious to read. Important bits: for the flat_alternating we're using get_stack_trace but get_async_stack_trace is also supported for relevant snippets, blocking-nocache is used as the best available reference. I don't believe it's the ground truth but it's the best one that we have available (we can call it the "oracle"). the errors% is measured similarly to how the collector does it, that is by catching the exception, impossible and impossible% is the number of samples that are simply impossible for a given snippet (eg: leafs simply do not match, they couldn't be called etc.) tvd_floor is how much TVD two samples of the same distribution would show at these sample sizes - like: what is the noise? tvd_excess is the TVD minus the floor, the distance that cannot be explained by the noise. The closer to the zero, the better we're here. In short, the idea behind tvd_* is to measure the drift (bias? noise?) against the reference. I tried to familiarize myself with the state of the literature what is the best measure of bias, drift etc. in case of statistical samplers. Maybe it's me but I couldn't find any undisputed gold standard here. I'm more than open to any other measure, and it's very likely that we'll find something better. The PR is big but most of it is copying the snippets to a separate file. The length of the script is on par with the existing benchmarking script. Closes #153804 Issue: gh-153804

Open Graph Description: Please read #153804 for raison d'être. The ultimate idea is that instead of "I generated a small script with Claude, therefore we're better than we were" we repeatedly and rigorou...

X Description: Please read #153804 for raison d'être. The ultimate idea is that instead of "I generated a small script with Claude, therefore we're better than we were" we repeat...

Opengraph URL: https://github.com/python/cpython/pull/153806

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:f9b6bb2c-5e33-e502-9fa0-da33efea9d59
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idA8FE:3DD488:496640D:5D12FB7:6B0253F5
html-safe-nonce3988210d9480a4b26e8a164e5366e8e9e7f1cdb18bf6c9ee33c4c84a84b79535
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBOEZFOjNERDQ4ODo0OTY2NDBEOjVEMTJGQjc6NkIwMjUzRjUiLCJ2aXNpdG9yX2lkIjoiMzM5MTQyNjAxNjg5NDQwNzY2OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacb0170b6e37aed5c6c1f29ffa38e16ba804584f4862e69dfac44d4e6a84eb709c
hovercard-subject-tagpull_request:4067396078
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/153806/files
twitter:imagehttps://avatars.githubusercontent.com/u/5383?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/5383?s=400&v=4
og:image:altPlease read #153804 for raison d'être. The ultimate idea is that instead of "I generated a small script with Claude, therefore we're better than we were" we repeatedly and rigorou...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/153806/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F153806%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F153806%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/153806/files
Reloadhttps://github.com/python/cpython/pull/153806/files
Reloadhttps://github.com/python/cpython/pull/153806/files
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
pablogsalhttps://github.com/pablogsal
python:mainhttps://github.com/python/cpython/tree/main
maurycy:tachyon-oraclehttps://github.com/maurycy/cpython/tree/tachyon-oracle
Conversation 18 https://github.com/python/cpython/pull/153806
Commits 18 https://github.com/python/cpython/pull/153806/commits
Checks 48 https://github.com/python/cpython/pull/153806/checks
Files changed https://github.com/python/cpython/pull/153806/files
Please reload this pagehttps://github.com/python/cpython/pull/153806/files
gh-153804: _remote_debugging: Tachyon Oracle https://github.com/python/cpython/pull/153806/files#top
Show all changes 18 commits https://github.com/python/cpython/pull/153806/files
e6a6696 tachyon oracle maurycy Jul 15, 2026 https://github.com/python/cpython/pull/153806/commits/e6a6696eec22f07e9010059af68457e703933a54
3e2909b news maurycy Jul 16, 2026 https://github.com/python/cpython/pull/153806/commits/3e2909ba72fa852ed47fb819033b6c9360d825c0
bf7006d news in tools, not library maurycy Jul 16, 2026 https://github.com/python/cpython/pull/153806/commits/bf7006d2adfc597a246050f1bc506f3f2dd2c838
955ccdb subprocess.DEVNULL maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/955ccdb04d485afdac4acce678d2bbfe748b2952
de0d2aa validate args.run maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/de0d2aab0c258ad2264884f11461ad5b67690af6
fafa4b7 do not sample past deadline maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/fafa4b72ba40927608cbef823649201e4dc51480
d35cf26 show raw_tvd maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/d35cf26da32c1e831c876bc00b525b53ee68a4c0
2431e3b do not loop forever maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/2431e3b107c743a4c35147b94f46d819e49e35fd
b5d08a5 comment on the limitation maurycy Jul 29, 2026 https://github.com/python/cpython/pull/153806/commits/b5d08a5218e2078a7b7e64b0c905fbb86147ebfd
784b7c6 docstring maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/784b7c6bf989a8acfb979c3d88c61a91627fc503
815dfba simple gen testing maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/815dfbafe3c1d79524e47d33bbea104b4d0d3b38
cc9157f test all classifiers maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/cc9157f7f9255de369f8fbf318f4552226f39194
d08fe69 better no lineno maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/d08fe69c511e3e9b915a02ab98d9d7970e3e3299
d922081 test both ways maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/d922081541570a01d572dcd6a9abb340ca99d172
071395a merge test_classify_gen maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/071395ab7f31d49cf0d03d290a090648b102d01e
53ef884 time.perf_counter maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/53ef884df2af5b22c68ecf4e3beadc61e425aa7c
4364604 docstring maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/43646044fc80425622d94070def37de51d8a870d
813259a -stderr=subprocess.DEVNULL, maurycy Jul 30, 2026 https://github.com/python/cpython/pull/153806/commits/813259ab17514df2f89ef297a2c99b3a9f8de86c
Clear filters https://github.com/python/cpython/pull/153806/files
Please reload this pagehttps://github.com/python/cpython/pull/153806/files
Please reload this pagehttps://github.com/python/cpython/pull/153806/files
test_inspection.py https://github.com/python/cpython/pull/153806/files#diff-b665967eaf882bc34f896b9017525f1f7998c22460c153485707f0a8384db45c
2026-07-16-11-38-15.gh-issue-153804.KSTYg7.rst https://github.com/python/cpython/pull/153806/files#diff-ed91a6f6872c90ab9bc1dfdaa203277be8011b1e664c49cf56efb9c81743521c
benchmark_external_inspection.py https://github.com/python/cpython/pull/153806/files#diff-751cdb5bae3d1d8d58b6ec72c71347d76b7b9f31f84e7cd950229c22a451b1ef
oracle_external_inspection.py https://github.com/python/cpython/pull/153806/files#diff-7608fd99ae74e00690ba709d4980679e964836913fd06510247043416b91a976
snippets.py https://github.com/python/cpython/pull/153806/files#diff-7d793e5461213e01f132e78fb4ddabc08e6452114c2551c5950c9bcecd4d931e
Lib/test/test_tools/test_inspection.pyhttps://github.com/python/cpython/pull/153806/files#diff-b665967eaf882bc34f896b9017525f1f7998c22460c153485707f0a8384db45c
View file https://github.com/maurycy/cpython/blob/813259ab17514df2f89ef297a2c99b3a9f8de86c/Lib/test/test_tools/test_inspection.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/153806/{{ revealButtonHref }}
Misc/NEWS.d/next/Tools-Demos/2026-07-16-11-38-15.gh-issue-153804.KSTYg7.rsthttps://github.com/python/cpython/pull/153806/files#diff-ed91a6f6872c90ab9bc1dfdaa203277be8011b1e664c49cf56efb9c81743521c
View file https://github.com/maurycy/cpython/blob/813259ab17514df2f89ef297a2c99b3a9f8de86c/Misc/NEWS.d/next/Tools-Demos/2026-07-16-11-38-15.gh-issue-153804.KSTYg7.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/153806/{{ revealButtonHref }}
Tools/inspection/benchmark_external_inspection.pyhttps://github.com/python/cpython/pull/153806/files#diff-751cdb5bae3d1d8d58b6ec72c71347d76b7b9f31f84e7cd950229c22a451b1ef
View file https://github.com/maurycy/cpython/blob/813259ab17514df2f89ef297a2c99b3a9f8de86c/Tools/inspection/benchmark_external_inspection.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/153806/{{ revealButtonHref }}
https://github.com/python/cpython/pull/153806/files#diff-751cdb5bae3d1d8d58b6ec72c71347d76b7b9f31f84e7cd950229c22a451b1ef
https://github.com/python/cpython/pull/153806/files#diff-751cdb5bae3d1d8d58b6ec72c71347d76b7b9f31f84e7cd950229c22a451b1ef
Please reload this pagehttps://github.com/python/cpython/pull/153806/files
Please reload this pagehttps://github.com/python/cpython/pull/153806/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.