René's URL Explorer Experiment


Title: bpo-40958: Avoid buffer overflow in the parser when indexing the current line by pablogsal · Pull Request #20875 · python/cpython · GitHub

Open Graph Title: bpo-40958: Avoid buffer overflow in the parser when indexing the current line by pablogsal · Pull Request #20875 · python/cpython

X Title: bpo-40958: Avoid buffer overflow in the parser when indexing the current line by pablogsal · Pull Request #20875 · python/cpython

Description: After thinking a bit more about the whole problem I found that we were processing the offsets incorrectly if the input is raw (no need to transform the offset to a character offset). Once we process the column offsets correctly, is quite straightforward to fix the incorrect access. As an example of why we were processing the offset incorrectly previously, consider this code (current master without this PR): ❯ ./python Python 3.9.0b1+ (heads/3.9:299d3d1c52, Jun 8 2020, 22:11:46) [GCC 10.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> Python = "Ṕýţĥòñ" + 34 $ lelummsmdsads + File "", line 1 Python = "Ṕýţĥòñ" + 34 $ lelummsmdsads + ^ SyntaxError: invalid syntax while the old parser points correctly to the $ token: ❯ ./python -Xoldparser Python 3.9.0b1+ (heads/3.9:299d3d1c52, Jun 8 2020, 22:11:46) [GCC 10.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> Python = "Ṕýţĥòñ" + 34 $ lelummsmdsads + File "", line 1 Python = "Ṕýţĥòñ" + 34 $ lelummsmdsads + ^ SyntaxError: invalid syntax https://bugs.python.org/issue40958

Open Graph Description: After thinking a bit more about the whole problem I found that we were processing the offsets incorrectly if the input is raw (no need to transform the offset to a character offset). Once we proces...

X Description: After thinking a bit more about the whole problem I found that we were processing the offsets incorrectly if the input is raw (no need to transform the offset to a character offset). Once we proces...

Opengraph URL: https://github.com/python/cpython/pull/20875

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:8656b2dd-2707-ff58-9da0-be3182bbf58f
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idB0A6:4BD90:F09DA7:150466F:696A3234
html-safe-nonce7aaf2ad1c0d1b363df198adc97ebf0eee43d0b5c513d799871f22e449affe969
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMEE2OjRCRDkwOkYwOURBNzoxNTA0NjZGOjY5NkEzMjM0IiwidmlzaXRvcl9pZCI6IjcyMzIyMjk3MzEzODgyNDA0MzYiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac1146e16712fa8871ac378f4aa26a1763488330c169234e189ca64acae2a030eb
hovercard-subject-tagpull_request:434227332
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/20875/files
twitter:imagehttps://avatars.githubusercontent.com/u/11718525?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/11718525?s=400&v=4
og:image:altAfter thinking a bit more about the whole problem I found that we were processing the offsets incorrectly if the input is raw (no need to transform the offset to a character offset). Once we proces...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneade130928b3890d20ecf9c887468a17e88d5c9a740ea2a85c2c2162e7c58e623
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasec00842167ca2e5b13e8171812965d503e925ece6
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/20875/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20875%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20875%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/20875/files
Reloadhttps://github.com/python/cpython/pull/20875/files
Reloadhttps://github.com/python/cpython/pull/20875/files
python https://github.com/python
cpythonhttps://github.com/python/cpython
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
Notifications https://github.com/login?return_to=%2Fpython%2Fcpython
Fork 33.9k https://github.com/login?return_to=%2Fpython%2Fcpython
Star 71.1k https://github.com/login?return_to=%2Fpython%2Fcpython
Code https://github.com/python/cpython
Issues 5k+ https://github.com/python/cpython/issues
Pull requests 2.1k https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects 31 https://github.com/python/cpython/projects
Security Uh oh! There was an error while loading. Please reload this page. https://github.com/python/cpython/security
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
Insights https://github.com/python/cpython/pulse
Code https://github.com/python/cpython
Issues https://github.com/python/cpython/issues
Pull requests https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
pablogsalhttps://github.com/pablogsal
python:masterhttps://github.com/python/cpython/tree/master
pablogsal:bpo-40958https://github.com/pablogsal/cpython/tree/bpo-40958
Conversation 19 https://github.com/python/cpython/pull/20875
Commits 4 https://github.com/python/cpython/pull/20875/commits
Checks 0 https://github.com/python/cpython/pull/20875/checks
Files changed https://github.com/python/cpython/pull/20875/files
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
bpo-40958: Avoid buffer overflow in the parser when indexing the current line https://github.com/python/cpython/pull/20875/files#top
Show all changes 4 commits https://github.com/python/cpython/pull/20875/files
308ac73 bpo-40958: Avoid buffer overflow in the parser when indexing the curr… pablogsal Jun 15, 2020 https://github.com/python/cpython/pull/20875/commits/308ac73eb4810f4c8a453547d9a96f9ee651e612
4654706 Use Py_ssize_t in more places and ensure that col_offset is positive pablogsal Jun 15, 2020 https://github.com/python/cpython/pull/20875/commits/4654706483a032ad315b0522df46682e19a36165
a58c5bb Allow col_offset=0 pablogsal Jun 15, 2020 https://github.com/python/cpython/pull/20875/commits/a58c5bb1aa8201d9fa98df0dd8377334517c3a03
626a2ed Use Py_ssize_t in the Token struct pablogsal Jun 15, 2020 https://github.com/python/cpython/pull/20875/commits/626a2eda8d897562a4c302e6515936e7853a4bad
Clear filters https://github.com/python/cpython/pull/20875/files
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
2020-06-15-01-20-44.bpo-40958.7O2Wh1.rst https://github.com/python/cpython/pull/20875/files#diff-87a8459575079efc8739ec7110e364b895a0f59294a4fb199faeec4dfbeda573
pegen.c https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
pegen.h https://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
Misc/NEWS.d/next/Core and Builtins/2020-06-15-01-20-44.bpo-40958.7O2Wh1.rsthttps://github.com/python/cpython/pull/20875/files#diff-87a8459575079efc8739ec7110e364b895a0f59294a4fb199faeec4dfbeda573
View file https://github.com/pablogsal/cpython/blob/626a2eda8d897562a4c302e6515936e7853a4bad/Misc/NEWS.d/next/Core%20and%20Builtins/2020-06-15-01-20-44.bpo-40958.7O2Wh1.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/20875/{{ revealButtonHref }}
Parser/pegen.chttps://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
View file https://github.com/pablogsal/cpython/blob/626a2eda8d897562a4c302e6515936e7853a4bad/Parser/pegen.c
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/20875/{{ revealButtonHref }}
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
Please reload this pagehttps://github.com/python/cpython/pull/20875/files
https://github.com/python/cpython/pull/20875/files#diff-ec9a9d08c5b1d518997db0f5b4795c528a41efc35507d6c2a384eee43f047cb7
Parser/pegen.hhttps://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
View file https://github.com/pablogsal/cpython/blob/626a2eda8d897562a4c302e6515936e7853a4bad/Parser/pegen.h
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/20875/{{ revealButtonHref }}
https://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
https://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
https://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
https://github.com/python/cpython/pull/20875/files#diff-5803bf7cd8fdced0a7448e74074f36018f2206ab77ad13dc7451b1948b950852
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.