René's URL Explorer Experiment


Title: Python extractor: overlay support by d10c · Pull Request #20206 · github/codeql · GitHub

Open Graph Title: Python extractor: overlay support by d10c · Pull Request #20206 · github/codeql

X Title: Python extractor: overlay support by d10c · Pull Request #20206 · github/codeql

Description: CodeQL: the libraries and queries that power security researchers around the world, as well as code scanning in GitHub Advanced Security - Python extractor: overlay support by d10c · Pull Request #20206 · github/codeql

Open Graph Description: This PR adds overlay support to the Python extractor, but no overlay compilation (to be merged separately since it needs further testing, see this PR). This PR also includes an initial pass at the ...

X Description: This PR adds overlay support to the Python extractor, but no overlay compilation (to be merged separately since it needs further testing, see this PR). This PR also includes an initial pass at the ...

Opengraph URL: https://github.com/github/codeql/pull/20206

X: @github

direct link

Domain: patch-diff.githubusercontent.com

route-pattern/:user_id/:repository/pull/:id/commits(.:format)
route-controllerpull_requests
route-actioncommits
fetch-noncev2:73de2518-9048-56ce-172b-ff92483f60bb
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idE9F0:6F99D:74A375:9FF25E:698F3124
html-safe-nonce5fc6b276bda5254d3791cfc9089fbdfd1eab62e909044dbf28b8ecd097e04065
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFOUYwOjZGOTlEOjc0QTM3NTo5RkYyNUU6Njk4RjMxMjQiLCJ2aXNpdG9yX2lkIjoiMTA5NzgyODMxMTYwNDYwNTIyMCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac60a572205788af27233f9f587717699eda5cfb9c9fe7e8b8eb8a2e4cd2544758
hovercard-subject-tagpull_request:2735779914
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/commits
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/github/codeql/pull/20206/commits
twitter:imagehttps://opengraph.githubassets.com/90f7f4950aa2191cc5ba25cc6b248fe3501c6e10bb2cf624ef5ca203aea1dd4b/github/codeql/pull/20206
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/90f7f4950aa2191cc5ba25cc6b248fe3501c6e10bb2cf624ef5ca203aea1dd4b/github/codeql/pull/20206
og:image:altThis PR adds overlay support to the Python extractor, but no overlay compilation (to be merged separately since it needs further testing, see this PR). This PR also includes an initial pass at the ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamed10c
hostnamegithub.com
expected-hostnamegithub.com
None6102991c714c1a6a27d05fb4f96ea6ca87a6750b4f093b95cc57ed1b84b145a1
turbo-cache-controlno-cache
go-importgithub.com/github/codeql git https://github.com/github/codeql.git
octolytics-dimension-user_id9919
octolytics-dimension-user_logingithub
octolytics-dimension-repository_id143040428
octolytics-dimension-repository_nwogithub/codeql
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id143040428
octolytics-dimension-repository_network_root_nwogithub/codeql
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasee2fe82f90e107669021e243add8f74ea2a9be1b3
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits#start-of-content
https://patch-diff.githubusercontent.com/
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F20206%2Fcommits
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F20206%2Fcommits
Sign up https://patch-diff.githubusercontent.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Fcommits&source=header-repo&source_repo=github%2Fcodeql
Reloadhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits
Reloadhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits
Reloadhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits
github https://patch-diff.githubusercontent.com/github
codeqlhttps://patch-diff.githubusercontent.com/github/codeql
Notifications https://patch-diff.githubusercontent.com/login?return_to=%2Fgithub%2Fcodeql
Fork 1.9k https://patch-diff.githubusercontent.com/login?return_to=%2Fgithub%2Fcodeql
Star 9.2k https://patch-diff.githubusercontent.com/login?return_to=%2Fgithub%2Fcodeql
Code https://patch-diff.githubusercontent.com/github/codeql
Issues 927 https://patch-diff.githubusercontent.com/github/codeql/issues
Pull requests 366 https://patch-diff.githubusercontent.com/github/codeql/pulls
Discussions https://patch-diff.githubusercontent.com/github/codeql/discussions
Actions https://patch-diff.githubusercontent.com/github/codeql/actions
Projects 0 https://patch-diff.githubusercontent.com/github/codeql/projects
Models https://patch-diff.githubusercontent.com/github/codeql/models
Security 0 https://patch-diff.githubusercontent.com/github/codeql/security
Insights https://patch-diff.githubusercontent.com/github/codeql/pulse
Code https://patch-diff.githubusercontent.com/github/codeql
Issues https://patch-diff.githubusercontent.com/github/codeql/issues
Pull requests https://patch-diff.githubusercontent.com/github/codeql/pulls
Discussions https://patch-diff.githubusercontent.com/github/codeql/discussions
Actions https://patch-diff.githubusercontent.com/github/codeql/actions
Projects https://patch-diff.githubusercontent.com/github/codeql/projects
Models https://patch-diff.githubusercontent.com/github/codeql/models
Security https://patch-diff.githubusercontent.com/github/codeql/security
Insights https://patch-diff.githubusercontent.com/github/codeql/pulse
d10chttps://patch-diff.githubusercontent.com/d10c
github:mainhttps://patch-diff.githubusercontent.com/github/codeql/tree/main
d10c:d10c/python-overlayhttps://patch-diff.githubusercontent.com/d10c/codeql/tree/d10c/python-overlay
Conversationhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206
Commits14 (14)https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits
Checkshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/checks
Files changedhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/files
Python extractor: overlay supporthttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits#top
d10chttps://patch-diff.githubusercontent.com/d10c
github:mainhttps://patch-diff.githubusercontent.com/github/codeql/tree/main
d10c:d10c/python-overlayhttps://patch-diff.githubusercontent.com/d10c/codeql/tree/d10c/python-overlay
Add overlay builtins to python dbschemehttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/5dbe2e2d81348465c5c3538e6eef4f6b3bc3b3c8
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
5dbe2e2https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/5dbe2e2d81348465c5c3538e6eef4f6b3bc3b3c8
https://patch-diff.githubusercontent.com/github/codeql/tree/5dbe2e2d81348465c5c3538e6eef4f6b3bc3b3c8
Turn on overlay support in codeql-extractor.ymlhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/a9f86402d5cf638196bcb75db6e571e89a48dbee
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
a9f8640https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/a9f86402d5cf638196bcb75db6e571e89a48dbee
https://patch-diff.githubusercontent.com/github/codeql/tree/a9f86402d5cf638196bcb75db6e571e89a48dbee
Add database upgrade/downgrade scriptshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/1f01b1126b3bec8e25edc9d9b55457bf3cc843ba
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
1f01b11https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/1f01b1126b3bec8e25edc9d9b55457bf3cc843ba
https://patch-diff.githubusercontent.com/github/codeql/tree/1f01b1126b3bec8e25edc9d9b55457bf3cc843ba
Support CODEQL_PATH_TRANSFORMER env var in python path renamerhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/c0497af72da81a52a61d8bef86c971f9682ead1d
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
c0497afhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/c0497af72da81a52a61d8bef86c971f9682ead1d
https://patch-diff.githubusercontent.com/github/codeql/tree/c0497af72da81a52a61d8bef86c971f9682ead1d
Python extractor: in overlay mode, traverse only changed fileshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/f4defdc22b932b42c3da3dbcb1042625db56f352
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
f4defdchttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/f4defdc22b932b42c3da3dbcb1042625db56f352
https://patch-diff.githubusercontent.com/github/codeql/tree/f4defdc22b932b42c3da3dbcb1042625db56f352
Write overlay metadata at end of extraction.https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/44219d0ced4883f204dd3cafd10ae2edb664d30f
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
44219d0https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/44219d0ced4883f204dd3cafd10ae2edb664d30f
https://patch-diff.githubusercontent.com/github/codeql/tree/44219d0ced4883f204dd3cafd10ae2edb664d30f
Discard predicates for dbscheme elementshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/8edd6c156eeba2c4969341df8af304402bd7ddac
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
8edd6c1https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/8edd6c156eeba2c4969341df8af304402bd7ddac
https://patch-diff.githubusercontent.com/github/codeql/tree/8edd6c156eeba2c4969341df8af304402bd7ddac
Add synthetic data to dbscheme.stats for `databaseMetadata`/`overlayChangedFiles`https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/693add0b0ec0456c70d7f0a4478f8b678f5a8251
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
693add0https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/693add0b0ec0456c70d7f0a4478f8b678f5a8251
https://patch-diff.githubusercontent.com/github/codeql/tree/693add0b0ec0456c70d7f0a4478f8b678f5a8251
Overlay.qll: remove `overlay[local?] module;` in favour of explicit annotationshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/627a7b47cbf307adbd0298c56c42ba428afdd29d
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
627a7b4https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/627a7b47cbf307adbd0298c56c42ba428afdd29d
https://patch-diff.githubusercontent.com/github/codeql/tree/627a7b47cbf307adbd0298c56c42ba428afdd29d
Extractor: fall back to full extraction on overlay changes json read errorhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/5ef321545e9f4296f21fb15d2e7493d65758d8fe
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
5ef3215https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/5ef321545e9f4296f21fb15d2e7493d65758d8fe
https://patch-diff.githubusercontent.com/github/codeql/tree/5ef321545e9f4296f21fb15d2e7493d65758d8fe
Path transformer: handle Windows-style pathshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/4bea5d727da3ef3eaed744a5b9f01d0530d8475f
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
4bea5d7https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/4bea5d727da3ef3eaed744a5b9f01d0530d8475f
https://patch-diff.githubusercontent.com/github/codeql/tree/4bea5d727da3ef3eaed744a5b9f01d0530d8475f
Extractor: use filesystem-canonical path comparison to determine built-in fileshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/01cf7e91b4f726917fae44f626fc1b0ba699188f
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
01cf7e9https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/01cf7e91b4f726917fae44f626fc1b0ba699188f
https://patch-diff.githubusercontent.com/github/codeql/tree/01cf7e91b4f726917fae44f626fc1b0ba699188f
Extractor: move overlay-changes check from traverser to workerhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/2ed8dc7e085b1cdb46b81183b722b27a6d4f94db
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
2ed8dc7https://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/2ed8dc7e085b1cdb46b81183b722b27a6d4f94db
https://patch-diff.githubusercontent.com/github/codeql/tree/2ed8dc7e085b1cdb46b81183b722b27a6d4f94db
Overlay.qll: Streamline discardable entities into one Discardable superclasshttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/c2f026d154145dbee0c0f9c489581dedff2f27ae
https://patch-diff.githubusercontent.com/d10c
d10chttps://patch-diff.githubusercontent.com/github/codeql/commits?author=d10c
c2f026dhttps://patch-diff.githubusercontent.com/github/codeql/pull/20206/commits/c2f026d154145dbee0c0f9c489581dedff2f27ae
https://patch-diff.githubusercontent.com/github/codeql/tree/c2f026d154145dbee0c0f9c489581dedff2f27ae
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.