René's URL Explorer Experiment


Title: Experiment: Make all data flow incremental by jbj · Pull Request #20028 · github/codeql · GitHub

Open Graph Title: Experiment: Make all data flow incremental by jbj · Pull Request #20028 · github/codeql

X Title: Experiment: Make all data flow incremental by jbj · Pull Request #20028 · github/codeql

Description: I've polished up this branch for testing and to communicate a proof-of-concept for what it would look like to have all data flow incremental, either diff-informed or "overlay-informed". The last commit uses forceLocal to add in the base-to-base results, but it has some problems: I wasn't able to test performance because it turns out we don't get any cache hits when forceLocal is in use. Alex is on it. It only works for Java because the predicate that calls forceLocal can't currently be marked as local?. The shared code in this PR won't compile for other languages than Java because it requires Node to be local. To achieve the full vision of incrementality, the type tracking library also needs to be overlay-aware. That library seems to be the main performance bottleneck that we know how to address in QL. As I'm going out on holiday now, I want to leave this PR for @alexet and @aschackmull to work from or take inspiration from.

Open Graph Description: I've polished up this branch for testing and to communicate a proof-of-concept for what it would look like to have all data flow incremental, either diff-informed or "overlay-informed"...

X Description: I've polished up this branch for testing and to communicate a proof-of-concept for what it would look like to have all data flow incremental, either diff-informed or "overlay-infor...

Opengraph URL: https://github.com/github/codeql/pull/20028

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:23cb2afe-a326-ebb8-91f7-12be9a220062
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idC894:4E072:212E29F:2FC9BC3:6A58E75E
html-safe-nonce7c3ab10724df0e97ca8da9e0f6bc85fc625d6a98c7928eb208bbc4f5badf5463
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDODk0OjRFMDcyOjIxMkUyOUY6MkZDOUJDMzo2QTU4RTc1RSIsInZpc2l0b3JfaWQiOiIzMzM2MTM0NTYyMTI0NzE5OTY2IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac98df83535ffaf97ab723ca97f4398527e890c3ae8c9c5fa932725e2ffabf1841
hovercard-subject-tagpull_request:2658998382
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/github/codeql/pull/20028/files
twitter:imagehttps://avatars.githubusercontent.com/u/308293?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/308293?s=400&v=4
og:image:altI've polished up this branch for testing and to communicate a proof-of-concept for what it would look like to have all data flow incremental, either diff-informed or "overlay-informed"...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None5f2a0c7865178af3d91dd9f13b0cdfc3c73a2529c873d2780bb4c01271a57ec6
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/github/codeql git https://github.com/github/codeql.git
octolytics-dimension-user_id9919
octolytics-dimension-user_logingithub
octolytics-dimension-repository_id143040428
octolytics-dimension-repository_nwogithub/codeql
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id143040428
octolytics-dimension-repository_network_root_nwogithub/codeql
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8aae7b8d6caacacf5c66eaeb2702d8dc88d85b4a
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/github/codeql/pull/20028/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F20028%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F20028%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=github%2Fcodeql
Reloadhttps://github.com/github/codeql/pull/20028/files
Reloadhttps://github.com/github/codeql/pull/20028/files
Reloadhttps://github.com/github/codeql/pull/20028/files
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
github https://github.com/github
codeqlhttps://github.com/github/codeql
Notifications https://github.com/login?return_to=%2Fgithub%2Fcodeql
Fork 2k https://github.com/login?return_to=%2Fgithub%2Fcodeql
Star 9.8k https://github.com/login?return_to=%2Fgithub%2Fcodeql
Code https://github.com/github/codeql
Issues 991 https://github.com/github/codeql/issues
Pull requests 433 https://github.com/github/codeql/pulls
Discussions https://github.com/github/codeql/discussions
Actions https://github.com/github/codeql/actions
Projects https://github.com/github/codeql/projects
Models https://github.com/github/codeql/models
Security and quality 0 https://github.com/github/codeql/security
Insights https://github.com/github/codeql/pulse
Code https://github.com/github/codeql
Issues https://github.com/github/codeql/issues
Pull requests https://github.com/github/codeql/pulls
Discussions https://github.com/github/codeql/discussions
Actions https://github.com/github/codeql/actions
Projects https://github.com/github/codeql/projects
Models https://github.com/github/codeql/models
Security and quality https://github.com/github/codeql/security
Insights https://github.com/github/codeql/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fgithub%2Fcodeql%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fgithub%2Fcodeql%2Fissues%2Fnew%2Fchoose
jbjhttps://github.com/jbj
github:mainhttps://github.com/github/codeql/tree/main
jbj:overlay-informed-dataflowhttps://github.com/jbj/ql/tree/overlay-informed-dataflow
Conversation 3 https://github.com/github/codeql/pull/20028
Findings https://github.com/github/codeql/pull/20028/findings
Commits 4 https://github.com/github/codeql/pull/20028/commits
Checks 25 https://github.com/github/codeql/pull/20028/checks
Files changed https://github.com/github/codeql/pull/20028/files
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
Experiment: Make all data flow incremental https://github.com/github/codeql/pull/20028/files#top
Show all changes 4 commits https://github.com/github/codeql/pull/20028/files
0c827a5 Java lib qlpack: Enable overlay compilation kaspersv Jun 23, 2025 https://github.com/github/codeql/pull/20028/commits/0c827a55525e75ee3b164e7b65559b49e6de3f47
41ef65e DataFlow:Run overlay-informed if not diff-informed jbj Jun 23, 2025 https://github.com/github/codeql/pull/20028/commits/41ef65e4c76e18a70df0183aed559ed397ad3879
799b285 DataFlow: overlay-informed flow includes the diff jbj Jun 30, 2025 https://github.com/github/codeql/pull/20028/commits/799b2859fe3b00d4db5b078a23ae8e11ce213aba
19acc8d TEMP: DataFlow: Java-only flow including base jbj Jul 4, 2025 https://github.com/github/codeql/pull/20028/commits/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8
Clear filters https://github.com/github/codeql/pull/20028/files
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
qlpack.yml https://github.com/github/codeql/pull/20028/files#diff-c1a7378f8b5d274938e402387158d27b2a101cde5f4d771b03bb484a7288edf2
DataFlowImplSpecific.qll https://github.com/github/codeql/pull/20028/files#diff-d085e0aca24cd8f2da65ea3b1dc67b78b8397a9becd7c413ca13b035d2a694ae
DataFlow.qll https://github.com/github/codeql/pull/20028/files#diff-518023a19a22b29249dfbdb30a6705a5c1d9f376061a5d2edb5058c9ebecda0c
DataFlowImpl.qll https://github.com/github/codeql/pull/20028/files#diff-422b0d1a44600cc15b2eb443f63e43d442017e3c5320c50a8d341433b41b1db7
DataFlowImplStage1.qll https://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
AlertFiltering.qll https://github.com/github/codeql/pull/20028/files#diff-408f67e35f0cbb44108659897cee9c4cfce82a07ead7128a8794c56a9607aae7
https://github.com/github/codeql/blob/main/CODEOWNERS#L16
java/ql/lib/qlpack.ymlhttps://github.com/github/codeql/pull/20028/files#diff-c1a7378f8b5d274938e402387158d27b2a101cde5f4d771b03bb484a7288edf2
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/java/ql/lib/qlpack.yml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-c1a7378f8b5d274938e402387158d27b2a101cde5f4d771b03bb484a7288edf2
https://github.com/github/codeql/blob/main/CODEOWNERS#L16
java/ql/lib/semmle/code/java/dataflow/internal/DataFlowImplSpecific.qllhttps://github.com/github/codeql/pull/20028/files#diff-d085e0aca24cd8f2da65ea3b1dc67b78b8397a9becd7c413ca13b035d2a694ae
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/java/ql/lib/semmle/code/java/dataflow/internal/DataFlowImplSpecific.qll
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-d085e0aca24cd8f2da65ea3b1dc67b78b8397a9becd7c413ca13b035d2a694ae
https://github.com/github/codeql/pull/20028/files#diff-d085e0aca24cd8f2da65ea3b1dc67b78b8397a9becd7c413ca13b035d2a694ae
https://github.com/github/codeql/blob/main/CODEOWNERS#L26
shared/dataflow/codeql/dataflow/DataFlow.qllhttps://github.com/github/codeql/pull/20028/files#diff-518023a19a22b29249dfbdb30a6705a5c1d9f376061a5d2edb5058c9ebecda0c
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/shared/dataflow/codeql/dataflow/DataFlow.qll
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-518023a19a22b29249dfbdb30a6705a5c1d9f376061a5d2edb5058c9ebecda0c
https://github.com/github/codeql/pull/20028/files#diff-518023a19a22b29249dfbdb30a6705a5c1d9f376061a5d2edb5058c9ebecda0c
https://github.com/github/codeql/blob/main/CODEOWNERS#L26
shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qllhttps://github.com/github/codeql/pull/20028/files#diff-422b0d1a44600cc15b2eb443f63e43d442017e3c5320c50a8d341433b41b1db7
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qll
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-422b0d1a44600cc15b2eb443f63e43d442017e3c5320c50a8d341433b41b1db7
https://github.com/github/codeql/pull/20028/files#diff-422b0d1a44600cc15b2eb443f63e43d442017e3c5320c50a8d341433b41b1db7
https://github.com/github/codeql/blob/main/CODEOWNERS#L26
shared/dataflow/codeql/dataflow/internal/DataFlowImplStage1.qllhttps://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/shared/dataflow/codeql/dataflow/internal/DataFlowImplStage1.qll
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
https://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
https://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
alexethttps://github.com/alexet
Aug 6, 2025https://github.com/github/codeql/pull/20028/files#r2257203913
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
jbjhttps://github.com/jbj
Aug 7, 2025https://github.com/github/codeql/pull/20028/files#r2259988288
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
https://github.com/github/codeql/pull/20028/files#diff-83269a1ba8238da82405f5488ec284c240fd03afca52a6b64f054ad273f43e64
https://github.com/github/codeql/blob/main/CODEOWNERS#L26
shared/util/codeql/util/AlertFiltering.qllhttps://github.com/github/codeql/pull/20028/files#diff-408f67e35f0cbb44108659897cee9c4cfce82a07ead7128a8794c56a9607aae7
View file https://github.com/jbj/ql/blob/19acc8dfe7302ed28e8656ea2049ca3bce2de9a8/shared/util/codeql/util/AlertFiltering.qll
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/20028/{{ revealButtonHref }}
https://github.com/github/codeql/pull/20028/files#diff-408f67e35f0cbb44108659897cee9c4cfce82a07ead7128a8794c56a9607aae7
https://github.com/github/codeql/pull/20028/files#diff-408f67e35f0cbb44108659897cee9c4cfce82a07ead7128a8794c56a9607aae7
https://github.com/github/codeql/pull/20028/files#diff-408f67e35f0cbb44108659897cee9c4cfce82a07ead7128a8794c56a9607aae7
Please reload this pagehttps://github.com/github/codeql/pull/20028/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.