Title: Python: unpin legacy CFG/ESSA from the AST cached stage (DCA measurement) by yoff · Pull Request #22116 · github/codeql · GitHub
Open Graph Title: Python: unpin legacy CFG/ESSA from the AST cached stage (DCA measurement) by yoff · Pull Request #22116 · github/codeql
X Title: Python: unpin legacy CFG/ESSA from the AST cached stage (DCA measurement) by yoff · Pull Request #22116 · github/codeql
Description: NoteDraft for measurement. This PR exists to run DCA and quantify how much of the shared-CFG dataflow flip's overhead is recovered by not computing the legacy CFG. It is based on the flip branch (yoff/python-shared-cfg-dataflow-flip, #21925), so the diff is exactly the one-commit change and the DCA a/b isolates the effect of the unpin. What The legacy CFG (Flow.qll) and legacy ESSA (Essa/SsaCompute/SsaDefinitions) are pinned into the always-on Stages::AST cached stage via Stages::AST::ref() (11 sites) and the matching Stages::AST::backref() disjuncts. backref() is referenced nowhere and optimizes to 1 = 1, so the ref/backref pattern only controls stage assignment. Because a cached stage is materialised as a unit once any of its predicates is demanded — and every query demands e.g. Expr.toString() — this forces the legacy CFG/ESSA to be computed for every query. After the shared-CFG dataflow flip, the security/dataflow queries no longer depend on the legacy CFG at all, so on the flip branch this computation is pure dead weight. This PR removes those pins. Since Stages::AST::ref() is 1 = 1, the change is result-preserving — it only changes stage scheduling. Verification (result-preserving) Full python-security-extended suite (52 queries) and django: legacy CFG/ESSA predicate families materialised drop from ~165 → 0, with byte-identical results. Cold, fresh-DB django: 59.9s → 57.6s wall (-j4); ~9.5s of legacy predicate self-time removed (≈1.3% of total analysis CPU). Question DCA should answer The legacy CFG the security queries drag in (~9.5s CPU, 1.3%) is a fraction of the new CFG+SSA the flip adds (~37.6s CPU, 5.3%), so a-priori this recovers only ~a quarter of the new-CFG cost, not the full flip overhead. DCA across the target set will give the authoritative per-project number. Caveats before this could merge This is a global stage change. Security queries win; the points-to/quality queries (which genuinely use legacy ESSA) will still compute it, but now in their own stage(s) rather than sharing the AST stage — need to confirm no regression there. A cleaner design would pin legacy CFG/ESSA to an explicit dedicated stage instead of leaving placement to the optimizer.
Open Graph Description: NoteDraft for measurement. This PR exists to run DCA and quantify how much of the shared-CFG dataflow flip's overhead is recovered by not computing the legacy CFG. It is based on the flip branc...
X Description: NoteDraft for measurement. This PR exists to run DCA and quantify how much of the shared-CFG dataflow flip's overhead is recovered by not computing the legacy CFG. It is based on the flip b...
Opengraph URL: https://github.com/github/codeql/pull/22116
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:fd71b5de-b217-267d-9db3-f1022e934a97 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 8020:9775E:41BD4C9:5D0707D:6A4FB4AD |
| html-safe-nonce | 9642055e5bf19e4f6cda302a9a642c04e8fa395e4e5e69278c6d685009108fee |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4MDIwOjk3NzVFOjQxQkQ0Qzk6NUQwNzA3RDo2QTRGQjRBRCIsInZpc2l0b3JfaWQiOiIzOTc0MTEyOTA4Mzk3MjI5MjI5IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | a141c7f26f20e18fa8f40fdfcfe5b644298f0046d867c5995c621f955c9ac2c7 |
| hovercard-subject-tag | pull_request:3981911582 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/github/codeql/pull/22116/files |
| twitter:image | https://avatars.githubusercontent.com/u/647347?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/647347?s=400&v=4 |
| og:image:alt | NoteDraft for measurement. This PR exists to run DCA and quantify how much of the shared-CFG dataflow flip's overhead is recovered by not computing the legacy CFG. It is based on the flip branc... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/github/codeql git https://github.com/github/codeql.git |
| octolytics-dimension-user_id | 9919 |
| octolytics-dimension-user_login | github |
| octolytics-dimension-repository_id | 143040428 |
| octolytics-dimension-repository_nwo | github/codeql |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 143040428 |
| octolytics-dimension-repository_network_root_nwo | github/codeql |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4b249b445842943ed31549e027f57a8ade9881ed |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width