Title: perf(admin): read dashboard ledger usage one bounded scan per entity by waleedlatif1 · Pull Request #9448 · simstudioai/sim · GitHub
Open Graph Title: perf(admin): read dashboard ledger usage one bounded scan per entity by waleedlatif1 · Pull Request #9448 · simstudioai/sim
X Title: perf(admin): read dashboard ledger usage one bounded scan per entity by waleedlatif1 · Pull Request #9448 · simstudioai/sim
Description: Summary The admin dashboard computed each page's per-entity spend and workflow runs in one statement that OR'd every entity's period together. The planner answers that with a BitmapOr and a heap fetch for every matched ledger row, or with a scan over every organization entry in the period index, filtered row by row. Both run on the primary. Each read is now a UNION ALL with one arm per entity. Every arm is bounded by that entity's period on the existing indexes: usage_log_billing_entity_period_cost_idx for stamped periods and usage_log_billing_entity_created_at_cost_idx for reporting periods. Spend is index-only. Workflow runs are a separate statement on the same indexes. No index carries execution_id, so this statement fetches each matched workflow row from the heap. Keeping it separate leaves the spend scans index-only. A dedicated index was rejected: it would add a write to every workflow ledger insert to speed up a staff-only page. The four copies of the period predicate are consolidated. The org list, org summary/detail, historical actors and personal users views now use buildUsageAnalyticsScope, which already defines the reporting-vs-stamped period predicate for usage analytics. The reads stay on the primary. The replica serves these index pages from disk rather than cache, and it cancels statements held past its replay delay (recovery conflicts). The largest entities' periods can reach that delay. What the dashboard shows is unchanged. A workflow run is still a distinct execution_id among the entity's source = 'workflow' ledger rows in the period. Execution logs are not a substitute: they are subject to retention, they include runs with no billed lines, and they don't record which entity a run was billed to. No schema change. Type of Change Bug fix New feature Breaking change Documentation Other: performance Testing dashboard-usage.integration.ts runs against real Postgres. It checks spend and run counts per entity, per user, filtered by user, and for historical actors. The seed covers: multi-row executions zero-cost runs a run whose rows have two different actors non-workflow sources and workflow-test workflow rows with a null execution_id rows stamped with the previous period a resumed run stamped with a period that has since ended rows billed to a user entity with the same id, or to no entity reporting-period start/end boundaries a run billed partly to another entity duplicate windows A local differential run (not committed) compared the new readers with the previous OR'd queries on 400k randomized ledger rows across 25 organizations and 25 user entities, mixing stamped and reporting periods. Results were identical for all four reads. Spend arms planned as Index Only Scans with zero heap fetches; run-count arms planned as Index Scans bounded per entity. Against the production planner (plain EXPLAIN on the primary), for a large yearly reporting period plus a stamped period: The previous statement scanned every organization entry in the period index and filtered it. The spend arms are now Index Only Scans bounded by entity and period. The run-count arms are Index Scans with entity, period and source in the index condition. There is no sequential scan anywhere. check:migrations (no new migrations), drizzle-kit generate (no changes), check:audits, type-check, lint, and the lib/admin unit tests all pass. Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)
Open Graph Description: Summary The admin dashboard computed each page's per-entity spend and workflow runs in one statement that OR'd every entity's period together. The planner answers that with a BitmapOr a...
X Description: Summary The admin dashboard computed each page's per-entity spend and workflow runs in one statement that OR'd every entity's period together. The planner answers that with ...
Opengraph URL: https://github.com/simstudioai/sim/pull/9448
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:f02072fc-eff8-8cbb-f5c8-cb8e1a2c5439 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | ACA0:12D29A:8B3DC:CF0E8:6AFB1976 |
| html-safe-nonce | 416a52af757526e196052f55f1d8d40fc17217ea00b37bd8f85831351dd0b0cf |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQ0EwOjEyRDI5QTo4QjNEQzpDRjBFODo2QUZCMTk3NiIsInZpc2l0b3JfaWQiOiI1OTY4MjE2ODQ4OTk3MjI2ODcwIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 9b7c844519a98e9dc4655fde8b8d399e60c247ad6fba435c94e8570760355ec8 |
| hovercard-subject-tag | pull_request:5073510008 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/9448/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Summary The admin dashboard computed each page's per-entity spend and workflow runs in one statement that OR'd every entity's period together. The planner answers that with a BitmapOr a... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 24933791aba0f10b2f189f8b953a1452dd79933933ac13a0a67b8ad3acee3d9f |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 030711cc180b563614a434f4376acf7da5d445fa |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width