Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim · GitHub
Open Graph Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim
X Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim
Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/page): sweep memory is one page of subscription rows regardless of fleet size, and an interrupted sweep (deploy, crash) resumes from the durable markers on the next run. Bounded concurrency (10 closes in flight via the shared mapWithConcurrency): closes of different subscriptions are independent atomic transactions serialized by the per-subscription guarded marker claim, so any interleaving is already safe — the bound only caps DB pressure. A total mapper keeps one failing close from rejecting its page. Deterministic member-lock ordering (ORDER BY user_id on the roster FOR UPDATE): today member_user_id_unique keeps org rosters disjoint and every other tracker locker is single-row, so no deadlock is currently constructible — the ordering is hygiene that also holds if membership exclusivity is ever relaxed. Partial index on exactly the sweep's candidate predicate (entitled + lagging marker), built CONCURRENTLY behind the migration runner's COMMIT breakpoint. The candidate scan becomes O(lagging) instead of O(fleet), and closes shrink the index as they land. The sweep summary log gains durationMs — the leading indicator to watch as the fleet grows. Why The serial executor was the deliberate simplest-correct choice at launch scale. Its wall clock grows linearly (~1s/close in practice), which stretches the post-rollover threshold-billing pause for tail subscriptions and would eventually outlive the cron's lock TTL. The coordination primitives (durable markers, guarded claims, tx-enlisted outbox, deterministic Stripe idempotency stems) were always parallel-safe; this PR just lets the executor use that. Validation Unit (cycle-close.test.ts): multi-page keyset iteration across the page-size boundary; a failing close is counted and isolated without sinking its page; the roster lock's sorted ordering is pinned. End-to-end against Stripe test mode (local harness, real test clocks, real webhooks, real invoices): staged 304 lagging subscriptions — 300 synthetic pro subs spanning two keyset pages, 2 multi-member orgs closing concurrently, and 2 real backdated test-clock subscriptions (pro + team) with genuine overage — then ran one sweep: candidates: 304, closed: 304, failed: 0, durationMs: 667 (the identical fleet took ~302s on the serial executor — ~450×) Both real subscriptions produced exactly one paid overage invoice each at the expected amounts (pro $14.75 = $45 − $0.25 refresh − $30 base; team $9.50 = $75 − $0.50 refresh − $60 base − $5 already threshold-billed), demonstrating exactly-once money under the parallel executor Zero invoices/outbox events for the no-overage fleet; per-member lastPeriodCost bookkeeping and tracker resets verified; markers exactly equal period_start (second-precision, matching Stripe's real period granularity) A second independent 300-sub fleet closed in 625ms; no-op sweeps run in 5–7ms Migration 0307 — additive partial index only, CREATE INDEX CONCURRENTLY IF NOT EXISTS after the runner's COMMIT; breakpoint with the standard invalid-build-replay guard. check:migrations passes. 🤖 Generated with Claude Code
Open Graph Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/pag...
X Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250...
Opengraph URL: https://github.com/simstudioai/sim/pull/7103
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:ce986b5c-4f71-394a-e0a8-1016ab63ef6b |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | ACF8:267719:B9A558:FE878C:6AFB544F |
| html-safe-nonce | 36583d814893e946b98880bf6bb8e086733b6a5de96060e1f41155984429c85b |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQ0Y4OjI2NzcxOTpCOUE1NTg6RkU4NzhDOjZBRkI1NDRGIiwidmlzaXRvcl9pZCI6IjUwMjM3MjYyNDA3Njg1NDM1MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | ea5811855dfb8d6b5ef0f9d2ffab2fd92608fa886d1f5091cc9d396f10648275 |
| hovercard-subject-tag | pull_request:4364355193 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/7103/files |
| twitter:image | https://avatars.githubusercontent.com/u/17950982?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/17950982?s=400&v=4 |
| og:image:alt | What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/pag... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | cbf66c024bb6d10e6955baa7413c54cf13a201bb |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width