René's URL Explorer Experiment


Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim · GitHub

Open Graph Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim

X Title: improvement(billing): paginate and parallelize the cycle-close sweep by icecrasher321 · Pull Request #7103 · simstudioai/sim

Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/page): sweep memory is one page of subscription rows regardless of fleet size, and an interrupted sweep (deploy, crash) resumes from the durable markers on the next run. Bounded concurrency (10 closes in flight via the shared mapWithConcurrency): closes of different subscriptions are independent atomic transactions serialized by the per-subscription guarded marker claim, so any interleaving is already safe — the bound only caps DB pressure. A total mapper keeps one failing close from rejecting its page. Deterministic member-lock ordering (ORDER BY user_id on the roster FOR UPDATE): today member_user_id_unique keeps org rosters disjoint and every other tracker locker is single-row, so no deadlock is currently constructible — the ordering is hygiene that also holds if membership exclusivity is ever relaxed. Partial index on exactly the sweep's candidate predicate (entitled + lagging marker), built CONCURRENTLY behind the migration runner's COMMIT breakpoint. The candidate scan becomes O(lagging) instead of O(fleet), and closes shrink the index as they land. The sweep summary log gains durationMs — the leading indicator to watch as the fleet grows. Why The serial executor was the deliberate simplest-correct choice at launch scale. Its wall clock grows linearly (~1s/close in practice), which stretches the post-rollover threshold-billing pause for tail subscriptions and would eventually outlive the cron's lock TTL. The coordination primitives (durable markers, guarded claims, tx-enlisted outbox, deterministic Stripe idempotency stems) were always parallel-safe; this PR just lets the executor use that. Validation Unit (cycle-close.test.ts): multi-page keyset iteration across the page-size boundary; a failing close is counted and isolated without sinking its page; the roster lock's sorted ordering is pinned. End-to-end against Stripe test mode (local harness, real test clocks, real webhooks, real invoices): staged 304 lagging subscriptions — 300 synthetic pro subs spanning two keyset pages, 2 multi-member orgs closing concurrently, and 2 real backdated test-clock subscriptions (pro + team) with genuine overage — then ran one sweep: candidates: 304, closed: 304, failed: 0, durationMs: 667 (the identical fleet took ~302s on the serial executor — ~450×) Both real subscriptions produced exactly one paid overage invoice each at the expected amounts (pro $14.75 = $45 − $0.25 refresh − $30 base; team $9.50 = $75 − $0.50 refresh − $60 base − $5 already threshold-billed), demonstrating exactly-once money under the parallel executor Zero invoices/outbox events for the no-overage fleet; per-member lastPeriodCost bookkeeping and tracker resets verified; markers exactly equal period_start (second-precision, matching Stripe's real period granularity) A second independent 300-sub fleet closed in 625ms; no-op sweeps run in 5–7ms Migration 0307 — additive partial index only, CREATE INDEX CONCURRENTLY IF NOT EXISTS after the runner's COMMIT; breakpoint with the standard invalid-build-replay guard. check:migrations passes. 🤖 Generated with Claude Code

Open Graph Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/pag...

X Description: What Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250...

Opengraph URL: https://github.com/simstudioai/sim/pull/7103

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:ce986b5c-4f71-394a-e0a8-1016ab63ef6b
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idACF8:267719:B9A558:FE878C:6AFB544F
html-safe-nonce36583d814893e946b98880bf6bb8e086733b6a5de96060e1f41155984429c85b
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQ0Y4OjI2NzcxOTpCOUE1NTg6RkU4NzhDOjZBRkI1NDRGIiwidmlzaXRvcl9pZCI6IjUwMjM3MjYyNDA3Njg1NDM1MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacea5811855dfb8d6b5ef0f9d2ffab2fd92608fa886d1f5091cc9d396f10648275
hovercard-subject-tagpull_request:4364355193
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/7103/files
twitter:imagehttps://avatars.githubusercontent.com/u/17950982?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/17950982?s=400&v=4
og:image:altWhat Scales the billing cycle-close sweep's executor from one-shot-serial to keyset-paginated + bounded-parallel, without touching the close logic itself: Keyset pagination (id cursor, 250/pag...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/simstudioai/sim git https://github.com/simstudioai/sim.git
octolytics-dimension-user_id199344406
octolytics-dimension-user_loginsimstudioai
octolytics-dimension-repository_id912559512
octolytics-dimension-repository_nwosimstudioai/sim
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id912559512
octolytics-dimension-repository_network_root_nwosimstudioai/sim
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasecbf66c024bb6d10e6955baa7413c54cf13a201bb
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/simstudioai/sim/pull/7103/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7103%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7103%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=simstudioai%2Fsim
Reloadhttps://github.com/simstudioai/sim/pull/7103/files
Reloadhttps://github.com/simstudioai/sim/pull/7103/files
Reloadhttps://github.com/simstudioai/sim/pull/7103/files
simstudioaihttps://github.com/simstudioai
simhttps://github.com/simstudioai/sim
Codehttps://github.com/simstudioai/sim
Issues129 (129)https://github.com/simstudioai/sim/issues
Pull requests197 (197)https://github.com/simstudioai/sim/pulls
Actionshttps://github.com/simstudioai/sim/actions
Projectshttps://github.com/simstudioai/sim/projects
Security and qualityhttps://github.com/simstudioai/sim/security
Insightshttps://github.com/simstudioai/sim/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
icecrasher321https://github.com/icecrasher321
staginghttps://github.com/simstudioai/sim/tree/staging
billing-cycle-close-scalehttps://github.com/simstudioai/sim/tree/billing-cycle-close-scale
Conversation 11 https://github.com/simstudioai/sim/pull/7103
Commits 2 https://github.com/simstudioai/sim/pull/7103/commits
Checks 30 https://github.com/simstudioai/sim/pull/7103/checks
Files changed https://github.com/simstudioai/sim/pull/7103/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7103/files
improvement(billing): paginate and parallelize the cycle-close sweep https://github.com/simstudioai/sim/pull/7103/files#top
Show all changes 2 commits https://github.com/simstudioai/sim/pull/7103/files
3e94f66 improvement(billing): paginate and parallelize the cycle-close sweep Aug 26, 2026 https://github.com/simstudioai/sim/pull/7103/commits/3e94f667c009d6c21693ab201b75c3724fe21ef8
0ef5651 fix(billing): restore the runner's lock_timeout and inline entitled s… Aug 26, 2026 https://github.com/simstudioai/sim/pull/7103/commits/0ef56510e8429dcb1fdc5db8e462dc4313a6ee9a
Clear filters https://github.com/simstudioai/sim/pull/7103/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7103/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7103/files
cycle-close.test.ts https://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
cycle-close.ts https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
0307_add_subscription_cycle_close_lagging_idx.sql https://github.com/simstudioai/sim/pull/7103/files#diff-aed4b64beb232af631673f6eb5f192403ce90e8e8f0519457d33931e2f898428
0307_snapshot.json https://github.com/simstudioai/sim/pull/7103/files#diff-dce63a53d3a209f69fd9e323368f335a21d6f1d95ae4d22fe6a6fcc537c140c0
_journal.json https://github.com/simstudioai/sim/pull/7103/files#diff-37150cc65ad16ee6fbc5fa318fc096be7730e16d63eaf863e3df5521b7030649
schema.ts https://github.com/simstudioai/sim/pull/7103/files#diff-9a959e2ce3cb4c86957bbae021d06eae364f7b0cb18964f5873bab53220af928
apps/sim/lib/billing/cycle-close.test.tshttps://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
View file https://github.com/simstudioai/sim/blob/0ef56510e8429dcb1fdc5db8e462dc4313a6ee9a/apps/sim/lib/billing/cycle-close.test.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7103/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
https://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
https://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
https://github.com/simstudioai/sim/pull/7103/files#diff-b2337272c53cc2156973e12119a51ddcef5670a4852c51d895a1fe8624e8bf4a
apps/sim/lib/billing/cycle-close.tshttps://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
View file https://github.com/simstudioai/sim/blob/0ef56510e8429dcb1fdc5db8e462dc4313a6ee9a/apps/sim/lib/billing/cycle-close.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7103/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
https://github.com/simstudioai/sim/pull/7103/files#diff-1573d069352d91221b13ba135abb1eb1cb348fe2c573cd534338339f452ed27b
packages/db/migrations/0307_add_subscription_cycle_close_lagging_idx.sqlhttps://github.com/simstudioai/sim/pull/7103/files#diff-aed4b64beb232af631673f6eb5f192403ce90e8e8f0519457d33931e2f898428
View file https://github.com/simstudioai/sim/blob/0ef56510e8429dcb1fdc5db8e462dc4313a6ee9a/packages/db/migrations/0307_add_subscription_cycle_close_lagging_idx.sql
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7103/{{ revealButtonHref }}
Please reload this pagehttps://github.com/simstudioai/sim/pull/7103/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7103/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.