Title: fix(billing): enforce the plan usage limit mid-run and bill runs that outlive their period by waleedlatif1 · Pull Request #8461 · simstudioai/sim · GitHub
Open Graph Title: fix(billing): enforce the plan usage limit mid-run and bill runs that outlive their period by waleedlatif1 · Pull Request #8461 · simstudioai/sim
X Title: fix(billing): enforce the plan usage limit mid-run and bill runs that outlive their period by waleedlatif1 · Pull Request #8461 · simstudioai/sim
Description: Summary
Chat runs are moving off a fixed wall-clock deadline, so spend has to be bounded by the plan's usage limit while a run is in progress, not only when it starts. This PR adds Sim's side of mid-run enforcement and makes sure a run that outlives its billing period is still invoiced exactly once.
Mid-run usage verdict
POST /api/billing/update-cost now returns usageExceeded (and, when true, a usageUpgrade card payload) at the top level of every answer: the 200 and the duplicate 409. The verdict is read through the cached execution usage gate, so a run under its limit pays no extra ledger read on most steps. Older workers ignore the fields.
update-cost answers for direct-v1 runs too, judging the payer saved in the run's account decision. The verdict read is bounded to 1 s, well inside the worker's 5 s callback timeout, and answers not-exceeded past it.
Continuation validation (POST /api/copilot/api-keys/validate, purpose: "continuation") also reads the original payer's spend, for attributed and direct-v1 runs alike, so a worker can poll it on a cadence. An admitted direct-v1 verdict is cached like an attributed one. A continuation 402 carries a body declared in the contract:
usage limit: { code: "USAGE_LIMIT_EXCEEDED", error, usageUpgrade }
blocked account: { code: "BILLING_BLOCKED", error }
A new turn's 402 stays empty, as before: the worker replaces any validation 402 body with its own message.
The update-cost schema ties the fields together: usageUpgrade is present exactly when usageExceeded is true.
The lifecycle's own continuation admission reads spend through the same gate. A refused continuation shows the existing upgrade card and stops the worker run, so the user's next message after upgrading continues normally instead of being refused as busy.
Card and blocked accounts
One resolver chooses the card's action and copy from the payer's plan. Sim's own card and the payload the worker writes into its durable log both use it, so live, replayed, and reloaded chats render the same card. A member over the cap their organization set gets copy naming who can raise it.
A blocked account (payment failed, dispute) is refused as blocked and never gets the usage card. The direct-v1 gate checks the actor and the saved payer for a block before their spend, through one account block check shared with continuation validation.
Every usage-limit refusal, including a worker 402 on the first leg, goes through one handler that shows the card and stops the worker run.
Fail-open mid-run, fail-closed at admission
Mid-run checks judge the period a run's charges land in. A Stripe-period payer's charges roll forward, so it is judged against its current subscription period (cached for a minute), and a rollover or an early anchor move is judged against the period charges now land in. Any other payer (an enterprise reporting window, or the open default period) never rolls: its run stays one ledger row whose creation time falls in the admitted window, which is summed by creation time, so the admitted window is judged even after it ends.
A direct-v1 run is judged against the payer saved in its account decision, never one re-selected from the actor's current memberships.
If usage or the current period cannot be read, or the period ends while its verdict is read, the mid-run paths report "unknown" and keep the run going; the next callback judges the next period. Admission before a run still refuses when usage cannot be read.
Runs that outlive their billing period
A cumulative charge is recorded against the billing period frozen at admission. For a long run, later top-ups would land in a period whose close had already invoiced it, so that overage was never billed.
When a Stripe-period payer's subscription has moved to a later period (including a period start that moved forward inside the old one), further spend is recorded in one new row per later period, stamped with the current period. Threshold settlement follows the stamped period. This applies to attributed runs and to direct-v1 runs, whose account decision now carries the payer's subscription from admission. Decisions minted before this still parse and keep today's behaviour.
The subscription row is share-locked for every period-aware write, so a rollover or an early period-start move waits for any in-flight write to the old period and the close's sums are final.
The cost callback refuses an idempotency key containing @, so a request's period row keys never collide with another request's.
Rows are only ever created forward, under the existing per-request lock, and totals converge on the maximum cumulative cost, so retries and out-of-order callbacks stay idempotent.
Mixed versions and rollback: code that predates period rows reads only the first row. If it handles a later callback for a run that already has period rows, during a deploy or after a rollback, it can re-add those rows' amount. That can only be invoiced if it lands between the rollover and that period's close, which waits at least an hour, and only for runs spanning a rollover. The exposure is one run's post-rollover spend, cents to dollars.
Other
The analytics settle window is unchanged. Its doc now notes that a turn running past it can under-report in a cached hour; invoices, thresholds, and the usage gate read live sums.
Syncs the worker's billing contract (usageExceeded, UsageUpgrade, UsageLimitRefusal).
Worker contract
Read usageExceeded and usageUpgrade from the top level of the update-cost answer.
Poll continuation validation. On a 402 with USAGE_LIMIT_EXCEEDED, write
Open Graph Description: Summary Chat runs are moving off a fixed wall-clock deadline, so spend has to be bounded by the plan's usage limit while a run is in progress, not only when it starts. This PR adds Sim's si...
X Description: Summary Chat runs are moving off a fixed wall-clock deadline, so spend has to be bounded by the plan's usage limit while a run is in progress, not only when it starts. This PR adds Sim&...
Opengraph URL: https://github.com/simstudioai/sim/pull/8461
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:9c406de9-29b1-e512-df7d-9c049cf63ea5 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D74C:345450:18334D5:21278C2:6AFB8ADE |
| html-safe-nonce | 51b4d7e9d1396d9a15aa707a2876e7c62f45794aefd717ad1c54493a63cfbc61 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENzRDOjM0NTQ1MDoxODMzNEQ1OjIxMjc4QzI6NkFGQjhBREUiLCJ2aXNpdG9yX2lkIjoiMzUxOTEzNzM0MTg4NjkyNTUzNCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | a2c72655c259abd20242df9d9781163e5efdf4d413d4ea124ac61c1799f10d79 |
| hovercard-subject-tag | pull_request:4689030194 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8461/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Summary Chat runs are moving off a fixed wall-clock deadline, so spend has to be bounded by the plan's usage limit while a run is in progress, not only when it starts. This PR adds Sim's si... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 9694464143465782ecae2a97650b2a2b0708d5c31c74f41ee86da57e315e2735 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 05d967182208691fc7f2e6af52e82fa7fabf6bd5 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width