Title: feat(tools): queue hosted-key tool calls instead of failing with 429 by TheodoreSpeaks · Pull Request #4416 · simstudioai/sim · GitHub
Open Graph Title: feat(tools): queue hosted-key tool calls instead of failing with 429 by TheodoreSpeaks · Pull Request #4416 · simstudioai/sim
X Title: feat(tools): queue hosted-key tool calls instead of failing with 429 by TheodoreSpeaks · Pull Request #4416 · simstudioai/sim
Description: Summary Hosted-key tool calls (Sim-provided keys, not BYOK) now enqueue onto a per-workspace+provider FIFO queue. Only the head of the queue consumes from the token bucket — strict ordering, no racing. Different workspaces have independent queues. BYOK paths short-circuit before any of this and are unaffected. Total wait (queue position + bucket refill) capped at 5 minutes; over the cap returns the existing 429 result. Crash-tolerant: each ticket has a heartbeat key (TTL 30s, refreshed every 10s while waiting). Dead heads are reaped lazily by the next caller. Queue list TTL is 10 minutes for fully abandoned queues. One Lua script per poll (reap + head-check + self-presence-check atomic) keeps Redis traffic low under contention. Bump Exa search hosted RPM from 5 → 60. New telemetry: platform.hosted_key.queue_waited (with queuePosition field) and platform.hosted_key.queue_wait_exceeded. Type of Change New feature Testing 39 hosted-key tests pass (15 queue + 24 rate-limiter, including FIFO ordering, head-only consume, dead-head reap, cap-exceeded, missing-ticket fall-through) 141/141 across rate-limiter + tools regression Manually verified in dev: depth, head rotation, heartbeat refresh, drain rate match the bucket config bun run lint clean bun run check:api-validation:strict passes Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)
Open Graph Description: Summary Hosted-key tool calls (Sim-provided keys, not BYOK) now enqueue onto a per-workspace+provider FIFO queue. Only the head of the queue consumes from the token bucket — strict ordering, no ra...
X Description: Summary Hosted-key tool calls (Sim-provided keys, not BYOK) now enqueue onto a per-workspace+provider FIFO queue. Only the head of the queue consumes from the token bucket — strict ordering, no ra...
Opengraph URL: https://github.com/simstudioai/sim/pull/4416
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:9777a5fb-04d8-825f-a724-bc5663c978dc |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | BE2E:2A369:915DA7:CEDE3A:6AFB51B0 |
| html-safe-nonce | 6b32ff3407fef9f98204fd5ebb38ad33e07e032c1144ca6b867e408d5e2f03ec |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCRTJFOjJBMzY5OjkxNURBNzpDRURFM0E6NkFGQjUxQjAiLCJ2aXNpdG9yX2lkIjoiMzkyMjY3NDQ0NzMzNjI5Njg4MCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | a1bbd7b1dd50de316ec498bd81522c8f78efac5097bfb61a6f00a19010da397e |
| hovercard-subject-tag | pull_request:3619106181 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/4416/files |
| twitter:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| og:image:alt | Summary Hosted-key tool calls (Sim-provided keys, not BYOK) now enqueue onto a per-workspace+provider FIFO queue. Only the head of the queue consumes from the token bucket — strict ordering, no ra... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | cbf66c024bb6d10e6955baa7413c54cf13a201bb |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width