Title: feat: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments · Issue #515 · github/squad · GitHub
Open Graph Title: feat: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments · Issue #515 · github/squad
X Title: feat: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments · Issue #515 · github/squad
Description: Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments Problem Issue #450 added a circuit breaker template for model rate limiting — great foundation. But in production with multiple Ralphs across mac...
Open Graph Description: Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments Problem Issue #450 added a circuit breaker template for model rate limiting — great foundation. But in pr...
X Description: Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments Problem Issue #450 added a circuit breaker template for model rate limiting — great foundation. But i...
Opengraph URL: https://github.com/github/squad/issues/515
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"feat: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments","articleBody":"## Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments\n\n### Problem\n\nIssue #450 added a circuit breaker template for model rate limiting — great foundation. But in production with multiple Ralphs across machines (or pods on K8s), single-instance circuit breakers aren't enough. Each Ralph independently hits API limits, causing cascading failures across the fleet.\n\nThe core problems at scale:\n1. **No coordination** — 5 Ralphs each think they have full API quota\n2. **Thundering herd** — All Ralphs retry simultaneously after rate limit resets\n3. **Priority inversion** — Low-priority work exhausts quota before critical work runs\n4. **Reactive only** — Circuit opens AFTER 429, wasting the failed request\n\n### Proposed Solution: 6-Pattern Rate Limiting Architecture\n\n**1. Traffic Light (RAAS)** — GREEN/AMBER/RED throttling from response headers:\n- GREEN (>20% remaining): Normal operation\n- AMBER (5-20%): Only P0 agents proceed\n- RED (\u003c5%): Block all except emergency P0\n\n**2. Cooperative Token Pool (CMARP)** — Shared `~/.squad/rate-pool.json`:\n- P0 agents (Lead) get 40% of quota\n- P1 agents (specialists) get 35%\n- P2 agents (Ralph, Scribe) get 25%\n- Stale leases auto-recovered after 5 minutes\n\n**3. Predictive Circuit Breaker (PCB)** — Opens BEFORE 429:\n- Tracks last 10 `(timestamp, remaining)` pairs from response headers\n- Linear regression predicts ETA to quota exhaustion\n- Opens circuit when ETA \u003c 120 seconds\n- Prevents the 429 from ever happening\n\n**4. Priority Retry Windows (PWJG)** — Non-overlapping jitter:\n- P0: 500ms–5s retry window\n- P1: 2s–30s retry window\n- P2: 5s–60s retry window\n- Higher priority agents recover first\n\n**5. Resource Epoch Tracker (RET)** — Heartbeat-leased allocations:\n- Each agent registers a 5-minute lease\n- Crashed agent allocations auto-recycle\n\n**6. Cascade Dependency Detector (CDD)** — Backpressure from downstream failures\n\n### Implementation Available\n\nI have all 6 patterns implemented and running in production:\n- `rate-limit-manager.ps1` (200+ lines) — Cooperative rate pooling\n- Predictive circuit breaker in `ralph-watch.ps1`\n- KEDA integration via `keda-copilot-scaler` (Go, 51 tests) — scales pods to zero when rate limited\n- Running with 7+ concurrent agents on AKS\n\nBuilds on top of the existing `ralph-circuit-breaker.md` template from #450/#451.\n\n### References\n- Extends #450 (circuit breaker template — merged as docs-only)\n- Related to #236 (persistent Ralph)","author":{"url":"https://github.com/tamirdresher","@type":"Person","name":"tamirdresher"},"datePublished":"2026-03-22T19:57:42Z","url":"https://github.com/github/squad/issues/515"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:8710f016-e72a-4b81-126b-6e97953b4f9f |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D4EC:14F601:26A6399:310E4BB:6B017A03 |
| html-safe-nonce | ba26c540c3ca6bc36544d9d9de7704d28702dfdde5b11a4902d7711e21f1e4a1 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENEVDOjE0RjYwMToyNkE2Mzk5OjMxMEU0QkI6NkIwMTdBMDMiLCJ2aXNpdG9yX2lkIjoiODQ0NDY2MDQxNzM4NDUxMjAwMyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 1550a315d06f9ad6639566c4d9ce58906b625cfd0725b5585bbcff19cb1bd294 |
| hovercard-subject-tag | issue:4116966761 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/github/squad/515/issue_layout |
| twitter:image | https://opengraph.githubassets.com/9780371a5dc1c5882bf9a7d094a21c85016a1673bfa558271b2fd9fedfcf35fa/github/squad/issues/515 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/9780371a5dc1c5882bf9a7d094a21c85016a1673bfa558271b2fd9fedfcf35fa/github/squad/issues/515 |
| og:image:alt | Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments Problem Issue #450 added a circuit breaker template for model rate limiting — great foundation. But in pr... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | tamirdresher |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/github/squad git https://github.com/github/squad.git |
| octolytics-dimension-user_id | 9919 |
| octolytics-dimension-user_login | github |
| octolytics-dimension-repository_id | 1151205052 |
| octolytics-dimension-repository_nwo | github/squad |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 1151205052 |
| octolytics-dimension-repository_network_root_nwo | github/squad |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width