Title: fix(knowledge): stop one env knob from setting the embedding request fan-out by waleedlatif1 · Pull Request #6852 · simstudioai/sim · GitHub
Open Graph Title: fix(knowledge): stop one env knob from setting the embedding request fan-out by waleedlatif1 · Pull Request #6852 · simstudioai/sim
X Title: fix(knowledge): stop one env knob from setting the embedding request fan-out by waleedlatif1 · Pull Request #6852 · simstudioai/sim
Description: Problem KB_CONFIG_CONCURRENCY_LIMIT was read in three places with three different meanings: Consumer What it actually controlled Effective value background/knowledge-processing.ts Trigger.dev queue depth — concurrent document task runs 20 lib/embeddings/client.ts Embedding API requests issued concurrently within one embed call 50 lib/knowledge/documents/service.ts /5 → concurrent documents in the in-process path 4 The first two multiply. Every admitted task run reaches the embed path and opens its own fan-out, so the default put on the order of 1,000 embedding requests in flight against a single provider key. A provider's rate limit is per key, so the pipeline held itself at its own limit — and no retry policy can absorb a load its own concurrency is generating. Two things hid this: The same variable was read with a different inline fallback in each place (50, 20, 20/5). createEnv runs with skipValidation: true, and t3-env returns runtimeEnv untouched in that mode, so the .default(...) declarations never execute — the inline fallbacks are the real defaults, and they disagreed with each other and with the declaration. KB_CONFIG_BATCH_SIZE had the same conflation, between chunks-per-embedding-request (2000) and documents-per-batch (/2 → 10). Separately, rate-limit rejections discarded what the provider said about when to retry. EmbeddingAPIError was built from status text only and the response headers were dropped, so retryWithExponentialBackoff's support for a server-stated wait was dead code on this path. Every attempt fired blind: 3 retries capped at 10s, ~7s of total backoff, all of it spent inside a window that had not reopened. Change One consumer per variable, so the fallbacks cannot drift: KB_CONFIG_CONCURRENCY_LIMIT — task-queue depth only KB_CONFIG_EMBEDDING_CONCURRENCY (new) — embedding fan-out, 50 → 8 KB_CONFIG_DOCUMENT_CONCURRENCY (new) — in-process document concurrency, default 4 KB_CONFIG_BATCH_SIZE — chunks per embedding request only KB_CONFIG_DOCUMENT_BATCH_SIZE (new) — documents per batch, default 10 The divisors are gone and the previous effective values are the declared defaults, so when these variables are unset the only behaviour change is the embedding fan-out. If a deployment sets KB_CONFIG_CONCURRENCY_LIMIT or KB_CONFIG_BATCH_SIZE explicitly, more moves — that is the whole point of the split, but it is worth stating exactly what: Set in env Before After KB_CONFIG_CONCURRENCY_LIMIT=50 queue 50, embedding 50, in-process docs 10 queue 50, embedding 8, in-process docs 4 KB_CONFIG_BATCH_SIZE=2000 2000 chunks/request, 1000 docs/batch 2000 chunks/request, 10 docs/batch Each of those collateral values was set by a divisor rather than by intent, so the new numbers are the intended ones — but they would be changes, not no-ops. Checked against every env source, and none of these variables are set anywhere: Source Keys KB_CONFIG_* set Secrets Manager, production 301 none Secrets Manager, staging 275 none Trigger.dev, prod (where the document task runs) 179 none Trigger.dev, staging 179 none trigger.config.ts syncs only DB_APP_NAME, so the worker env is the dashboard's and had to be checked separately from Secrets Manager. Since every variable is unset, the inline fallbacks are what run today, and for this deployment the sole behavioural change is the embedding fan-out, 50 → 8. Honor the provider's stated wait. The headers now travel with the error the way fetchWithRetry already does for connectors, via the existing attachRetryHeaders (non-enumerable, so the bag stays out of log lines). The wait is read from Retry-After, falling back to the reset header for whichever limit dimension is actually exhausted — a tokens-per-minute rejection and a requests-per-minute rejection reopen at different times, and a reset read while quota remains says nothing about this request. Those reset headers carry a Go duration (6m0s, 12ms, 23h47m36.648s), not the epoch seconds the shared connector helper reads. Teaching the shared reader both spellings would change retry behaviour for every connector on fetchWithRetry, so the provider-specific reading lives in a new lib/embeddings/rate-limit.ts instead. Note m vs ms: a loosely-scanning parser reads 12ms as twelve minutes, so the parser matches sticky and rejects anything it cannot fully consume. There is a test for exactly that. The retry budget is sized against a rate-limit window rather than a blip (5 retries, 30s ceiling) — the previous 10s ceiling clamped every stated wait to below the reopen time, spending the whole budget for nothing. It stays well inside KB_CONFIG_MAX_DURATION, and batches wait concurrently rather than in sequence. Verification 14,422 tests pass; 29/29 audits; type-check clean in touched code Each new test was verified to fail without its fix — including reordering the duration-unit alternation to confirm the ms/m test genuinely catches it Retry-budget assertions now derive from an exported constant rather than a hardcoded attempt count, so they cannot silently drift again Not in this PR The stuck-document sweep re-queues every pending/failed doc in its 7-day window on every sync, with no attempt counter and no cooldown (there is no retry-count column). Under sustained failure that re-attempts the whole backlog each cycle. Fixing it properly needs a schema migration, which does not belong in a config and retry change — filed as follow-up.
Open Graph Description: Problem KB_CONFIG_CONCURRENCY_LIMIT was read in three places with three different meanings: Consumer What it actually controlled Effective value background/knowledge-processing.ts Trigger.de...
X Description: Problem KB_CONFIG_CONCURRENCY_LIMIT was read in three places with three different meanings: Consumer What it actually controlled Effective value background/knowledge-processing.ts Trigger.de...
Opengraph URL: https://github.com/simstudioai/sim/pull/6852
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:7c139f68-dd9f-af7d-0908-ae7210d497bd |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | A08E:CE688:CA32D:11816C:6AFB7597 |
| html-safe-nonce | b342f0644437d3e31a2e285ba0aa0be4c850937d3d97840c4727356228753d7c |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMDhFOkNFNjg4OkNBMzJEOjExODE2Qzo2QUZCNzU5NyIsInZpc2l0b3JfaWQiOiI4NzgyNzUzMzA2NTUzMDU0NjE1IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 7d15c9f44d2439fe102ae799d5cf62e9463473bf77252d74c4821f1dfac9e599 |
| hovercard-subject-tag | pull_request:4316622520 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/6852/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Problem KB_CONFIG_CONCURRENCY_LIMIT was read in three places with three different meanings: Consumer What it actually controlled Effective value background/knowledge-processing.ts Trigger.de... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 0e29e7974fdda2feeaf5c891efd4b23b0734b23480232b62c7bdef13f40b5b48 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 2e8872b770cc4d5d0cc1f2026afab747a268e2ce |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width