René's URL Explorer Experiment


Title: fix(knowledge): wait for embedding admission instead of deferring documents for an hour by waleedlatif1 · Pull Request #7720 · simstudioai/sim · GitHub

Open Graph Title: fix(knowledge): wait for embedding admission instead of deferring documents for an hour by waleedlatif1 · Pull Request #7720 · simstudioai/sim

X Title: fix(knowledge): wait for embedding admission instead of deferring documents for an hour by waleedlatif1 · Pull Request #7720 · simstudioai/sim

Description: Summary Knowledge-path embedding batches now wait up to 60 s for the shared admission bucket instead of 5 s. Under load the expected queueing time is minutes, so a 5 s wait turned almost every contested batch into a full document re-dispatch with a 1-to-60-minute delay, for a limiter we run ourselves while the provider was healthy The embedding request bucket admits 64 concurrent starts instead of 8, so documents that begin together no longer lose a race for slots while the token budget sits unused. The per-minute rate is unchanged and still governs sustained throughput When an admission wait still expires, the document resumes after the bucket's stated wait, clamped to 10–60 s with jitter, and the yield counts against the processing-slice budget rather than the provider-failure attempts. Provider-side 429s and quota exhaustion keep the exponential ladder The admission deadline path now carries the bucket's last stated wait so the scheduler can use it Type of Change Bug fix Testing New tests: admission-timeout continuation uses the short jittered delay and slice budget, provider throttling keeps the exponential ladder, admission resumes are bounded independently, delay helper clamps and jitters; request burst asserted in admission tests; wait constant asserted in the embedding client test vitest across lib/knowledge/documents, lib/core/rate-limiter, lib/embeddings, background/knowledge-processing: 828 passing bun run type-check, bun run lint, bun run check:audits, docs-manifest:check pass Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Open Graph Description: Summary Knowledge-path embedding batches now wait up to 60 s for the shared admission bucket instead of 5 s. Under load the expected queueing time is minutes, so a 5 s wait turned almost every con...

X Description: Summary Knowledge-path embedding batches now wait up to 60 s for the shared admission bucket instead of 5 s. Under load the expected queueing time is minutes, so a 5 s wait turned almost every con...

Opengraph URL: https://github.com/simstudioai/sim/pull/7720

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:98c681a9-09b7-0803-ea35-ad7f59637bed
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idE6DE:277BD5:4759A7:634FDE:6AFB3931
html-safe-nonce23ac640c8e4dfe866f96c37fcbba5b25c55714e26591600d58d5ee84f767f373
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNkRFOjI3N0JENTo0NzU5QTc6NjM0RkRFOjZBRkIzOTMxIiwidmlzaXRvcl9pZCI6IjI0ODg3MDcxMjc3NDA5Mzg1NDUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmaca4db12ed13b9f5e6480e9d9f374c30b8adcfb8eccf28dc6735d31a170ba8d545
hovercard-subject-tagpull_request:4492825592
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/7720/files
twitter:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
og:image:altSummary Knowledge-path embedding batches now wait up to 60 s for the shared admission bucket instead of 5 s. Under load the expected queueing time is minutes, so a 5 s wait turned almost every con...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None2169b0dfbef67ed6b128074a5ed45bb7d5d1859783b10178b09a7765ad5f3693
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/simstudioai/sim git https://github.com/simstudioai/sim.git
octolytics-dimension-user_id199344406
octolytics-dimension-user_loginsimstudioai
octolytics-dimension-repository_id912559512
octolytics-dimension-repository_nwosimstudioai/sim
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id912559512
octolytics-dimension-repository_network_root_nwosimstudioai/sim
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release3b4fc8df5ef1c6757e733a23e6b81b6725021823
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/simstudioai/sim/pull/7720/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7720%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F7720%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=simstudioai%2Fsim
Reloadhttps://github.com/simstudioai/sim/pull/7720/files
Reloadhttps://github.com/simstudioai/sim/pull/7720/files
Reloadhttps://github.com/simstudioai/sim/pull/7720/files
simstudioaihttps://github.com/simstudioai
simhttps://github.com/simstudioai/sim
Codehttps://github.com/simstudioai/sim
Issues129 (129)https://github.com/simstudioai/sim/issues
Pull requests199 (199)https://github.com/simstudioai/sim/pulls
Actionshttps://github.com/simstudioai/sim/actions
Projectshttps://github.com/simstudioai/sim/projects
Security and qualityhttps://github.com/simstudioai/sim/security
Insightshttps://github.com/simstudioai/sim/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
waleedlatif1https://github.com/waleedlatif1
staginghttps://github.com/simstudioai/sim/tree/staging
fix/embedding-admission-waithttps://github.com/simstudioai/sim/tree/fix/embedding-admission-wait
Conversation 10 https://github.com/simstudioai/sim/pull/7720
Commits 2 https://github.com/simstudioai/sim/pull/7720/commits
Checks 33 https://github.com/simstudioai/sim/pull/7720/checks
Files changed https://github.com/simstudioai/sim/pull/7720/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7720/files
fix(knowledge): wait for embedding admission instead of deferring documents for an hour https://github.com/simstudioai/sim/pull/7720/files#top
Show all changes 2 commits https://github.com/simstudioai/sim/pull/7720/files
dba94dd fix(knowledge): wait for embedding admission instead of deferring doc… waleedlatif1 Sep 10, 2026 https://github.com/simstudioai/sim/pull/7720/commits/dba94ddfb6c26be5e9c6af7466921988b54db9f0
83a1c7d fix(knowledge): report only the admission wait still left at the dead… waleedlatif1 Sep 10, 2026 https://github.com/simstudioai/sim/pull/7720/commits/83a1c7d328bb9c99a67d5a037be1318099b47927
Clear filters https://github.com/simstudioai/sim/pull/7720/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7720/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/7720/files
provider-admission.test.ts https://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
provider-admission.ts https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
client.test.ts https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
client.ts https://github.com/simstudioai/sim/pull/7720/files#diff-bff32d70819c246fcf759a2ace839aa7e4569d2c62cf0658c5d2613a89d1170c
processing-provider-continuation.test.ts https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
processing-provider-continuation.ts https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
apps/sim/lib/core/rate-limiter/provider-admission.test.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/core/rate-limiter/provider-admission.test.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
https://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
https://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
https://github.com/simstudioai/sim/pull/7720/files#diff-ed7166be6bffcab5ce98866b7dc35f07d72928f9abfafc20f568c64844af5dff
apps/sim/lib/core/rate-limiter/provider-admission.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/core/rate-limiter/provider-admission.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
https://github.com/simstudioai/sim/pull/7720/files#diff-cc01384cab14bd6e1602686e2e245bc8aa7c43f8ed64a9715a3ab14a5373107e
apps/sim/lib/embeddings/client.test.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/embeddings/client.test.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
https://github.com/simstudioai/sim/pull/7720/files#diff-776dd477f81ddfab85944a291f03f7cd5365106f51de7615adc673aac72a589a
apps/sim/lib/embeddings/client.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-bff32d70819c246fcf759a2ace839aa7e4569d2c62cf0658c5d2613a89d1170c
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/embeddings/client.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-bff32d70819c246fcf759a2ace839aa7e4569d2c62cf0658c5d2613a89d1170c
Please reload this pagehttps://github.com/simstudioai/sim/pull/7720/files
https://github.com/simstudioai/sim/pull/7720/files#diff-bff32d70819c246fcf759a2ace839aa7e4569d2c62cf0658c5d2613a89d1170c
apps/sim/lib/knowledge/documents/processing-provider-continuation.test.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/knowledge/documents/processing-provider-continuation.test.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
https://github.com/simstudioai/sim/pull/7720/files#diff-97cd8c5b77f55124c501bae791288d16a580ae5b89223137161f82c418c77330
apps/sim/lib/knowledge/documents/processing-provider-continuation.tshttps://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
View file https://github.com/simstudioai/sim/blob/83a1c7d328bb9c99a67d5a037be1318099b47927/apps/sim/lib/knowledge/documents/processing-provider-continuation.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/7720/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
https://github.com/simstudioai/sim/pull/7720/files#diff-8ba5f4e5c0363ce883381fe0600a1c6132b396b5ed749e2474606ef37992671f
Please reload this pagehttps://github.com/simstudioai/sim/pull/7720/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.