René's URL Explorer Experiment


Title: improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper by waleedlatif1 · Pull Request #8713 · simstudioai/sim · GitHub

Open Graph Title: improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper by waleedlatif1 · Pull Request #8713 · simstudioai/sim

X Title: improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper by waleedlatif1 · Pull Request #8713 · simstudioai/sim

Description: Summary The per-minute outbox and workspace-file-search crons started a Trigger.dev run every tick even with nothing to do. Both now check first, and all three gated scheduled passes share one helper. Shared helper — lib/core/async-jobs/scheduled-pass.ts triggerScheduledPass: the windowed tasks.trigger (idempotency key ${keyPrefix ?? taskId}:${window}, idempotencyKeyTTL: '5m', region) that outbox, file search, and knowledge projection each hand-rolled. Keys and options are byte-identical to before, so in-flight dedupe is unaffected across the deploy startScheduledPass: due → inline or Trigger.dev → shared ScheduledPassResult ({ triggered, backend, jobId }), replacing KnowledgeProjectionSweepResult / the file-search result type Each call site keeps its policy explicit by computing due itself: outbox maintenance window + fail-open, file search fail-open, knowledge projection fail-closed (unchanged). Outbox keeps its synchronous self-hosted branch local because the route reports its output. cleanup-stale-executions uses the job-queue path and is left as is Outbox gate — starts a run only when: a pending event is due — the claim phase's own readyEventTypesQuery, so gate and claimer share one predicate a processing lease is stale — the reaper's predicate, now shared via isStuckProcessing / STUCK_PROCESSING_THRESHOLD_MS or it's a maintenance tick (every 5th minute), so document recovery, background-work reaping, and pruning keep running on an idle queue File-search dispatch gate — starts a run only when one of the dispatcher's own phases would act, each sharing its predicate with that phase: backfill page due (isBackfillPageDue), expired build (fileSearchBuildExpired), stale claim (staleClaim), or a queued workspace. Reconcile interval unchanged (hourly) since db:push environments rely on that walk. enqueue-dispatch.ts now uses import type for the task instead of a runtime import Both gates fail open (a check error starts the run as before — billing, seat sync, invitations, and KB/file indexing ride these crons) and cover both the Trigger.dev and inline backends. Cron routes return 200 triggered: false on an idle tick; response bodies otherwise unchanged. Latency for due work stays ≤ 1 tick. Supersedes #8714 (folded in so both gates can share the helper). Type of Change Improvement Testing Integration on real Postgres: outbox: future pending / fresh lease / completed / dead-letter → no work; event due exactly at now → work; lease exactly at the stale threshold → work; plan has no Seq Scan and stays under 100 buffers on a 100k-row backlog file search: each work leg reports work; the idle case reports no work and prepareWorkspaceFileSearchDispatch does nothing on the same data; no Seq Scan Unit: each backend's own result for idle / maintenance / check-failure ticks; exact tasks.trigger call pinned for all three passes (keys, TTLs, maxDuration, region) Every guard and the helper's branches shown red by reverting them (maintenance window, fail-open, window instant, key prefix, inline path, due) bun run lint, bun run type-check, bun run check:audits, check:test-patterns — green; 26 files / 309 unit tests; 2 integration suites / 40 tests Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing (new tests pass the test-audit authoring gate) No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Open Graph Description: Summary The per-minute outbox and workspace-file-search crons started a Trigger.dev run every tick even with nothing to do. Both now check first, and all three gated scheduled passes share one help...

X Description: Summary The per-minute outbox and workspace-file-search crons started a Trigger.dev run every tick even with nothing to do. Both now check first, and all three gated scheduled passes share one help...

Opengraph URL: https://github.com/simstudioai/sim/pull/8713

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:19adb6ce-7510-4b78-88e0-c58178255d41
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idD06A:34FC0A:E08DD2:132E999:6AFB5DEE
html-safe-nonce2d220ec48ef6341b85bed6ced64c6e5a5311207cebb1735ef80f09f81d5a23de
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMDZBOjM0RkMwQTpFMDhERDI6MTMyRTk5OTo2QUZCNURFRSIsInZpc2l0b3JfaWQiOiI0NTIxMDk3NTAzNjIwMjQ3OCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac4ae84e6b45120c3ccf5f7056c3e77e3621a901eca1ffbb981639e1407cb4cd0d
hovercard-subject-tagpull_request:4766582840
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8713/files
twitter:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
og:image:altSummary The per-minute outbox and workspace-file-search crons started a Trigger.dev run every tick even with nothing to do. Both now check first, and all three gated scheduled passes share one help...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneea33d13c96d0ff66bd8fa906b193d2213536b78cb06b0c50a4632c58b1418f94
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/simstudioai/sim git https://github.com/simstudioai/sim.git
octolytics-dimension-user_id199344406
octolytics-dimension-user_loginsimstudioai
octolytics-dimension-repository_id912559512
octolytics-dimension-repository_nwosimstudioai/sim
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id912559512
octolytics-dimension-repository_network_root_nwosimstudioai/sim
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release59ef97164c52cf59bd10558ddec9c4f482763988
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/simstudioai/sim/pull/8713/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F8713%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F8713%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=simstudioai%2Fsim
Reloadhttps://github.com/simstudioai/sim/pull/8713/files
Reloadhttps://github.com/simstudioai/sim/pull/8713/files
Reloadhttps://github.com/simstudioai/sim/pull/8713/files
simstudioaihttps://github.com/simstudioai
simhttps://github.com/simstudioai/sim
Codehttps://github.com/simstudioai/sim
Issues129 (129)https://github.com/simstudioai/sim/issues
Pull requests197 (197)https://github.com/simstudioai/sim/pulls
Actionshttps://github.com/simstudioai/sim/actions
Projectshttps://github.com/simstudioai/sim/projects
Security and qualityhttps://github.com/simstudioai/sim/security
Insightshttps://github.com/simstudioai/sim/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
waleedlatif1https://github.com/waleedlatif1
staginghttps://github.com/simstudioai/sim/tree/staging
improvement/outbox-work-gatehttps://github.com/simstudioai/sim/tree/improvement/outbox-work-gate
Conversation 36 https://github.com/simstudioai/sim/pull/8713
Commits 7 https://github.com/simstudioai/sim/pull/8713/commits
Checks 32 https://github.com/simstudioai/sim/pull/8713/checks
Files changed https://github.com/simstudioai/sim/pull/8713/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/8713/files
improvement(cron): only start outbox and file-search passes when work is due, via a shared scheduled-pass helper https://github.com/simstudioai/sim/pull/8713/files#top
Show all changes 7 commits https://github.com/simstudioai/sim/pull/8713/files
a9a825d improvement(outbox): only start the processor when work is due or mai… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/a9a825df4e69dcf4cf850c4f61416604c12a1637
349979c test(outbox): assert each backend's own result; describe prune capaci… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/349979c0aebe1074f7962f4182a5f395e11010d9
9c97dc0 improvement(file-search): only start the dispatcher when there is dis… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/9c97dc005939b8c9c32df00ab726988b8ed404b2
154f1c8 refactor(cron): share the scheduled-pass trigger across outbox, file … waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/154f1c856138a9ee6fa9c9238b4fafcc8f7b6724
e7892cd refactor(file-search): key the dispatch window by the instant the gat… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/e7892cd357a8018ab78fcc31e6885e9e2e337597
0509b49 test(cron): assert scheduled-pass outcomes through a deduplicating Tr… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/0509b4923ee10ed8c96ae08f2b1584dbb4b2b645
e448279 test(testing): key the idempotent Trigger fake by an unambiguous task… waleedlatif1 Oct 7, 2026 https://github.com/simstudioai/sim/pull/8713/commits/e448279813ded67194a085131c8cc4600aa5a32c
Clear filters https://github.com/simstudioai/sim/pull/8713/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/8713/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/8713/files
route.ts https://github.com/simstudioai/sim/pull/8713/files#diff-c129a422a957b12f3ef095e8beeeba0ddbf0341433a9a6f81156156fb3a756e5
route.ts https://github.com/simstudioai/sim/pull/8713/files#diff-e6f2f0588c858f23509df693266fb235b84200ce4e45ad4fef722701f9f5e3b0
scheduled-pass.ts https://github.com/simstudioai/sim/pull/8713/files#diff-b887cea743baba6f13a08c10c920a6985f34f62bc2668ff42952308aed6b1985
constants.ts https://github.com/simstudioai/sim/pull/8713/files#diff-f888d5c34fb388434c0367737d53819f72d692916b8ad1cdf16eef6ecc35d1a9
enqueue.test.ts https://github.com/simstudioai/sim/pull/8713/files#diff-458942db90c8f89094dfdd3d7c82402eb3ea5cbb89be1d0cb0b0ace3a536ef2d
enqueue.ts https://github.com/simstudioai/sim/pull/8713/files#diff-3d4cdd80c7336851b9b6f41ad1e49895e969c84ed2ce177cc8e23ad0798ae1d6
queries.ts https://github.com/simstudioai/sim/pull/8713/files#diff-9cd6e0be23d8a86ed6e0043cbb667f8d62d515afc951c7d118af4bcf9aff4773
retention.ts https://github.com/simstudioai/sim/pull/8713/files#diff-219895d477156c3abd1a7c8521715724e5f3aeffa28ebe8608168700a9014502
service.integration.ts https://github.com/simstudioai/sim/pull/8713/files#diff-0e2f541e035fb627f997fa57da9639dcff5ae4c63e6bd6277dec88177525d7a1
service.ts https://github.com/simstudioai/sim/pull/8713/files#diff-c069279adecf7cf40480604ec2439555ad63b23e6def9ee4bbd068348752d984
enqueue.ts https://github.com/simstudioai/sim/pull/8713/files#diff-e91127163cfca95322d776b7fc71b9127c4cc4b1fc9176dcacce39c4b7eec2b6
dispatcher.integration.ts https://github.com/simstudioai/sim/pull/8713/files#diff-5f580fffd39ddd529ae710b571e855f28636a56713b8d9f1889d03127d070e02
dispatcher.ts https://github.com/simstudioai/sim/pull/8713/files#diff-6683656d968a1a183bf327531c469e9b0b8eea626f303f8482a0504340be2d1d
enqueue-dispatch.test.ts https://github.com/simstudioai/sim/pull/8713/files#diff-9599480be410bb7783080766b927ed762b0cb827dd9482c8572445612473529a
enqueue-dispatch.ts https://github.com/simstudioai/sim/pull/8713/files#diff-83c13fccc21c6353fb9f6f20101c7dd4b79926a690e04211f73d58bb75919a26
index-state.ts https://github.com/simstudioai/sim/pull/8713/files#diff-1fc4408f29d8c373126b9cbfd29d6e8e33a65830fa6f497662476876a0c68066
outbox-service.mock.ts https://github.com/simstudioai/sim/pull/8713/files#diff-0739d6b444e7d9eabd9b7a5255e25f7205e8f38b20ce73bc476fb8c4f4ca3529
trigger-sdk.mock.ts https://github.com/simstudioai/sim/pull/8713/files#diff-f70cd816973a843bc378531b04e8233508d4f6317d07754b3dd01c3d9c899403
apps/sim/app/api/cron/workspace-file-search-dispatch/route.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-c129a422a957b12f3ef095e8beeeba0ddbf0341433a9a6f81156156fb3a756e5
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/app/api/cron/workspace-file-search-dispatch/route.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-c129a422a957b12f3ef095e8beeeba0ddbf0341433a9a6f81156156fb3a756e5
https://github.com/simstudioai/sim/pull/8713/files#diff-c129a422a957b12f3ef095e8beeeba0ddbf0341433a9a6f81156156fb3a756e5
apps/sim/app/api/webhooks/outbox/process/route.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-e6f2f0588c858f23509df693266fb235b84200ce4e45ad4fef722701f9f5e3b0
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/app/api/webhooks/outbox/process/route.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-e6f2f0588c858f23509df693266fb235b84200ce4e45ad4fef722701f9f5e3b0
https://github.com/simstudioai/sim/pull/8713/files#diff-e6f2f0588c858f23509df693266fb235b84200ce4e45ad4fef722701f9f5e3b0
apps/sim/lib/core/async-jobs/scheduled-pass.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-b887cea743baba6f13a08c10c920a6985f34f62bc2668ff42952308aed6b1985
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/async-jobs/scheduled-pass.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
apps/sim/lib/core/outbox/constants.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-f888d5c34fb388434c0367737d53819f72d692916b8ad1cdf16eef6ecc35d1a9
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/outbox/constants.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-f888d5c34fb388434c0367737d53819f72d692916b8ad1cdf16eef6ecc35d1a9
apps/sim/lib/core/outbox/enqueue.test.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-458942db90c8f89094dfdd3d7c82402eb3ea5cbb89be1d0cb0b0ace3a536ef2d
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/outbox/enqueue.test.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-458942db90c8f89094dfdd3d7c82402eb3ea5cbb89be1d0cb0b0ace3a536ef2d
apps/sim/lib/core/outbox/enqueue.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-3d4cdd80c7336851b9b6f41ad1e49895e969c84ed2ce177cc8e23ad0798ae1d6
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/outbox/enqueue.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
apps/sim/lib/core/outbox/queries.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-9cd6e0be23d8a86ed6e0043cbb667f8d62d515afc951c7d118af4bcf9aff4773
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/outbox/queries.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-9cd6e0be23d8a86ed6e0043cbb667f8d62d515afc951c7d118af4bcf9aff4773
https://github.com/simstudioai/sim/pull/8713/files#diff-9cd6e0be23d8a86ed6e0043cbb667f8d62d515afc951c7d118af4bcf9aff4773
apps/sim/lib/core/outbox/retention.tshttps://github.com/simstudioai/sim/pull/8713/files#diff-219895d477156c3abd1a7c8521715724e5f3aeffa28ebe8608168700a9014502
View file https://github.com/simstudioai/sim/blob/e448279813ded67194a085131c8cc4600aa5a32c/apps/sim/lib/core/outbox/retention.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/8713/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/8713/files#diff-219895d477156c3abd1a7c8521715724e5f3aeffa28ebe8608168700a9014502
https://github.com/simstudioai/sim/pull/8713/files#diff-219895d477156c3abd1a7c8521715724e5f3aeffa28ebe8608168700a9014502
Please reload this pagehttps://github.com/simstudioai/sim/pull/8713/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/8713/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.