René's URL Explorer Experiment


Title: feat(run-engine): queue gates enforced from the message payload by matt-aitken · Pull Request #4826 · triggerdotdev/trigger.dev · GitHub

Open Graph Title: feat(run-engine): queue gates enforced from the message payload by matt-aitken · Pull Request #4826 · triggerdotdev/trigger.dev

X Title: feat(run-engine): queue gates enforced from the message payload by matt-aitken · Pull Request #4826 · triggerdotdev/trigger.dev

Description: Summary Stacked on #4823. Adds queue gates to the run queue: a run can name up to three additional declared queues (in practice a task's anonymous inline-limit gate plus two named limits) it must also hold a concurrency slot in while it executes. The run waits in its own queue as today; each gate contributes an extra admit condition and an extra slot held until release. This is the engine mechanism behind two long-requested shapes: a per-tenant cap that spans task queues (every run gated on a tenant queue keyed by tenant id, so one tenant's total across all tasks is bounded), and a shared external-resource cap (every run that calls a rate-limited provider gated on one provider queue). Gates ride the message payload as gates: [{ queue, concurrencyKey? }]. At a gate, the gate queue's concurrencyLimit applies per key value and its totalConcurrencyLimit bounds the sum across keys, exactly the same semantics a run meets in its own queue. Enforcement is gated behind RUN_ENGINE_QUEUE_GATES_ENABLED (default off). The SDK and trigger API surface that populates gates follows in a PR stacked on this one; nothing produces gated payloads yet. Design Admit: both dequeue scripts already decode each message's payload, so the gate check reads it directly and skips a message whose gates are full (it stays queued in place). The enqueue fast paths probe the raw payload for "gates" before decoding, so ungated messages pay a substring scan and nothing more. On admit the run is added to each gate's per-key concurrency set, and to the gate's group set when the entry is keyed, reusing the exact structures the base queue and total cap already use. Gate keys are built in Lua from the payload's own org/project/environment, so they always live in the message's hash slot. Release: every release path (ack, nack, dead-letter, concurrency release, sweeper clear, TTL expiry, re-enqueue cleanup) removes the run from its gate sets, driven by the payload rather than the flag, so slots acquired while the flag was on always drain after it is turned off. A saturated gate self-heals with the same bounded cursor reconciliation as the total cap in #4823: a member whose message key no longer exists was terminally released by a path without the removal and is pruned, one bounded batch per interval. Head-of-line note: a message whose gate is full blocks the messages behind it in the same (sub-)queue. That is the intended trade and the contract PR enforces the rule that makes it benign: a gate's key must be a function of the run's own concurrency key, or a constant. Covered by tests for unkeyed and keyed gates, flag-off behavior, nack release, fast-path fallback, and leaked-member reconciliation, plus the existing run-queue suite (129 tests).

Open Graph Description: Summary Stacked on #4823. Adds queue gates to the run queue: a run can name up to three additional declared queues (in practice a task's anonymous inline-limit gate plus two named limits) it mu...

X Description: Summary Stacked on #4823. Adds queue gates to the run queue: a run can name up to three additional declared queues (in practice a task's anonymous inline-limit gate plus two named limits) i...

Opengraph URL: https://github.com/triggerdotdev/trigger.dev/pull/4826

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:feb17375-69ae-5c61-a732-c93ce40b739f
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idEDB2:27EABE:6307ADB:84DDD8C:6AFAF713
html-safe-nonce77a2697bb608c2ae630ee28158377058186d472fbe22221e07fb835accc96bf3
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFREIyOjI3RUFCRTo2MzA3QURCOjg0REREOEM6NkFGQUY3MTMiLCJ2aXNpdG9yX2lkIjoiODM1MjU2MjAxMjAyMTc4MjI5MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacb6f190424281ca9e99133ddeb6cd390d190cbdd4b895a696ca1da89c4394fc81
hovercard-subject-tagpull_request:4390835235
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/triggerdotdev/trigger.dev/pull/4826/files
twitter:imagehttps://avatars.githubusercontent.com/u/10635986?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/10635986?s=400&v=4
og:image:altSummary Stacked on #4823. Adds queue gates to the run queue: a run can name up to three additional declared queues (in practice a task's anonymous inline-limit gate plus two named limits) it mu...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None0178036d8230524a18422872393ab7e708443ae4c2047f758560a2db870c1913
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/triggerdotdev/trigger.dev git https://github.com/triggerdotdev/trigger.dev.git
octolytics-dimension-user_id95297378
octolytics-dimension-user_logintriggerdotdev
octolytics-dimension-repository_id572570113
octolytics-dimension-repository_nwotriggerdotdev/trigger.dev
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id572570113
octolytics-dimension-repository_network_root_nwotriggerdotdev/trigger.dev
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasea85e96a83bed7871993e6b1ffda24105928eee17
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/triggerdotdev/trigger.dev/pull/4826/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ftriggerdotdev%2Ftrigger.dev%2Fpull%2F4826%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ftriggerdotdev%2Ftrigger.dev%2Fpull%2F4826%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=triggerdotdev%2Ftrigger.dev
Reloadhttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
Reloadhttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
Reloadhttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
triggerdotdevhttps://github.com/triggerdotdev
trigger.devhttps://github.com/triggerdotdev/trigger.dev
Codehttps://github.com/triggerdotdev/trigger.dev
Issues203 (203)https://github.com/triggerdotdev/trigger.dev/issues
Pull requests67 (67)https://github.com/triggerdotdev/trigger.dev/pulls
Discussionshttps://github.com/triggerdotdev/trigger.dev/discussions
Actionshttps://github.com/triggerdotdev/trigger.dev/actions
Projectshttps://github.com/triggerdotdev/trigger.dev/projects
Security and quality39 (39)https://github.com/triggerdotdev/trigger.dev/security
Insightshttps://github.com/triggerdotdev/trigger.dev/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Ftriggerdotdev%2Ftrigger.dev%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Ftriggerdotdev%2Ftrigger.dev%2Fissues%2Fnew%2Fchoose
matt-aitkenhttps://github.com/matt-aitken
feat/queue-total-concurrency-limithttps://github.com/triggerdotdev/trigger.dev/tree/feat/queue-total-concurrency-limit
feat/queue-gates-enginehttps://github.com/triggerdotdev/trigger.dev/tree/feat/queue-gates-engine
Conversation 54 https://github.com/triggerdotdev/trigger.dev/pull/4826
Commits 8 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits
Checks 60 https://github.com/triggerdotdev/trigger.dev/pull/4826/checks
Files changed https://github.com/triggerdotdev/trigger.dev/pull/4826/files
Please reload this pagehttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
feat(run-engine): queue gates enforced from the message payload https://github.com/triggerdotdev/trigger.dev/pull/4826/files#top
Show all changes 8 commits https://github.com/triggerdotdev/trigger.dev/pull/4826/files
906cda7 feat(run-engine): queue gates enforced from the message payload matt-aitken Aug 29, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/906cda7a27eb49109c921b5f027333aca727ea68
b899541 fix(run-engine): gates inherit the run key and never deadlock on thei… matt-aitken Aug 29, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/b899541fe6fd1da9d24545ef2895ab24c2a798b5
2b85daf fix(run-engine): reconcile prunes members whose runs are queued matt-aitken Aug 29, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/2b85daf356fc93fd0d26f5e12adf5a5ae282c627
b568120 fix(run-engine): reconcile group and gate members against the home co… matt-aitken Aug 31, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/b5681203c23961f6c038f24aeb64acedda225a0b
916c6b6 fix(run-engine): a gate's total limit counts keyed and keyless holder… matt-aitken Sep 6, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/916c6b69428345dd63e7f4a73fad274c181bab49
f8bb7b0 fix(run-engine): queue payloads accept three gates matt-aitken Sep 6, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/f8bb7b0c6c794e518585c29b725dd35f9688f7af
c372d2b docs(run-engine): the gate cap is three, matching the schema matt-aitken Sep 6, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/c372d2b142a20e519da94fdf4228024c8d5e5192
8d9cc1e fix(run-engine): queue payloads accept four gates matt-aitken Sep 6, 2026 https://github.com/triggerdotdev/trigger.dev/pull/4826/commits/8d9cc1eac9cc9c0a82e9cc3bf9b5519174f8076c
Clear filters https://github.com/triggerdotdev/trigger.dev/pull/4826/files
Please reload this pagehttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
Please reload this pagehttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
env.server.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f8823615ef72da3494e74de9e62b2fa31315ed9360f17bc06ab943f4dc25f4fd
runEngine.server.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-44b2620e883d51908283afcb71504e2318df61525d0fbbf15b60d18b400a8c59
index.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-d29236a78d70dd8a019a2ec308d8eab0eda9f5fbf116d19d5c99b8f573a1b5df
types.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-81c21d7967ac61777b1a20d53a0f821068d182af9f44ec4bd64b967c8fbd6d92
index.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-0365b31224e46e323c8868fee705d14146349e29abd175070c7e7a49e9cb73ae
queueGates.test.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f795b9483460c55cdc5ba568705f74381367df77a0f3056265dcfda3573a4663
totalConcurrency.test.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f9c853f3fb4c31c3bc65a9b93412a83036ada9b63ac7f8a9c38c808e86736e28
types.ts https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-cbcd44d22184883324208ec29b1591dfe40d0f057b3ef9ac4430866c4fc1143d
apps/webapp/app/env.server.tshttps://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f8823615ef72da3494e74de9e62b2fa31315ed9360f17bc06ab943f4dc25f4fd
View file https://github.com/triggerdotdev/trigger.dev/blob/8d9cc1eac9cc9c0a82e9cc3bf9b5519174f8076c/apps/webapp/app/env.server.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/triggerdotdev/trigger.dev/pull/4826/{{ revealButtonHref }}
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f8823615ef72da3494e74de9e62b2fa31315ed9360f17bc06ab943f4dc25f4fd
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-f8823615ef72da3494e74de9e62b2fa31315ed9360f17bc06ab943f4dc25f4fd
apps/webapp/app/v3/runEngine.server.tshttps://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-44b2620e883d51908283afcb71504e2318df61525d0fbbf15b60d18b400a8c59
View file https://github.com/triggerdotdev/trigger.dev/blob/8d9cc1eac9cc9c0a82e9cc3bf9b5519174f8076c/apps/webapp/app/v3/runEngine.server.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/triggerdotdev/trigger.dev/pull/4826/{{ revealButtonHref }}
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-44b2620e883d51908283afcb71504e2318df61525d0fbbf15b60d18b400a8c59
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-44b2620e883d51908283afcb71504e2318df61525d0fbbf15b60d18b400a8c59
internal-packages/run-engine/src/engine/index.tshttps://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-d29236a78d70dd8a019a2ec308d8eab0eda9f5fbf116d19d5c99b8f573a1b5df
View file https://github.com/triggerdotdev/trigger.dev/blob/8d9cc1eac9cc9c0a82e9cc3bf9b5519174f8076c/internal-packages/run-engine/src/engine/index.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/triggerdotdev/trigger.dev/pull/4826/{{ revealButtonHref }}
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-d29236a78d70dd8a019a2ec308d8eab0eda9f5fbf116d19d5c99b8f573a1b5df
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-d29236a78d70dd8a019a2ec308d8eab0eda9f5fbf116d19d5c99b8f573a1b5df
internal-packages/run-engine/src/engine/types.tshttps://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-81c21d7967ac61777b1a20d53a0f821068d182af9f44ec4bd64b967c8fbd6d92
View file https://github.com/triggerdotdev/trigger.dev/blob/8d9cc1eac9cc9c0a82e9cc3bf9b5519174f8076c/internal-packages/run-engine/src/engine/types.ts
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/triggerdotdev/trigger.dev/pull/4826/{{ revealButtonHref }}
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-81c21d7967ac61777b1a20d53a0f821068d182af9f44ec4bd64b967c8fbd6d92
https://github.com/triggerdotdev/trigger.dev/pull/4826/files#diff-81c21d7967ac61777b1a20d53a0f821068d182af9f44ec4bd64b967c8fbd6d92
Please reload this pagehttps://github.com/triggerdotdev/trigger.dev/pull/4826/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.