René's URL Explorer Experiment


Title: RFC: SQL-Based Distributed Queue - SubmitQueue

Description: A speculative submission queue that keeps trunk green at scale.

Mail addresses
preetam@uber.com

Generator: mkdocs-1.6.1, mkdocs-material-9.7.7

direct link

Domain: uber.github.io

Links:

Skip to content https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#rfc-sql-based-distributed-queue
https://uber.github.io/submitqueue/
uber/submitqueue https://github.com/uber/submitqueue
Home https://uber.github.io/submitqueue/
Guides https://uber.github.io/submitqueue/howto/QUICKSTART/
Design (RFCs) https://uber.github.io/submitqueue/rfc/
https://uber.github.io/submitqueue/
uber/submitqueue https://github.com/uber/submitqueue
Home https://uber.github.io/submitqueue/
Quickstart https://uber.github.io/submitqueue/howto/QUICKSTART/
Development https://uber.github.io/submitqueue/howto/DEVELOPMENT/
Testing https://uber.github.io/submitqueue/howto/TESTING/
Design (RFCs) https://uber.github.io/submitqueue/rfc/
Change URIs https://uber.github.io/submitqueue/rfc/change-uri/
Consumer Gate https://uber.github.io/submitqueue/rfc/consumer-gate/
Consumer Hold https://uber.github.io/submitqueue/rfc/consumer-hold/
Hooks Framework https://uber.github.io/submitqueue/rfc/hook-framework/
Message Queue Contract https://uber.github.io/submitqueue/rfc/messagequeue-contract/
RFC: Per-Tenant Sharding for the MySQL Message Queue https://uber.github.io/submitqueue/rfc/messagequeue-tenant-sharding/
Service-Scoped Extensions https://uber.github.io/submitqueue/rfc/service-scoped-extensions/
RFC: SQL-Based Distributed Queue https://uber.github.io/submitqueue/rfc/sql-queue-rfc/
Metadata https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#metadata
Summary https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#summary
Background https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#background
Motivation https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#motivation
Existing Solutions https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#existing-solutions
Decision https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#decision
Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#requirements
Functional Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#functional-requirements
Non-Functional Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-functional-requirements
Non-Goals https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-goals
Design Overview https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#design-overview
High-Level Architecture https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#high-level-architecture
Core Concepts https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#core-concepts
Database Schema https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#database-schema
Messages Table (Immutable Log) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#messages-table-immutable-log
Delivery State Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#delivery-state-table
Partition Leases Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#partition-leases-table
Consumer Offsets Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-offsets-table
Subscriber Heartbeats Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#subscriber-heartbeats-table
Dead Letter Queue https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#dead-letter-queue
Message Flow https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-flow
Consumer Group Isolation https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-group-isolation
Ordering and Serialization https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#ordering-and-serialization
Default: Concurrent Delivery Within Partition https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#default-concurrent-delivery-within-partition
Non-Blocking Nack https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-blocking-nack
Strict Serialization (Opt-In) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#strict-serialization-opt-in
Watermark Advancement https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermark-advancement
Message Durability https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-durability
Crash Recovery https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#crash-recovery
Distributed Processing https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#distributed-processing
Fair Partition Distribution https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#fair-partition-distribution
Alternatives Considered https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#alternatives-considered
Watermill Library https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermill-library
Single-Table Per Topic https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#single-table-per-topic
Mutable Message State (Previous Design) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#mutable-message-state-previous-design
Trade-offs https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#trade-offs
Appendix https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#appendix
References https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#references
Web UI as a Mountable Module https://uber.github.io/submitqueue/rfc/web-ui/
Phabricator Support in Runway https://uber.github.io/submitqueue/rfc/runway/phabricator-support/
Runway Workflow https://uber.github.io/submitqueue/rfc/runway/workflow/
Stovepipe List API https://uber.github.io/submitqueue/rfc/stovepipe/list-api/
Open issues API https://uber.github.io/submitqueue/rfc/stovepipe/open-issues-api/
Request issue/project attribution API https://uber.github.io/submitqueue/rfc/stovepipe/project-issues-api/
Stovepipe Project Status APIs https://uber.github.io/submitqueue/rfc/stovepipe/project-status-api/
Queue enablement and status https://uber.github.io/submitqueue/rfc/stovepipe/queue-state/
Stovepipe Request History API https://uber.github.io/submitqueue/rfc/stovepipe/request-history-api/
Request issues API https://uber.github.io/submitqueue/rfc/stovepipe/request-issues-api/
Stovepipe Request Log https://uber.github.io/submitqueue/rfc/stovepipe/request-log/
Stovepipe Workflow https://uber.github.io/submitqueue/rfc/stovepipe/workflow/
Build stage https://uber.github.io/submitqueue/rfc/stovepipe/steps/build/
Build issue detection https://uber.github.io/submitqueue/rfc/stovepipe/steps/buildanalysis/
Buildsignal stage https://uber.github.io/submitqueue/rfc/stovepipe/steps/buildsignal/
Process stage https://uber.github.io/submitqueue/rfc/stovepipe/steps/process/
Record stage https://uber.github.io/submitqueue/rfc/stovepipe/steps/record/
Build Runner https://uber.github.io/submitqueue/rfc/submitqueue/build-runner/
Extension Contract https://uber.github.io/submitqueue/rfc/submitqueue/extension-contract/
Gateway Request History APIs https://uber.github.io/submitqueue/rfc/submitqueue/history-api/
Modular Queue Wiring https://uber.github.io/submitqueue/rfc/submitqueue/modular-queue-wiring/
Outcome Scorer https://uber.github.io/submitqueue/rfc/submitqueue/outcome-scorer/
Pausing and Unpausing a Queue https://uber.github.io/submitqueue/rfc/submitqueue/queue-pause-and-unpause/
Best-First Speculation Path Generation https://uber.github.io/submitqueue/rfc/submitqueue/speculation-generator-best-first/
Speculation https://uber.github.io/submitqueue/rfc/submitqueue/speculation/
Gateway Request Summary and List APIs https://uber.github.io/submitqueue/rfc/submitqueue/status-list-api/
Tango Conflict Analyzer https://uber.github.io/submitqueue/rfc/submitqueue/tango-conflict-analyzer/
Orchestrator Workflow https://uber.github.io/submitqueue/rfc/submitqueue/workflow/
Metadata https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#metadata
Summary https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#summary
Background https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#background
Motivation https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#motivation
Existing Solutions https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#existing-solutions
Decision https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#decision
Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#requirements
Functional Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#functional-requirements
Non-Functional Requirements https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-functional-requirements
Non-Goals https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-goals
Design Overview https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#design-overview
High-Level Architecture https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#high-level-architecture
Core Concepts https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#core-concepts
Database Schema https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#database-schema
Messages Table (Immutable Log) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#messages-table-immutable-log
Delivery State Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#delivery-state-table
Partition Leases Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#partition-leases-table
Consumer Offsets Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-offsets-table
Subscriber Heartbeats Table https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#subscriber-heartbeats-table
Dead Letter Queue https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#dead-letter-queue
Message Flow https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-flow
Consumer Group Isolation https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-group-isolation
Ordering and Serialization https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#ordering-and-serialization
Default: Concurrent Delivery Within Partition https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#default-concurrent-delivery-within-partition
Non-Blocking Nack https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-blocking-nack
Strict Serialization (Opt-In) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#strict-serialization-opt-in
Watermark Advancement https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermark-advancement
Message Durability https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-durability
Crash Recovery https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#crash-recovery
Distributed Processing https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#distributed-processing
Fair Partition Distribution https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#fair-partition-distribution
Alternatives Considered https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#alternatives-considered
Watermill Library https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermill-library
Single-Table Per Topic https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#single-table-per-topic
Mutable Message State (Previous Design) https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#mutable-message-state-previous-design
Trade-offs https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#trade-offs
Appendix https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#appendix
References https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#references
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#rfc-sql-based-distributed-queue
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#metadata
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#summary
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#background
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#motivation
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#existing-solutions
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#decision
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#requirements
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#functional-requirements
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-functional-requirements
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-goals
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#design-overview
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#high-level-architecture
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#core-concepts
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#database-schema
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#messages-table-immutable-log
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#delivery-state-table
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#partition-leases-table
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-offsets-table
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#subscriber-heartbeats-table
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#dead-letter-queue
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-flow
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#consumer-group-isolation
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#ordering-and-serialization
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#default-concurrent-delivery-within-partition
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#non-blocking-nack
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#strict-serialization-opt-in
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermark-advancement
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#message-durability
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#crash-recovery
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#distributed-processing
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#fair-partition-distribution
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#alternatives-considered
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#watermill-library
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#single-table-per-topic
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#mutable-message-state-previous-design
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#trade-offs
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#appendix
https://uber.github.io/submitqueue/rfc/sql-queue-rfc/#references
MySQL InnoDB Lockinghttps://dev.mysql.com/doc/refman/8.0/en/innodb-locking.html
MySQL AUTO_INCREMENThttps://dev.mysql.com/doc/refman/8.0/en/example-auto-increment.html
Amazon SQS Visibility Timeouthttps://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
Kafka Consumer Groupshttps://kafka.apache.org/documentation/#consumergroups
RabbitMQ Dead Letter Exchangeshttps://www.rabbitmq.com/dlx.html
Watermill Documentationhttps://watermill.io/
PostgreSQL SKIP LOCKEDhttps://www.postgresql.org/docs/current/sql-select.html#SQL-FOR-UPDATE-SHARE
Previous Service-Scoped Extensions https://uber.github.io/submitqueue/rfc/service-scoped-extensions/
Next Web UI as a Mountable Module https://uber.github.io/submitqueue/rfc/web-ui/
Material for MkDocs https://squidfunk.github.io/mkdocs-material/
https://github.com/uber/submitqueue
https://join.slack.com/t/submitqueue/shared_invite/zt-46gkqj682-7zcQphxm2pYqkjDo9lbmYA

Viewport: width=device-width,initial-scale=1


URLs of crawlers that visited me.