Title: fix(tables): drop the per-table row-order lock from inserts by TheodoreSpeaks · Pull Request #8811 · simstudioai/sim · GitHub
Open Graph Title: fix(tables): drop the per-table row-order lock from inserts by TheodoreSpeaks · Pull Request #8811 · simstudioai/sim
X Title: fix(tables): drop the per-table row-order lock from inserts by TheodoreSpeaks · Pull Request #8811 · simstudioai/sim
Description: Summary Inserts no longer take the per-table row-order advisory lock (user_table_rows_pos). It was held until commit, so one insert stuck waiting on synchronous replication made every other insert to that table wait behind it and fail at the 3 s lock_timeout. With #8765, a row write no longer holds any lock shared by the whole table. Append keys: appendKeys mints inside the integer range after the last key, narrowed to a random sub-range (32 random halvings). Two appends that read the same last key never collide, a batch stays contiguous instead of interleaving with another writer's, and keys don't grow across appends (8–9 characters after 2,000 sequential appends). Positions stay max + 1, read without a lock, so concurrent appends can share one. The run dispatcher pages on (position, id). When a window is full, it also takes the remaining rows at the last position, so a tie split across windows is never skipped. No schema or API change: the cursor is still the last position processed. Positional inserts take the next key strictly above the lower neighbour, the same rule the neighbour-anchored path already used, so a tied key no longer throws a >= b. A position past the last row now appends; it used to mint a0 and land the row near the top. Upsert: the re-check behind the order lock is gone. The conflict target is always a unique column, and lockUniqueValues already serializes same-value upserts before the lookup. Replace still excludes a concurrent replace through lockUniqueColumns, which every replace takes exclusively, even on tables with no unique columns. An insert that commits during a replace survives it, the same outcome as running right after it. Not changed: the order_key backfill script migration and the collation repair script still take the old lock. Neither concurrent appends nor these scripts are protected by it any more; the scripts only rewrite tables without valid keys. Benchmark Local Postgres, 48 concurrent writers on one table through the real row services, 10 s per scenario, with a 20 ms commit delay per transaction to stand in for the replication wait (locks still held). Successful ops/s, p99, failures: Scenario Op staging + #8765 + #8765 + this Inserts only insert 17/s · 3115 ms · 13 timeouts 22/s · 2919 ms · 0 1510/s · 80 ms · 0 Mixed insert 2/s · 4856 ms · 39/62 timed out 13/s · 1881 ms · 0 404/s · 84 ms · 0 Mixed + one insert stalled 4 s insert 1/s · 5901 ms · 46/63 timed out 7/s · 3039 ms · 16/86 timed out 461/s · 81 ms · 0 Mixed + one insert stalled 4 s update 7/s · 8011 ms · 0 92/s · 672 ms · 0 384/s · 118 ms · 0 Live row count matched COUNT(*) in every run. Absolute numbers vary between runs; read the columns relative to each other. Type of Change Bug fix Testing New integration tests, each red on the pre-change code: Every insert path (append, at a position, after a row, batch, upsert, import with a new column) commits while another transaction holds the retired lock. Before: each waits out the 3 s lock timeout. 4 batches and 4 single inserts that all read the same last key get distinct keys, and each batch stays contiguous. Red with deterministic append keys: 25 rows, 6 distinct keys. Insert at a position between two rows sharing a key. Before: throws. Insert at a position past the last row lands last. Before: lands near the top. Dispatcher (dispatcher.integration.ts): rows that share positions across a window boundary each run exactly once. Before: tied rows skipped. Two concurrent replaces leave exactly one row set (guards the unique-lock serialization, green before and after). The unique-value race tests used the order lock to pause inserts mid-flight. They now pause on a definition row held FOR UPDATE, which an insert's foreign-key check waits on after its unique check. The obsolete lock-order unit tests are removed. Table + TTL integration suites: migrated 161 passed, push-provisioned 131 passed. Table unit tests 987 passed. lint:check and check:audits pass. Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA) 🤖 Generated with Claude Code https://claude.ai/code/session_01CJxC2fgucGVpT59RV5Z7xv
Open Graph Description: Summary Inserts no longer take the per-table row-order advisory lock (user_table_rows_pos). It was held until commit, so one insert stuck waiting on synchronous replication made every other insert...
X Description: Summary Inserts no longer take the per-table row-order advisory lock (user_table_rows_pos). It was held until commit, so one insert stuck waiting on synchronous replication made every other insert...
Opengraph URL: https://github.com/simstudioai/sim/pull/8811
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:790312ad-6273-45e5-4e5e-55375ab92916 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 9DFC:3988A1:E087B2:1341850:6AFB5EB6 |
| html-safe-nonce | bb9ddd9ecff7db5951c366700b4f681100431654ee099ab490faded9b85374d4 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5REZDOjM5ODhBMTpFMDg3QjI6MTM0MTg1MDo2QUZCNUVCNiIsInZpc2l0b3JfaWQiOiI1NjkyNTMxNjEyNzU1NjQ4MTgyIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | c72b355ccf6fab8f5d08207f8950af7f1a246e13b66fca66d57bcb6d0832dbac |
| hovercard-subject-tag | pull_request:4792506238 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8811/files |
| twitter:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| og:image:alt | Summary Inserts no longer take the per-table row-order advisory lock (user_table_rows_pos). It was held until commit, so one insert stuck waiting on synchronous replication made every other insert... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | ea33d13c96d0ff66bd8fa906b193d2213536b78cb06b0c50a4632c58b1418f94 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 59ef97164c52cf59bd10558ddec9c4f482763988 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width