Title: fix(files): stop the collaborative editor rewriting and reflowing a document on open by icecrasher321 · Pull Request #6652 · simstudioai/sim · GitHub
Open Graph Title: fix(files): stop the collaborative editor rewriting and reflowing a document on open by icecrasher321 · Pull Request #6652 · simstudioai/sim
X Title: fix(files): stop the collaborative editor rewriting and reflowing a document on open by icecrasher321 · Pull Request #6652 · simstudioai/sim
Description: Opening a file rewrote it, and the pane visibly re-laid-out a beat after it painted. Both came from the live document drifting out of the shape its own markdown can describe. What was happening ProseMirror appends an empty paragraph to any document that does not end in one. The server seeded the CRDT with the raw parse, so the first client to bind wrote that paragraph back into the shared document — which the relay saw as a real edit and persisted. Every open therefore: uploaded the file under a fresh storage key and deleted the old one, 404ing the page's own in-flight content read (FileNotFoundError on a file you are looking at), bumped contentUpdatedAt, so "Last Updated" moved just from viewing a file, churned a blob write + delete + DB transaction per open. And because a trailing blank line cannot serialize (postProcessSerializedMarkdown collapses it), the file never recorded that paragraph, so nothing ever reconciled the two — each client that seeded without seeing another's contribution stacked one more. Measured on a real document: 18 stacked empty paragraphs in the live doc against the placeholder's 1, i.e. the pane growing several hundred pixels the instant the live editor took over. Changes Seed and merge through the editor's own normal form (editorNormalForm in collab-doc/converter.ts), so binding is a no-op and canonicalizeYDoc — which is parse ∘ serialize — collapses an accumulated run back to one. Placed at the collab boundary rather than in parseMarkdownToDoc: only the CRDT has to agree with the editor. Every other consumer of the parse (paste, the round-trip probe, the read-only placeholder, note blocks, skills) renders through a real editor that normalizes itself — putting it in the parse broke 17 tests on exactly those surfaces. Skip a persist whose projection already matches the durable bytes. Byte length is the free reject, so the compare read only happens when a no-op write is actually on the table. On a no-op it reports the file's current version, resyncing a stale If-Match instead of conflicting. Revoke collaborative readiness on a fatal join. The sticky syncedOnce latch outlived the document: after a readiness timeout the provider drops synced so the gate closes, but the latch re-opened it on the offline fallback's seed flag — handing back an editable editor on a document the provider had abandoned, with client autosave gated off because collaboration is nominally on. Keystrokes went nowhere and vanished on reload, with no error shown. Recover from a superseded storage key. A 404 from a content read means the pointer is stale, not that the file is gone: re-resolve the record so the read re-keys onto the current object. Invalidates the record, never the failed query, which is what keeps it loop-proof. And don't focus-refetch durable bytes while the relay owns durability. Prefetch the workspace file list in the layout. The sidebar reads that query on every workspace route, so it registers before any page renders; HydrationBoundary hands an already-seen query to a useEffect that SSR never runs, so a page-level prefetch of that key could not reach the server render. The file route rendered a spinner and disagreed with the client about the header's markup — a hydration mismatch. Verified: the SSR probe went from filesLen:0, hasSelected:false to filesLen:19, hasSelected:true, and the served HTML now carries the real breadcrumb trail instead of the … placeholder. Load the document font with display: block. A swap repaints prose in metric-adjusted Arial first, so paragraphs re-wrap when the real face lands — visible on every hard refresh, since that bypasses the font cache. Also: a detail-route loading.tsx (the segment was inheriting the list chrome — an options bar and table header a document page doesn't have), normalize.ts renamed to field.ts now that it holds only the field constant, and the duplicate COLLAB_DOC_FIELD in the streaming path folded into it. Verification Measured in a headless browser against real documents, not just unit tests: placeholder vs live editor on a cold room: 18 blocks vs 18, zero differing positions, no geometry change at the swap (previously 52 vs 69). a warm, edited room joined by a second client: identical again — the accumulation path is closed, not just healed. POST /api/internal/file-doc/seed unaffected at ~25ms. New regression tests: 6 "binding an editor to the seed changes nothing" cases (list/heading/table/rule/paragraph/blank-line endings), 6 no-op-persist cases including the fail-open paths, 4 stale-storage-key cases, and 2 readiness cases that both return ready: true under the old formula. The parity helper now compares against the shared normal form — it had been asserting a shape neither side renders. markdown-parse.test.ts's two 400-seed property tests were re-budgeted from 30s to 60s: this branch adds the second one, and at 30s both timed out under whole-suite parallelism while passing standalone. Full apps/sim suite: 1825 files / 23948 tests, 0 failures. tsc, api-validation, react-query, client-boundary, utils, import-specifiers all clean. Reveal the live document only once it has settled collabReady flipped the moment the provider reported synced, but a room's remaining updates can still be in flight — reconnecting to a room edited moments ago, the handshake lands on the base state and the edit arrives milliseconds later. The live editor was therefore un-hidden onto an intermediate CRDT state and corrected itself in view. Measured on a reload right after a Mod-Shift-ArrowDown block move: t=1484ms editor#1 (placeholder) T|BBB|AAA|CCC ← correct the whole time t=1570ms editor#2 (live) REVEALED T|AAA|BBB|CCC ← pre-move state t=1577ms editor#2 (live) T|BBB|AAA|CCC ← corrects 7ms later The placeholder was right; the swap was the only reason anything moved. Readiness now waits for one animation frame with no update to the shared document, and any update restarts the wait. Going NOT-ready stays immediate, so nothing keeps a fatal or unsynced document editable a moment longer than before. After the fix the same repro absorbs the stale state while still hidden: t=2154ms editor#2 hidden=true (empty) t=2246ms editor#2 hidden=true T|AAA|BBB|CCC ← stale, but not on screen t=2256ms editor#2 hidden=false T|BBB|AAA|CCC ← revealed already correct Cost is at most one frame of a placeholder that is already showing the correct content. Ruled out along the way, by measurement rather than reasoning: the move does reach the durable markdown (within 6s, and the typing control too), so this was never a lost write. Unrelated flake seen while validating lib/workflows/diff/diff-engine.test.ts times out at 10s under whole-suite parallelism (2.4s standalone) in roughly two of three full runs. Untouched by this branch — flagging because CI may hit it. Second round: serve a whole document on join, and stop persistence deadlocking The reflow survived the fixes above, and driving a real browser against the running app found two more causes — one in the relay, one in persistence — plus a client recovery that was never running. Opening a file replayed it A room rebuilds itself from the file's Redis stream one entry at a time, into the same Y.Doc that fans every update out to its room — and the join attached the socket before that finished, and before the server seed. A client was therefore never sent the document; it was sent the document's history, and it watched that replay. Reload right after moving a block and the block moved again in front of you. file-doc.join-readiness.test.ts reproduces it against the real store: with the fix reverted it reports ['AAA', 'BBBAAA'] where one state was due. The join now assembles the room before attaching a client to it — it awaits the stream catch-up and the seed, so the client's first sync is the finished document, in one message. The seed is memoized on the room so concurrent joins wait for the same one rather than the second being served an empty doc, and a task that loses the seed lock pulls the winner's seed out of the stream instead of waiting for the tailer to push it. That last part is also the "a freshly uploaded file is never editable" bug: it was waiting on a delivery that never came, until the client's readiness deadline lapsed into read-only. The client-side quiet-frame gate that had been standing in for this is deleted. It was unsound in both directions — it delayed a document that was already correct, and it opened mid-flight anyway whenever updates arrived more than a frame apart, which is exactly what a cross-region Redis and a long room history produce. Persistence had deadlocked, so the placeholder was permanently stale The If-Match token is a remembered timestamp: held in the room, which dies with it, and in a cluster key written fire-and-forget. A relay that exits in the moments after a successful write comes back holding a version older than the file's; every persist then fails the CAS; and because a conflict neither writes nor advances the token, that document can never be persisted again. Measured on a live file: value cluster If-Match token 1786594348911 file's content_updated_at 1786594418409 unpersisted entries in the stream 103 The durable markdown froze there — and the editor's placeholder is built from it, so every reload painted the pre-edit document and the live one corrected it on screen. That is what still looked like a "replay" after the relay fix. The cluster version write after a successful persist is now awaited. It is the only record that survives a teardown. On a conflict, ask the content, not the clock: if the file still holds the bytes this document last projected (the tag written with every persist), nothing out-of-band exists to protect, so re-sync the token and write. Any other bytes and the conflict stands exactly as before. The stale-key recovery was never running useStaleKeyRecovery was requested from inside the failing read's own queryFn — where react-query drops it. Instrumented: the recovery fired, and no fetch happened. So nothing re-resolved the record and the reader sat on a dead key showing "Failed to load file content" until something unrelated refetched. It now runs off that cycle and cancels a record read already in flight (which could only hand back the key we know is dead), and while the record is re-resolving the surface reports loading, not failure. One document per file, for its whole life A document rebuilt from markdown is a different document to Yjs — its items carry new client ids — so a client still holding the old one merges the file into itself, twice, on both sides, and the relay persists that. Confirmed through the real converter. The rebuild happened whenever the stored binary was absent (a file opened but never edited had no stored document, so every cold open minted a new one) or stale (any external write). The seed now stores what it builds and resumes it with a CRDT diff when the markdown moved on out-of-band, so there is only ever one. As a backstop the document carries an identity the join ack names, and a client offered a different one refuses to sync instead of merging — read-only until a reload, never corruption. Embedded images The inline route sent no-cache with no validator, so every open re-downloaded the whole image — ~1 MB per open of a real document, with the image area blank until it landed. A ?key= names one storage object and a content write never rewrites one (updateWorkspaceFileContent always mints a fresh key), so those bytes are immutable; ?fileId= names the file, whose bytes move under it, so that form still revalidates. Verified in a browser: inline FROM CACHE on repeat opens, previously 1012 KB every time. Verification Relay: 277 tests, including a new store-enabled join suite that fails without the fix. App: 1034 tests across collab-doc, workspace-files, copilot/tools/server/files, the file-viewer, the serve/inline routes and the query hooks. In a browser, against the running app: edit-then-reload with zero delay now shows placeholder and live editor identical (T|BBB|AAA|CCC both) where it previously revealed the pre-move order; a plain open no longer rewrites the file (byte-identical, key unchanged). tsc, check:api-validation, check:react-query, check:client-boundary, check:import-specifiers, biome — all clean. Known residual The first view of an image the browser has never fetched still reflows once: its box is reserved from stored dimensions, and those are written the first time the browser measures it. Fixing that means carrying the intrinsic size on the image node so the very first paint reserves the box — a schema change, deliberately not folded in here.
Open Graph Description: Opening a file rewrote it, and the pane visibly re-laid-out a beat after it painted. Both came from the live document drifting out of the shape its own markdown can describe. What was happening Pro...
X Description: Opening a file rewrote it, and the pane visibly re-laid-out a beat after it painted. Both came from the live document drifting out of the shape its own markdown can describe. What was happening Pro...
Opengraph URL: https://github.com/simstudioai/sim/pull/6652
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c061d734-d859-862f-206b-2e1465809e59 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | BE4E:303206:97733D:D5EA9E:6AFB51B5 |
| html-safe-nonce | 9f29c86728142140dd8ac090b7dbca01f81950964d509f0413eefb58f0475e7e |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCRTRFOjMwMzIwNjo5NzczM0Q6RDVFQTlFOjZBRkI1MUI1IiwidmlzaXRvcl9pZCI6IjgwNjg4Njg0Njk5NTQ0MDA2OTMiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 33ad5aa4a3f959dbb62f811c16e1b442e090ed17ee3ec6d935e12f39ae9090be |
| hovercard-subject-tag | pull_request:4267411918 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/6652/files |
| twitter:image | https://avatars.githubusercontent.com/u/17950982?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/17950982?s=400&v=4 |
| og:image:alt | Opening a file rewrote it, and the pane visibly re-laid-out a beat after it painted. Both came from the live document drifting out of the shape its own markdown can describe. What was happening Pro... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | cbf66c024bb6d10e6955baa7413c54cf13a201bb |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width