Title: improvement(knowledge): finish Confluence passes past unreadable attachments, rank unauthenticated keyword search in one statement by waleedlatif1 · Pull Request #8975 · simstudioai/sim · GitHub
Open Graph Title: improvement(knowledge): finish Confluence passes past unreadable attachments, rank unauthenticated keyword search in one statement by waleedlatif1 · Pull Request #8975 · simstudioai/sim
X Title: improvement(knowledge): finish Confluence passes past unreadable attachments, rank unauthenticated keyword search in one statement by waleedlatif1 · Pull Request #8975 · simstudioai/sim
Description: Summary Confluence: a 403/404 on one parent's attachments no longer fails the listing listConfluenceAttachments used to record attachment_access_denied (403/404 on /pages/{id}/attachments) as a listing failure. Any pass with one permanently restricted page therefore stayed incomplete: the sync was partial, lastSyncAt never advanced, and every slice recounted the whole connector. That parent is now skipped and logged as a warning. The page still returns reconciliationSafe: false and sets syncContext.reconciliationUnsafe. Because checkpoint.unsafe stays set for the rest of the pass, reconciliation never deletes attachments it did not list. This is the same deletion behavior as before. Missing scope (401 attachment_scope_mismatch) and server failures (attachment_listing_unavailable, plus the 3-in-a-row abort) are still listing failures. A failure listing the pages themselves still throws. Connector document count The member engine now uses the same gate the content engine has had since #8944. Once a pass has finished, an unfinished slice keeps the recorded count instead of scanning the whole connector. Until a pass finishes, every slice still counts, so the first-pass fix from ed2e401 still holds. countLiveConnectorDocuments and the gate (keepsRecordedDocCount) moved to sync-primitives so both engines share them. With the Confluence fix, a connector stuck on restricted attachments now finishes passes, so the existing gate applies to it. Keyword search for callers with no person behind them (workflow, API, Copilot) These callers now rank through keywordCandidateRankingQuery, the materialized match → readable documents → rank statement signed-in readers already use. The old ORDER BY ts_rank_cd over a per-chunk embedding ⋈ document join (with a per-base fan-out for many bases) is gone. Over 7 days in prod the old shape ran p50 ~400 ms / p99 ~9.2 s; the CTE shape ran p50 ~27 ms / p99 ~1.2 s. The ACL predicate is the same read.rankCondition, and hydration is unchanged. The ordering is still a global ts_rank_cd DESC over every readable match. Ties are now broken by chunk id instead of arbitrarily. The candidate side also requires document.knowledge_base_id to be in the searched bases, which only makes the filter stricter. The search budget is deliberately unchanged: these callers still run unbudgeted. Adding a deadline would turn slow searches into partial results for API callers, which is a behavior change. Not changed (deliberately) I did not add a "skip the count when a slice changed no membership" gate for connectors that have never finished a pass. I could not prove it exact. Restores (deletedAt: null on relisted tombstones) and resurrecting updates are not counted separately. Out-of-band excludes, deletes and restores never touch lastSyncDocCount. Without a recorded count timestamp there is also no bounded-staleness variant. Test plan attachments.test.ts: a 403/404 parent is skipped, the listing stays complete, no listingFailures, and it is still reconciliation-unsafe. Fails on the old code. member-sync-engine-content-credential.test.ts: an unfinished slice after a finished pass writes no lastSyncDocCount. Fails on the old code. Focused suites pass: sync-engine, member-sync-engine (both files), confluence, search queries, application search. CI: type-check, audits, integration (filtered-search / scale cover keyword search end to end).
Open Graph Description: Summary Confluence: a 403/404 on one parent's attachments no longer fails the listing listConfluenceAttachments used to record attachment_access_denied (403/404 on /pages/{id}/attachments) as ...
X Description: Summary Confluence: a 403/404 on one parent's attachments no longer fails the listing listConfluenceAttachments used to record attachment_access_denied (403/404 on /pages/{id}/attachments)...
Opengraph URL: https://github.com/simstudioai/sim/pull/8975
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c13d379b-f6fd-41ca-d8da-38893541ac6b |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D32A:2C34A2:16F387:2039B8:6AFB4E01 |
| html-safe-nonce | 60c30a07b9270167f07675937e0759144b0f4b7a938dd05a8a5abe87b45ac2e3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMzJBOjJDMzRBMjoxNkYzODc6MjAzOUI4OjZBRkI0RTAxIiwidmlzaXRvcl9pZCI6IjI5NTgwODYwMzAyOTg0MDIzMDUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | eead42efc5018417e85876d873308e9cfe278b56af32c4bc766a9a4b9c7d64f4 |
| hovercard-subject-tag | pull_request:4847693702 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8975/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Summary Confluence: a 403/404 on one parent's attachments no longer fails the listing listConfluenceAttachments used to record attachment_access_denied (403/404 on /pages/{id}/attachments) as ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | acf7616d06cc051644705b379878c9bb13dc5e70ecf4ffd9b34140f8ac01d7e1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 26504db649b58525278f77a52b80ba5f10d5f4de |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width