René's URL Explorer Experiment


Title: fix(connectors): contentDeferred pattern + validation fixes across all connectors by waleedlatif1 · Pull Request #3793 · simstudioai/sim · GitHub

Open Graph Title: fix(connectors): contentDeferred pattern + validation fixes across all connectors by waleedlatif1 · Pull Request #3793 · simstudioai/sim

X Title: fix(connectors): contentDeferred pattern + validation fixes across all connectors by waleedlatif1 · Pull Request #3793 · simstudioai/sim

Description: Summary Converted all 10 connectors to contentDeferred: true pattern — listDocuments now returns lightweight metadata stubs (no content download), and content is fetched lazily via getDocument only for new/changed documents. This prevents task timeouts on large syncs (root cause of Google Drive, Fireflies failures). Ran full validation audit on all 10 connectors against their respective API docs. Fixed all critical and warning issues found. Updated add-connector and validate-connector skills to document the contentDeferred best practice. Connectors converted to contentDeferred Connector contentHash strategy Notable fixes Google Drive gdrive:${id}:${modifiedTime} orderBy, precise maxFiles, byte-length truncation OneDrive onedrive:${id}:${lastModifiedDateTime} orderBy for deterministic pagination SharePoint sharepoint:${id}:${lastModifiedDateTime} byte-length size check Dropbox dropbox:${id}:${content_hash} uses Dropbox native content_hash Notion notion:${pageId}:${last_edited_time} code/equation blocks, empty page fallback, concurrency reduction Confluence confluence:${pageId}:${versionNumber} syncContext cloudId caching, label concurrency reduction Gmail gmail:${threadId}:${historyId} joinTagArray for labels Obsidian stub: obsidian:stub:${path}:${syncRunId}, doc: obsidian:${path}:${mtime} forced re-fetch per sync (API lacks mtime in listing) Evernote evernote:${guid}:${updated} retryOptions threaded through Thrift RPC calls GitHub git-sha:${sha} (pre-existing) added contentDeferred: false to getDocument Infrastructure changes sync-engine.ts: added syncRunId to syncContext for Obsidian change detection confluence/utils.ts: replaced raw fetch() with fetchWithRetry, added retryOptions parameter oauth.ts: added supportsRefreshTokenRotation: false for Dropbox Test plan Trigger sync for Google Drive connector — should complete without timeout Trigger sync for each converted connector — documents should appear without refresh Verify lastSyncDocCount matches actual KB document count after sync Verify previously stuck "processing" documents get retried on next sync Run bun run lint — passes clean Run bunx tsc --noEmit — no errors in changed files

Open Graph Description: Summary Converted all 10 connectors to contentDeferred: true pattern — listDocuments now returns lightweight metadata stubs (no content download), and content is fetched lazily via getDocument onl...

X Description: Summary Converted all 10 connectors to contentDeferred: true pattern — listDocuments now returns lightweight metadata stubs (no content download), and content is fetched lazily via getDocument onl...

Opengraph URL: https://github.com/simstudioai/sim/pull/3793

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:e6fa4fc2-f849-6fa9-452b-ff901d5e0522
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idC4CC:217101:1BEAA4:276C6A:6AFB798B
html-safe-nonce79d4b9647c55e7f01d41382dad527ed7c08b1dd3cd1e2ebdb1e483b7a4f0746e
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDNENDOjIxNzEwMToxQkVBQTQ6Mjc2QzZBOjZBRkI3OThCIiwidmlzaXRvcl9pZCI6IjMzMjk0ODE1NjMzMDQ1ODk3MDciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac6c3b961223810c28924a0e724d1944eda79558d4925659795338317586ba6694
hovercard-subject-tagpull_request:3453969942
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/3793/files
twitter:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/40672544?s=400&v=4
og:image:altSummary Converted all 10 connectors to contentDeferred: true pattern — listDocuments now returns lightweight metadata stubs (no content download), and content is fetched lazily via getDocument onl...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None0e29e7974fdda2feeaf5c891efd4b23b0734b23480232b62c7bdef13f40b5b48
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/simstudioai/sim git https://github.com/simstudioai/sim.git
octolytics-dimension-user_id199344406
octolytics-dimension-user_loginsimstudioai
octolytics-dimension-repository_id912559512
octolytics-dimension-repository_nwosimstudioai/sim
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id912559512
octolytics-dimension-repository_network_root_nwosimstudioai/sim
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release2e8872b770cc4d5d0cc1f2026afab747a268e2ce
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/simstudioai/sim/pull/3793/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F3793%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim%2Fpull%2F3793%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=simstudioai%2Fsim
Reloadhttps://github.com/simstudioai/sim/pull/3793/files
Reloadhttps://github.com/simstudioai/sim/pull/3793/files
Reloadhttps://github.com/simstudioai/sim/pull/3793/files
simstudioaihttps://github.com/simstudioai
simhttps://github.com/simstudioai/sim
Codehttps://github.com/simstudioai/sim
Issues129 (129)https://github.com/simstudioai/sim/issues
Pull requests202 (202)https://github.com/simstudioai/sim/pulls
Actionshttps://github.com/simstudioai/sim/actions
Projectshttps://github.com/simstudioai/sim/projects
Security and qualityhttps://github.com/simstudioai/sim/security
Insightshttps://github.com/simstudioai/sim/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsimstudioai%2Fsim%2Fissues%2Fnew%2Fchoose
waleedlatif1https://github.com/waleedlatif1
staginghttps://github.com/simstudioai/sim/tree/staging
waleedlatif1/fix-connector-sync-timeouthttps://github.com/simstudioai/sim/tree/waleedlatif1/fix-connector-sync-timeout
Conversation 72 https://github.com/simstudioai/sim/pull/3793
Commits 11 https://github.com/simstudioai/sim/pull/3793/commits
Checks 11 https://github.com/simstudioai/sim/pull/3793/checks
Files changed https://github.com/simstudioai/sim/pull/3793/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/3793/files
fix(connectors): contentDeferred pattern + validation fixes across all connectors https://github.com/simstudioai/sim/pull/3793/files#top
Show all changes 11 commits https://github.com/simstudioai/sim/pull/3793/files
4c19754 fix(knowledge): enqueue connector docs per-batch to survive sync time… waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/4c19754b05c3e89ac7fe8e66b9752c67624b07b4
f9785be fix(connectors): convert all connectors to contentDeferred pattern an… waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/f9785be5be5541b84d39f1436d07f09a4e5f9a7e
228afa4 fix(connectors): address PR review comments - metadata merge, retryOp… waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/228afa40b8e60d55a130fa9e07aea30ecfb25904
eb78814 fix(evernote): use correct RetryOptions property names waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/eb788141bca6b055ba3aaa8b4f767cb6aa4c1741
8c409c2 fix(sync-engine): merge title from getDocument and skip unchanged doc… waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/8c409c2f85b6ee31f8ce5f601fd6366bf4d5249c
70e9726 fix(sync-engine): dedup externalIds, enable deletion reconciliation, … waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/70e9726d7162d99ad025d1182b0318338e8016da
39e7768 lint waleedlatif1 Mar 26, 2026 https://github.com/simstudioai/sim/pull/3793/commits/39e7768d2a5f0d9307ca361f25af99690114b7d4
bc73154 fix(connectors): confluence version metadata fallback and google driv… waleedlatif1 Mar 27, 2026 https://github.com/simstudioai/sim/pull/3793/commits/bc73154d6dc69d30f5c1df20be947058e898c8c0
6ffc0e3 fix(connectors): blogpost labels and capped listing deletion reconcil… waleedlatif1 Mar 27, 2026 https://github.com/simstudioai/sim/pull/3793/commits/6ffc0e3666227a2e9adb2aeee2405ddab37d1954
50182d1 fix(connectors): set syncContext.listingCapped in all connectors with… waleedlatif1 Mar 27, 2026 https://github.com/simstudioai/sim/pull/3793/commits/50182d154900b12d07c085a9abc3d24a1adcebbe
fcde037 fix(evernote): thread retryOptions through apiListTags and apiListNot… waleedlatif1 Mar 27, 2026 https://github.com/simstudioai/sim/pull/3793/commits/fcde0375a26b233e762db7f3bfbb5e8848ed00a6
Clear filters https://github.com/simstudioai/sim/pull/3793/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/3793/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/3793/files
SKILL.md https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
SKILL.md https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
confluence.ts https://github.com/simstudioai/sim/pull/3793/files#diff-f18758063260e6e3580a3ed9396dc9eddd8d3ee9c8949a75dae3021e7d3522c6
dropbox.ts https://github.com/simstudioai/sim/pull/3793/files#diff-7725a693f80f563c9ab98d6c982ac19e3713b1e138b8b58d54e06cf5efec86ad
evernote.ts https://github.com/simstudioai/sim/pull/3793/files#diff-f9a0d43d04bec85aec02ddab4d7038b9ee8885c6d182693a681c930ff59111d4
github.ts https://github.com/simstudioai/sim/pull/3793/files#diff-568e3e321373fb7709612cdf0818a435418300d87fee10988cb65094cf048613
gmail.ts https://github.com/simstudioai/sim/pull/3793/files#diff-964bead70eb6cb464699f4a10af9db86082ca098ef99d0a24483f3e3edb0bf9e
google-drive.ts https://github.com/simstudioai/sim/pull/3793/files#diff-a7820fe9749a8ab70e16ce3488df54419d326c6619f6d0b62382bc27d0c94cfd
notion.ts https://github.com/simstudioai/sim/pull/3793/files#diff-c40647c88f5403af432999773a09298b0a92cd8bce7fb89c3d321a8f4e22627f
obsidian.ts https://github.com/simstudioai/sim/pull/3793/files#diff-2f4917fb547fcbdefc5bb58156a7079b3c6a5ba8aa79b8c4190af3e59d4f9645
onedrive.ts https://github.com/simstudioai/sim/pull/3793/files#diff-60e130e26008d9eb6e671ee3195a984336ebd186565dced245cd820268143da1
sharepoint.ts https://github.com/simstudioai/sim/pull/3793/files#diff-a39a21bb15e6c27b68de3dc7899e31ba8e4ee2fa37e7c34ce1c93a23ab25c625
sync-engine.ts https://github.com/simstudioai/sim/pull/3793/files#diff-277f76464f02682e44e6de58ebd215c927f0f6a3a7165af669a0717b593790e9
oauth.ts https://github.com/simstudioai/sim/pull/3793/files#diff-8a2b3eb4f06390885c9e87d5fc4949ad359f7f6c0aa55fbf9e281be1c1fe639d
utils.ts https://github.com/simstudioai/sim/pull/3793/files#diff-3a6a00447e464d33558162ad684136c47bf6946e241af118f5432f89b393e431
.agents/skills/add-connector/SKILL.mdhttps://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
View file https://github.com/simstudioai/sim/blob/fcde0375a26b233e762db7f3bfbb5e8848ed00a6/.agents/skills/add-connector/SKILL.md
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/3793/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
https://github.com/simstudioai/sim/pull/3793/files#diff-966e9361f45dfe9ab99d7ae4f27d36dc91353b9baaa6171f301bbee728d554c2
.agents/skills/validate-connector/SKILL.mdhttps://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
View file https://github.com/simstudioai/sim/blob/fcde0375a26b233e762db7f3bfbb5e8848ed00a6/.agents/skills/validate-connector/SKILL.md
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/simstudioai/sim/pull/3793/{{ revealButtonHref }}
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
https://github.com/simstudioai/sim/pull/3793/files#diff-36b2790e93210d444fa5b8b74229d0f642a1e4c48c5ea6ed0ac48563b50aed2a
Please reload this pagehttps://github.com/simstudioai/sim/pull/3793/files
Please reload this pagehttps://github.com/simstudioai/sim/pull/3793/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.