Title: fix(mothership): default chat effort back to high by waleedlatif1 · Pull Request #8659 · simstudioai/sim · GitHub
Open Graph Title: fix(mothership): default chat effort back to high by waleedlatif1 · Pull Request #8659 · simstudioai/sim
X Title: fix(mothership): default chat effort back to high by waleedlatif1 · Pull Request #8659 · simstudioai/sim
Description: Summary Restores high as the reasoning effort a Mothership chat (and a Sim Chat block) runs at until the user picks one. #8634 lowered that default to medium; this flips it back and keeps everything else from #8634. Why A paired benchmark (3 reps, 160 pairs) of main-agent medium vs high effort: medium cost 24% less and ran 27% faster. It also lost correctness: pass rate 90.0% (high) vs 86.9% (medium), 5 losses and 0 wins (one-sided McNemar p = 0.031). Follow-up chats fell from 100% to 90.5%, and late-run recall fell 11 points. The losses repeat on the same tasks: scheduling, multi-turn user correction, and plan-vs-do. The product rule is no intelligence regression, so the default goes back to high. What changed The default lives in one constant: DEFAULT_MOTHERSHIP_EFFORT in apps/sim/lib/mothership/model-options.ts. It changes from 'medium' to 'high'. #8634 already routed every default through this constant, so the change covers: The server fallback (sent effort, then the stored pick, then the default) and the effort the picker shows for a new chat, both through resolveMothershipModelSettings. The none fallback when the model can't run none. The Sim Chat block's dropdown default for new blocks. The block handler's fallback, so a block with no stored value runs at high again. No docs or UI copy state the default, so none changed. Explicit picks under the default flip Turn admission records the effort a send carries as the chat's explicit pick. If the composer sent the default on every send, every chat created while the default was medium would have medium stored as a pick, and this flip would never reach them. It doesn't. The composer sends effort only when the user picked one: the chat's pick from this session, or its stored pick (a real pick, re-sent); for a chat not yet created, the new-chat composer's pick. With no pick, the request omits effort, and admission records nothing (effortChoice is undefined, already asserted in post.test.ts). This PR adds a client test that pins this behaviour. Stored rows: #8634 is on staging only (d32d4c93f7 is not on origin/main), so production has no config.effort rows. Every stored value came from a real pick, so no data migration is needed. What stays from #8634 Low mode in the simple picker. Each label sends exactly its effort: Low/Medium/High/Extra High send low/medium/high/xhigh. A per-chat explicit pick, stored in copilot_chats.config.effort, and PUT /api/mothership/chats/[chatId]/effort. An explicit pick still beats the default: chats where the user picked medium or low keep it. The client pick logic, including #8655's rollback. Test plan post.test.ts: with no stored pick, a chat now runs at high, and a submitted none falls back to high. A stored explicit medium pick is kept. mothership-handler.test.ts: a Sim Chat block with no effort set sends high. set-effort.integration.ts (real Postgres): a chat with no pick, and one whose stored value is out of range, both run at high. use-chat.dom.test.tsx: from a new or an existing chat, a send with no pick omits effort; a medium pick is sent as medium. Red before the fix: with the constant set back to medium, the no-pick server, none fallback and block assertions fail. With the client mutated to always send an effort, the no-pick send tests fail. Local gate: lint, type-check, check:audits, docs-manifest:check, check-block-registry origin/staging, and bun run test (all 19 workspaces green).
Open Graph Description: Summary Restores high as the reasoning effort a Mothership chat (and a Sim Chat block) runs at until the user picks one. #8634 lowered that default to medium; this flips it back and keeps everythin...
X Description: Summary Restores high as the reasoning effort a Mothership chat (and a Sim Chat block) runs at until the user picks one. #8634 lowered that default to medium; this flips it back and keeps everythin...
Opengraph URL: https://github.com/simstudioai/sim/pull/8659
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:4a991f9d-e53c-40e3-37bf-878404d1c36f |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | EDEC:C876F:ACEDDE:EEB0B5:6AFB527D |
| html-safe-nonce | 10de815b564e6b5023f820b917e4aa52b7b20347c78b90f0bb163543e6711d7b |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFREVDOkM4NzZGOkFDRURERTpFRUIwQjU6NkFGQjUyN0QiLCJ2aXNpdG9yX2lkIjoiNTQ3MzIyMzA4MzYwODk4NjIzNyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 7753429a5f182c8c0ee101e6cd06d8d91afae296b3a702edeca922b930aacc09 |
| hovercard-subject-tag | pull_request:4753661024 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8659/files |
| twitter:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/40672544?s=400&v=4 |
| og:image:alt | Summary Restores high as the reasoning effort a Mothership chat (and a Sim Chat block) runs at until the user picks one. #8634 lowered that default to medium; this flips it back and keeps everythin... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7122bb414b542dba14d71a3f0b9f6fd2abce715b9f5912d082800563e6c806d1 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | cbf66c024bb6d10e6955baa7413c54cf13a201bb |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width