Title: Bedrock invoke-model: gateway replaces every chat's model with the provider row's settings.model, so model configs silently run (and are priced as) the wrong model · Issue #29511 · coder/coder · GitHub
Open Graph Title: Bedrock invoke-model: gateway replaces every chat's model with the provider row's settings.model, so model configs silently run (and are priced as) the wrong model · Issue #29511 · coder/coder
X Title: Bedrock invoke-model: gateway replaces every chat's model with the provider row's settings.model, so model configs silently run (and are priced as) the wrong model · Issue #29511 · coder/coder
Description: Is there an existing issue for this? I have searched the existing issues Current Behavior Under the default Bedrock protocol (invoke-model), the AI gateway replaces the model in every request body with the provider row's settings.model (...
Open Graph Description: Is there an existing issue for this? I have searched the existing issues Current Behavior Under the default Bedrock protocol (invoke-model), the AI gateway replaces the model in every request body ...
X Description: Is there an existing issue for this? I have searched the existing issues Current Behavior Under the default Bedrock protocol (invoke-model), the AI gateway replaces the model in every request body ...
Opengraph URL: https://github.com/coder/coder/issues/29511
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bedrock invoke-model: gateway replaces every chat's model with the provider row's settings.model, so model configs silently run (and are priced as) the wrong model","articleBody":"### Is there an existing issue for this?\n\n- [x] I have searched the existing issues\n\n### Current Behavior\n\nUnder the default Bedrock protocol (`invoke-model`), the AI gateway **replaces the model in every request body with the provider row's `settings.model`** (or `settings.small_fast_model` when the requested id contains `haiku`). The chat's `model_config_id` decides only the display name and the price table. `aibridge/intercept/messages/base.go` `Model()` still does this on `main` today.\n\nConsequences we measured on a production deployment (v2.36.5, 2026-09-17):\n\n- A 6-hour Coder Agents delegation on a model config named \"Claude Fable 5\" (`global.anthropic.claude-fable-5`, 1,453 steps) ran **entirely on Sonnet 5**. CloudWatch `AWS/Bedrock` metrics by `ModelId` for the whole day show a single dimension, `global.anthropic.claude-sonnet-5`; no fable/opus id exists as a metric at all.\n- `GET /api/experimental/chats/{id}/cost` reported **$854** at the Fable config's prices. Real spend at Sonnet rates was ~$256. Every Opus- and Fable-labelled chat had been overbilled ~3.3× and nobody received the model they selected, since at least 2026-08-27.\n- Non-Anthropic configs on the same row (`openai.gpt-5.6-luna`, which was our default, `xai.grok-4.3`) were also silently Sonnet 5. Nothing in the UI, the chat, or the cost endpoint reveals the substitution.\n\nThe only workaround on `invoke-model` is **one Bedrock provider row per model** whose `settings.model` equals the config's model (we now run three identical-looking rows: Sonnet, Opus 5, Fable 5.1). The `mantle` protocol passes the model through, but Mantle does not yet serve every model (no `claude-fable-5-1`, no `gpt-6-astra` as of 2026-09-18), so it cannot replace the legacy path.\n\n### Relevant Log Output\n\n```text\n# CloudWatch AWS/Bedrock Invocations by ModelId, 2026-09-17 18:00Z – 2026-09-18 00:30Z\nglobal.anthropic.claude-sonnet-5 Invocations 2003 (only dimension present)\n\n# Coder cost for the Fable-labelled chat tree over the same window\n{\"total_cost_micros\":854160585,\"priced_message_count\":1453}\n\n# Provider row (no protocol → invoke-model)\n\"settings\": {\"_type\":\"bedrock\",\"_version\":1,\"model\":\"global.anthropic.claude-sonnet-5\",\n \"small_fast_model\":\"global.anthropic.claude-haiku-4-5-20251001-v1:0\",\"region\":\"us-east-1\",\"role_arn\":\"...\"}\n```\n\n### Expected Behavior\n\nThe model a chat is pinned to is the model Bedrock is asked for, under every protocol. Concretely:\n\n1. **On `invoke-model`, send the config's model.** `settings.model` / `settings.small_fast_model` should only be a fallback when the request carries no model (Claude Code's `ANTHROPIC_MODEL` / `ANTHROPIC_SMALL_FAST_MODEL` use case), never an override of an explicit `model_config`. The chat daemon always knows the config's model, so it can pass it through; the `haiku`-substring heuristic should not apply to a request that already names a Bedrock inference profile.\n2. **Reject the mismatch instead of hiding it.** If the gateway must substitute, a model config whose `model` differs from what the provider row will send should fail validation on create/update (the same place #29112 already resolves inference-profile ARNs), and the chat should surface the substituted model in `messages[].usage` / the cost endpoint so pricing follows the model that actually ran.\n3. **Price what ran.** Cost should be computed from the model id the upstream response reports (Bedrock returns it), not from the config the chat was created with.\n\nAny one of these would have made the problem visible; (1) makes it go away without per-model provider rows.\n\n### Steps to Reproduce\n\n1. One Bedrock provider (default protocol) with `settings.model = global.anthropic.claude-sonnet-5`.\n2. Two enabled model configs on it: `global.anthropic.claude-sonnet-5` and `global.anthropic.claude-opus-5` (or any second Anthropic id), each with its own prices.\n3. Create a chat pinned to the Opus config, send one prompt.\n4. Look at CloudTrail `InvokeModelWithResponseStream` `resources[].ARN` or CloudWatch `AWS/Bedrock` by `ModelId`: the request went to Sonnet. `GET /chats/{id}/cost` priced it as Opus.\n\n### Environment\n\n- Coder v2.36.5+28c4c83 (same code on `main` as of 2026-09-18, `aibridge` `intercept/messages/base.go` `Model()`)\n- Provider type `bedrock`, `role_arn` auth, region us-east-1, protocol unset (`invoke-model`)\n- Coder Agents chats created via `POST /api/experimental/chats` with `model_config_id`\n\n### Additional Context\n\n- Workaround in use: one provider row per model (`bedrock`, `bedrock-fable`, `bedrock-opus`) with matching `settings.model`. It works but is invisible in the model editor and nothing warns when a config points at a row that will send a different model.\n- #28404 (Mantle, non-Anthropic models) and #29230 (reasoning-model override) improve the Mantle path, but deployments that need a model Mantle does not list still depend on `invoke-model`, where the substitution remains.\n- Related: #26734, #27361 (other cases of Bedrock model ids being reinterpreted silently).\n","author":{"url":"https://github.com/amaingot","@type":"Person","name":"amaingot"},"datePublished":"2026-09-18T02:38:48Z","url":"https://github.com/coder/coder/issues/29511"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:bdda41cc-1c0d-dffe-2bdb-5153f746bdba |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | B062:363D8E:2F7A8FB:3C39285:6B01ABAB |
| html-safe-nonce | 87cae7cccd2681d1714d8a3ed1f513f45d314b1658d622bfc193f8dd4ac4c456 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMDYyOjM2M0Q4RToyRjdBOEZCOjNDMzkyODU6NkIwMUFCQUIiLCJ2aXNpdG9yX2lkIjoiMTEyMTgyNzAxNjkyMDkwMjU3MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 80e55a149b5fab90b75d2881196396ce78c6d15e495aba442197cd0e58d8696b |
| hovercard-subject-tag | issue:5495008831 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/coder/coder/29511/issue_layout |
| twitter:image | https://opengraph.githubassets.com/a1823bb70d0fc5d98bc54d89efd984a0b8fc0c5b749727723ce9bddc4c861a3f/coder/coder/issues/29511 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/a1823bb70d0fc5d98bc54d89efd984a0b8fc0c5b749727723ce9bddc4c861a3f/coder/coder/issues/29511 |
| og:image:alt | Is there an existing issue for this? I have searched the existing issues Current Behavior Under the default Bedrock protocol (invoke-model), the AI gateway replaces the model in every request body ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | amaingot |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/coder/coder git https://github.com/coder/coder.git |
| octolytics-dimension-user_id | 95932066 |
| octolytics-dimension-user_login | coder |
| octolytics-dimension-repository_id | 440752086 |
| octolytics-dimension-repository_nwo | coder/coder |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 440752086 |
| octolytics-dimension-repository_network_root_nwo | coder/coder |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width