Title: bug: self-hosted webapp cannot obtain IRSA credentials — fast-xml-parser 5.7.x rejects @aws-sdk/xml-builder's addEntity("#xD") · Issue #4951 · triggerdotdev/trigger.dev · GitHub
Open Graph Title: bug: self-hosted webapp cannot obtain IRSA credentials — fast-xml-parser 5.7.x rejects @aws-sdk/xml-builder's addEntity("#xD") · Issue #4951 · triggerdotdev/trigger.dev
X Title: bug: self-hosted webapp cannot obtain IRSA credentials — fast-xml-parser 5.7.x rejects @aws-sdk/xml-builder's addEntity("#xD") · Issue #4951 · triggerdotdev/trigger.dev
Description: Summary On a self-hosted deployment on EKS, annotating the trigger-webapp service account for IRSA makes every AWS call from the webapp fail during credential resolution. The pinned fast-xml-parser@5.7.1 throws while the AWS SDK configur...
Open Graph Description: Summary On a self-hosted deployment on EKS, annotating the trigger-webapp service account for IRSA makes every AWS call from the webapp fail during credential resolution. The pinned fast-xml-parser...
X Description: Summary On a self-hosted deployment on EKS, annotating the trigger-webapp service account for IRSA makes every AWS call from the webapp fail during credential resolution. The pinned fast-xml-parser...
Opengraph URL: https://github.com/triggerdotdev/trigger.dev/issues/4951
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"bug: self-hosted webapp cannot obtain IRSA credentials — fast-xml-parser 5.7.x rejects @aws-sdk/xml-builder's addEntity(\"#xD\")","articleBody":"## Summary\n\nOn a self-hosted deployment on EKS, annotating the `trigger-webapp` service account for IRSA makes every AWS call from the webapp fail during **credential resolution**. The pinned `fast-xml-parser@5.7.1` throws while the AWS SDK configures its XML parser, so the STS `AssumeRoleWithWebIdentity` response can never be deserialized.\n\nThe most visible symptom is that magic-link login email fails every time — which locks users out completely when email is the only configured auth provider.\n\n## Error\n\n```\nFailed to send email to \u003credacted>, Magic sign-in link for Trigger.dev.\nError Error: [EntityReplacer] Invalid character '#' in entity name: \"#xD\"\n Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.\n{\"message\":\"Error sending magic link email\",\"level\":\"error\"}\n```\n\nStack:\n\n```\nat fast-xml-parser@5.7.1 … setExternalEntities\nat fast-xml-parser@5.7.1 … parse\nat @aws-sdk/xml-builder@3.930.0/dist-cjs/xml-parser.js:17 … parseXML\nat @aws-sdk/core@3.940.0 … XmlShapeDeserializer.parseXml\nat @aws-sdk/core@3.940.0 … AwsQueryProtocol.deserializeResponse\n```\n\nThe STS response itself is HTTP 200 and well formed — it contains no character references and the string `#xD` appears nowhere in it. The throw happens while *configuring* the parser, not while reading the document, so it is entirely content-independent.\n\n## Cause\n\n`@aws-sdk/xml-builder@3.930.0/dist-cjs/xml-parser.js` registers two character references as entities:\n\n```js\nparser.addEntity(\"#xD\", \"\\r\");\nparser.addEntity(\"#10\", \"\\n\");\n```\n\n`fast-xml-parser` 5.7.0 tightened entity-name validation to reject `#`, so `setExternalEntities` throws on every `parse()` call.\n\n## Minimal reproduction (no AWS credentials, no network)\n\n```js\nconst { XMLParser } = require('fast-xml-parser');\nconst parser = new XMLParser({ htmlEntities: true });\nparser.addEntity('#xD', '\\r'); // exactly what @aws-sdk/xml-builder does\nparser.parse('\u003ca>1\u003c/a>', true); // throws on 5.7.0 and 5.7.1\n```\n\n## Affected versions\n\n| fast-xml-parser | Result |\n| --- | --- |\n| 4.5.6 | ok |\n| 5.6.0 | ok |\n| **5.7.0** | **throws** |\n| **5.7.1** | **throws** |\n| 5.8.0 | ok |\n| 5.9.3 / 5.10.1 / 5.11.0 / 5.11.1 | ok |\n\nBroken only in 5.7.0–5.7.1, fixed in 5.8.0.\n\n## Why this affects every current release\n\n`pnpm-lock.yaml` pins `fast-xml-parser@5.7.1` on `v4.5.9`, `v4.5.11`, `v4.5.13`–`v4.5.16`, `v4.6.0`–`v4.6.3` and `main`. No published image avoids the broken window, so operators cannot work around it by moving image tags.\n\nThe repo also resolves `fast-xml-parser@4.5.6` for the older `@aws-sdk/xml-builder@3.821.0`. Which copy a given client resolves to decides whether it works: the ECR push path keeps working, while `@aws-sdk/client-sesv2@3.940.0` (which pulls `xml-builder@3.930.0`) does not. That makes the failure look service-specific when it is really protocol-specific — every awsQuery/XML service, STS included, is affected.\n\n## Impact\n\nAny self-hosted deployment using IRSA. Because STS uses the awsQuery (XML) protocol, credential resolution itself fails, so this is not limited to email — it breaks every AWS call made through the affected SDK copy.\n\nDeployments using node/instance credentials are unaffected, since IMDS returns JSON and never touches the XML parser. That is why the problem appears only once a pod is switched to IRSA, and why it can sit latent for a long time before surfacing.\n\n## Suggested fix\n\nForce a non-affected version:\n\n```jsonc\n// package.json\n\"pnpm\": { \"overrides\": { \"fast-xml-parser\": \"^5.8.0\" } }\n```\n\n## Environment\n\n- `ghcr.io/triggerdotdev/trigger.dev:v4.6.0`, Helm chart, self-hosted on EKS\n- Node 24\n- `EMAIL_TRANSPORT=aws-ses`\n- Credentials via IRSA (service account annotated with `eks.amazonaws.com/role-arn`)\n\nPossibly related: #3364 (service-account annotation support for IRSA).\n","author":{"url":"https://github.com/vlakshmanan-silkline","@type":"Person","name":"vlakshmanan-silkline"},"datePublished":"2026-09-18T03:40:42Z","url":"https://github.com/triggerdotdev/trigger.dev/issues/4951"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:3e0825a9-4d44-4ce1-c3da-6b1f07e6dd80 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | DDC2:27EABE:5C136CF:7AEE9ED:6AFAD4CB |
| html-safe-nonce | a338d2728211f4290ebdbd84a7253e182e000a4b152fa2637d214514422d04b4 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEREMyOjI3RUFCRTo1QzEzNkNGOjdBRUU5RUQ6NkFGQUQ0Q0IiLCJ2aXNpdG9yX2lkIjoiMzczOTY4NjM5OTMyMjU0MzMwNyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 9ccc363e5aef668f40af4ad6f581f7e9c891f2f7e8d9c548a9706ff9fc3ea1c1 |
| hovercard-subject-tag | issue:5495370482 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/triggerdotdev/trigger.dev/4951/issue_layout |
| twitter:image | https://opengraph.githubassets.com/3fcab43be8193da618bc94f96a94a85a7dc0a91f88ad4b27528b7b0ba059a8e8/triggerdotdev/trigger.dev/issues/4951 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/3fcab43be8193da618bc94f96a94a85a7dc0a91f88ad4b27528b7b0ba059a8e8/triggerdotdev/trigger.dev/issues/4951 |
| og:image:alt | Summary On a self-hosted deployment on EKS, annotating the trigger-webapp service account for IRSA makes every AWS call from the webapp fail during credential resolution. The pinned fast-xml-parser... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | vlakshmanan-silkline |
| hostname | github.com |
| expected-hostname | github.com |
| None | 02f4c00e74911c7cd779fc232c3130cee49eb39fd0ed4bfe3eea3b82c7c18084 |
| turbo-cache-control | no-preview |
| go-import | github.com/triggerdotdev/trigger.dev git https://github.com/triggerdotdev/trigger.dev.git |
| octolytics-dimension-user_id | 95297378 |
| octolytics-dimension-user_login | triggerdotdev |
| octolytics-dimension-repository_id | 572570113 |
| octolytics-dimension-repository_nwo | triggerdotdev/trigger.dev |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 572570113 |
| octolytics-dimension-repository_network_root_nwo | triggerdotdev/trigger.dev |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 33c41e9d150e44415272934d676dc63c57d89c3b |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width