Title: Preflight all externalized payload sizes before starting uploads · Issue #308 · microsoft/durabletask-java · GitHub
Open Graph Title: Preflight all externalized payload sizes before starting uploads · Issue #308 · microsoft/durabletask-java
X Title: Preflight all externalized payload sizes before starting uploads · Issue #308 · microsoft/durabletask-java
Description: The Azure Blob interceptor externalizes response fields sequentially, validating each size only immediately before that field's upload. A later deterministic size violation can therefore occur after earlier valid uploads. The failed-comp...
Open Graph Description: The Azure Blob interceptor externalizes response fields sequentially, validating each size only immediately before that field's upload. A later deterministic size violation can therefore occur afte...
X Description: The Azure Blob interceptor externalizes response fields sequentially, validating each size only immediately before that field's upload. A later deterministic size violation can therefore occur ...
Opengraph URL: https://github.com/microsoft/durabletask-java/issues/308
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Preflight all externalized payload sizes before starting uploads","articleBody":"The Azure Blob interceptor externalizes response fields sequentially, validating each size only immediately before that field's upload. A later deterministic size violation can therefore occur after earlier valid uploads. The failed-completion fallback discards the partially transformed builder and its references, but does not undo those storage writes.\r\n\r\n## Current source evidence\r\n\r\nInspected current `main` at [`0262c53c8f13401bfe2c25a3e058128963ad51a9`](https://github.com/microsoft/durabletask-java/commit/0262c53c8f13401bfe2c25a3e058128963ad51a9).\r\n\r\n- [`externalizeOrchestratorResponse`](https://github.com/microsoft/durabletask-java/blob/0262c53c8f13401bfe2c25a3e058128963ad51a9/azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadInterceptor.java#L211-L256) processes custom status and then actions. On a size exception, it returns a new failed completion rather than the builder containing earlier uploaded references.\r\n- [`maybeExternalize`](https://github.com/microsoft/durabletask-java/blob/0262c53c8f13401bfe2c25a3e058128963ad51a9/azure-blob-payloads/src/main/java/com/microsoft/durabletask/azureblobpayloads/LargePayloadInterceptor.java#L1102-L1123) validates only the current field and then calls `payloadStore.upload`.\r\n- Immutable protobuf messages prevent some caller-visible mutation, but they do not roll back successful Blob uploads.\r\n\r\nThe still-open [parity PR 306](https://github.com/microsoft/durabletask-java/pull/306), inspected at `a6105169da7c72fed25d6411b2fe295a4d4fe2e7`, changes transient failure handling but still has this per-field check/upload sequence.\r\n\r\n## Requested regression\r\n\r\nUse the existing `LargePayloadInterceptorTest` harness, including its mock store and `invokeExternalize` helper:\r\n\r\n```java\r\noptions.setThresholdBytes(8).setMaxPayloadBytes(32);\r\ninterceptor = new LargePayloadInterceptor(mockStore, options);\r\nOrchestratorResponse request = OrchestratorResponse.newBuilder()\r\n .setInstanceId(\"size-preflight\")\r\n .setCompletionToken(\"completion\")\r\n .setCustomStatus(StringValue.of(\"s\".repeat(16)))\r\n .addActions(OrchestratorAction.newBuilder()\r\n .setId(1)\r\n .setScheduleTask(ScheduleTaskAction.newBuilder()\r\n .setName(\"activity\")\r\n .setInput(StringValue.of(\"x\".repeat(33)))))\r\n .build();\r\n\r\nOrchestratorResponse result = (OrchestratorResponse) invokeExternalize(request);\r\n\r\nverify(mockStore, never()).upload(anyString());\r\nassertEquals(1, result.getActionsCount());\r\nassertEquals(\r\n OrchestrationStatus.ORCHESTRATION_STATUS_FAILED,\r\n result.getActions(0).getCompleteOrchestration().getOrchestrationStatus());\r\n```\r\n\r\nThe current sequence can upload the valid custom status before checking the oversized activity input, then omit its token from the failure response. Cover entity batches and completion carryover fields in the preflight as well.\r\n\r\n**Evidence boundary:** this is a source-backed report. The Java regression recipe has not been run for this filing; the equivalent JavaScript storage-side-effect cases were reproduced and fixed.\r\n\r\n## Acceptance Criteria\r\n\r\n- [ ] Inspect and validate the entire set of relevant outgoing payload fields before the first store upload, retaining original values for the later transformation pass.\r\n- [ ] Any deterministic maximum-size violation causes zero uploads for that message.\r\n- [ ] Include all currently supported fields: custom status, action inputs/results/details, carryover history, relevant failure details and entity payloads.\r\n- [ ] Preserve existing permanent failure representation and the independently tracked entity-release obligations; do not retain invalid business actions merely to preserve uploaded references.\r\n- [ ] Preserve empty/absent fields, threshold semantics, configured limits and cancellation/error behavior.\r\n- [ ] Keep the eventual implementation compatible with known-token idempotency: references that are already final must not be re-uploaded or treated as new raw payloads.\r\n- [ ] Add valid-first/oversized-later orchestration/entity tests, UTF-8 boundaries and all-valid transformations.\r\n- [ ] Document that deterministic size validation now occurs before hypothetical earlier upload errors.\r\n\r\n## Non-goals\r\n\r\nThis does not make backend submission and Blob uploads one transaction, recover existing orphan blobs, or guarantee zero orphans after real I/O failures. No unconditional catch-and-delete rollback should be added: an uncertain completion response is not proof that the backend did not record a reference.\r\n\r\nThe current Java interceptor already traverses `CompleteOrchestrationAction.CarryoverEvents`; the JavaScript carryover omission does not need a separate Java issue.\r\n\r\n## Related\r\n\r\n- [JavaScript zero-upload preflight fix](https://github.com/microsoft/durabletask-js/commit/55dfcf37da0ed3a2791f778d75621cb7f91144a7).\r\n- [.NET tracking issue 820](https://github.com/microsoft/durabletask-dotnet/issues/820).\r\n- [Java issue 307](https://github.com/microsoft/durabletask-java/issues/307) addresses entity unlock retention and known-token handling. This issue is the distinct deterministic-size-before-storage guarantee.\r\n","author":{"url":"https://github.com/YunchuWang","@type":"Person","name":"YunchuWang"},"datePublished":"2026-10-19T21:01:17Z","url":"https://github.com/microsoft/durabletask-java/issues/308"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:68a936e3-de77-57ee-6cfe-91075b92c705 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | A202:248224:139C6A:1AAE61:6AFE3568 |
| html-safe-nonce | c25eb7278f701044aa990863a5c8f718e78925eb7eb2374418499d6e54355f57 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMjAyOjI0ODIyNDoxMzlDNkE6MUFBRTYxOjZBRkUzNTY4IiwidmlzaXRvcl9pZCI6IjI2NzA0OTY2ODY2MDU3NDM0NjQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | f18edf037e8c03abde87ee9d0f680c49b8af433341687c0318ee237cd4ae6e7f |
| hovercard-subject-tag | issue:5931503466 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/durabletask-java/308/issue_layout |
| twitter:image | https://opengraph.githubassets.com/9de1bb2d799f1b4b5060e2d8d72b3b44638d0170d60e8945defce5d8d16472dd/microsoft/durabletask-java/issues/308 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/9de1bb2d799f1b4b5060e2d8d72b3b44638d0170d60e8945defce5d8d16472dd/microsoft/durabletask-java/issues/308 |
| og:image:alt | The Azure Blob interceptor externalizes response fields sequentially, validating each size only immediately before that field's upload. A later deterministic size violation can therefore occur afte... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | YunchuWang |
| hostname | github.com |
| expected-hostname | github.com |
| None | 260deedcf4630abb0a6b4aaa02fd2af70eab44d2529712694ceae59ca131b02e |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/durabletask-java git https://github.com/microsoft/durabletask-java.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 452823832 |
| octolytics-dimension-repository_nwo | microsoft/durabletask-java |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 452823832 |
| octolytics-dimension-repository_network_root_nwo | microsoft/durabletask-java |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8b413ea7a9b010f4f2ebbb923b041b0d6091dd94 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width