Title: Preflight all externalized payload sizes before starting uploads · Issue #298 · microsoft/durabletask-python · GitHub
Open Graph Title: Preflight all externalized payload sizes before starting uploads · Issue #298 · microsoft/durabletask-python
X Title: Preflight all externalized payload sizes before starting uploads · Issue #298 · microsoft/durabletask-python
Description: The synchronous and asynchronous payload helpers validate and upload one protobuf field at a time. A valid earlier field can therefore be uploaded and replaced by a token before a later field is rejected for a size that was already knowa...
Open Graph Description: The synchronous and asynchronous payload helpers validate and upload one protobuf field at a time. A valid earlier field can therefore be uploaded and replaced by a token before a later field is re...
X Description: The synchronous and asynchronous payload helpers validate and upload one protobuf field at a time. A valid earlier field can therefore be uploaded and replaced by a token before a later field is re...
Opengraph URL: https://github.com/microsoft/durabletask-python/issues/298
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Preflight all externalized payload sizes before starting uploads","articleBody":"The synchronous and asynchronous payload helpers validate and upload one protobuf field at a time. A valid earlier field can therefore be uploaded and replaced by a token before a later field is rejected for a size that was already knowable before any storage I/O. If that response is never submitted, the earlier upload has no backend-recorded reference.\r\n\r\n## Current source evidence\r\n\r\nInspected current `main` at [`cb75760f250099d5b3b49e18ed03e10f8fe46d0d`](https://github.com/microsoft/durabletask-python/commit/cb75760f250099d5b3b49e18ed03e10f8fe46d0d).\r\n\r\n- The [synchronous recursive walker](https://github.com/microsoft/durabletask-python/blob/cb75760f250099d5b3b49e18ed03e10f8fe46d0d/durabletask/payload/helpers.py#L93-L170) visits fields and immediately calls `_try_externalize_field`, which checks only the current payload's maximum before uploading and mutating that field.\r\n- The [asynchronous walker](https://github.com/microsoft/durabletask-python/blob/cb75760f250099d5b3b49e18ed03e10f8fe46d0d/durabletask/payload/helpers.py#L226-L300) has the same sequencing with `await store.upload_async(...)`.\r\n- The [worker completion path](https://github.com/microsoft/durabletask-python/blob/cb75760f250099d5b3b49e18ed03e10f8fe46d0d/durabletask/worker.py#L1278-L1293) calls externalization before submitting `CompleteOrchestratorTask`; there is no whole-message preflight or compensation for successful earlier uploads in that path.\r\n\r\nThis is not a claim that the business request partially succeeds. The problem is the avoidable storage side effect before a deterministically invalid message fails. A cleanup process based on backend-recorded references cannot discover a reference that was never submitted or recorded.\r\n\r\n## Requested regression\r\n\r\nAdd this case to the existing `tests/durabletask/test_large_payload.py`, using its real protobuf types, helper and `FakePayloadStore`:\r\n\r\n```python\r\nstore = FakePayloadStore(threshold_bytes=8, max_stored_payload_bytes=32)\r\nresponse = pb.OrchestratorResponse(\r\n instanceId=\"size-preflight\",\r\n actions=[\r\n pb.OrchestratorAction(\r\n id=1,\r\n scheduleTask=pb.ScheduleTaskAction(input=sv(\"a\" * 20)),\r\n ),\r\n pb.OrchestratorAction(\r\n id=2,\r\n scheduleTask=pb.ScheduleTaskAction(input=sv(\"b\" * 33)),\r\n ),\r\n ],\r\n)\r\nbefore = response.SerializeToString()\r\n\r\nwith pytest.raises(ValueError, match=\"exceeds the maximum\"):\r\n externalize_payloads(response, store, instance_id=\"size-preflight\")\r\n\r\nassert len(store._blobs) == 0\r\nassert response.SerializeToString() == before\r\n```\r\n\r\nThe current source uploads the first input before inspecting the second input's maximum. Repeat the regression with `externalize_payloads_async`, plus an entity result with a valid earlier field and oversized later field.\r\n\r\n> [!NOTE]\r\n> This report is based on the pinned Python source and caller path. The Python regression above has not been executed for this filing. The equivalent JavaScript cases were reproduced and fixed with zero-upload assertions.\r\n\r\n## Acceptance Criteria\r\n\r\n- [ ] Collect and snapshot all relevant non-token payload fields before performing storage I/O, and validate their sizes according to the existing Python eligibility rules.\r\n- [ ] A deterministic maximum-size violation causes zero uploads and leaves all original protobuf payload values unchanged.\r\n- [ ] Apply the guarantee to both synchronous and asynchronous helpers, including nested/repeated messages, carryover events and entity payloads.\r\n- [ ] Preserve known-token idempotency, empty/absent fields, the existing strict-greater threshold behavior, and async cancellation/error propagation.\r\n- [ ] Preserve existing exception behavior for the helper; do not silently convert invalid input into a successful result.\r\n- [ ] Add valid-first/oversized-later, UTF-8 boundary, already-tokenized and all-valid regression cases.\r\n- [ ] Document the intentional sequencing change: deterministic invalid size is discovered before hypothetical earlier upload failures because no upload is attempted.\r\n- [ ] Avoid introducing new abstract members or mandatory dependencies for custom payload stores.\r\n\r\n## Non-goals\r\n\r\nThis does not introduce a cross-blob transaction, recover existing orphan blobs, or guarantee zero orphans after actual I/O/cancellation/submission failures. Unconditionally deleting uploads after an uncertain backend response is unsafe because the backend may already have accepted the reference.\r\n\r\nCarryover fields are already reached by Python's recursive protobuf traversal; this issue does not request a missing-field fix.\r\n\r\n## Related\r\n\r\n- [JavaScript zero-upload preflight fix](https://github.com/microsoft/durabletask-js/commit/55dfcf37da0ed3a2791f778d75621cb7f91144a7).\r\n- [.NET tracking issue 820](https://github.com/microsoft/durabletask-dotnet/issues/820).\r\n- [Python issue 297](https://github.com/microsoft/durabletask-python/issues/297) tracks permanent worker failure completion and entity-lock cleanup. Those acceptance criteria do not cover validating the entire message before any upload; this issue tracks the separate storage-side-effect guarantee.\r\n","author":{"url":"https://github.com/YunchuWang","@type":"Person","name":"YunchuWang"},"datePublished":"2026-10-19T21:01:17Z","url":"https://github.com/microsoft/durabletask-python/issues/298"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:caa5a95d-80c0-e9b9-7cb2-c4ba5d37ed71 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 9910:ADA14:C9817F:11995AA:6AFE2CE9 |
| html-safe-nonce | e66cabf292e65bf5b994ffbf3fae9da7ecded5ca8fecc63e8cae9827f9518a83 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5OTEwOkFEQTE0OkM5ODE3RjoxMTk5NUFBOjZBRkUyQ0U5IiwidmlzaXRvcl9pZCI6IjU1NzQ5OTE1MzQ4MjM5NzYxNjkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | f302067873d9a67a62ac94999140b3067ea4c30a48f81320046756998e2dad1b |
| hovercard-subject-tag | issue:5931503300 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/durabletask-python/298/issue_layout |
| twitter:image | https://opengraph.githubassets.com/7664a700d81501da51483cfe4f6671cf69b0d36127a0706991605e872dabac5f/microsoft/durabletask-python/issues/298 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/7664a700d81501da51483cfe4f6671cf69b0d36127a0706991605e872dabac5f/microsoft/durabletask-python/issues/298 |
| og:image:alt | The synchronous and asynchronous payload helpers validate and upload one protobuf field at a time. A valid earlier field can therefore be uploaded and replaced by a token before a later field is re... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | YunchuWang |
| hostname | github.com |
| expected-hostname | github.com |
| None | e973f9ec3625f3faad8cac5c2d11a214b7ffe45f3cc563b828459b290ecaeed3 |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/durabletask-python git https://github.com/microsoft/durabletask-python.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 627664515 |
| octolytics-dimension-repository_nwo | microsoft/durabletask-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 627664515 |
| octolytics-dimension-repository_network_root_nwo | microsoft/durabletask-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 1f92c367595b9190e0be293896078033886b1913 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width