Title: [Azure Functions] Reduce non-Latin payload inflation in df_dumps · Issue #291 · microsoft/durabletask-python · GitHub
Open Graph Title: [Azure Functions] Reduce non-Latin payload inflation in df_dumps · Issue #291 · microsoft/durabletask-python
X Title: [Azure Functions] Reduce non-Latin payload inflation in df_dumps · Issue #291 · microsoft/durabletask-python
Description: Parent: #269 — ask 4. Problem The reporter's activity results contained 2.55 MB of UTF-8 data, mostly Cyrillic, while the rejected orchestration completion request was 11.05 MB. The report identifies azure.functions._durable_functions.df...
Open Graph Description: Parent: #269 — ask 4. Problem The reporter's activity results contained 2.55 MB of UTF-8 data, mostly Cyrillic, while the rejected orchestration completion request was 11.05 MB. The report identifi...
X Description: Parent: #269 — ask 4. Problem The reporter's activity results contained 2.55 MB of UTF-8 data, mostly Cyrillic, while the rejected orchestration completion request was 11.05 MB. The report iden...
Opengraph URL: https://github.com/microsoft/durabletask-python/issues/291
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[Azure Functions] Reduce non-Latin payload inflation in df_dumps","articleBody":"Parent: #269 — ask 4.\n\n## Problem\n\nThe reporter's activity results contained 2.55 MB of UTF-8 data, mostly Cyrillic, while the rejected orchestration completion request was 11.05 MB. The report identifies `azure.functions._durable_functions.df_dumps` using `json.dumps` with its default `ensure_ascii=True` as a source of inflation: a Cyrillic character that occupies two bytes in UTF-8 becomes a six-byte `\\uXXXX` escape.\n\nThese figures describe the reported workload; the full request also has other content and overhead, so they should not be treated as a universal serialization ratio.\n\n## Agreed approach: explicit opt-in, unchanged defaults\n\nThe encoding API belongs in the shared `azure-functions` SDK, not in a competing Durable-specific serializer. The shared SDK now exposes the keyword-only option `df_dumps(value, *, ensure_ascii=True)`, covering every existing codec branch. The existing default remains byte-for-byte compatible. Explicit `ensure_ascii=False` emits ordinary Unicode literally while preserving transport-safe surrogate-code-unit escapes and the legacy ASCII DEL escape.\n\nThe companion `azure-functions-durable` change exposes root-app configuration at startup: `app.configure_serialization(ensure_ascii=False)`. The process-wide option must be applied consistently to synchronous/asynchronous clients, orchestration and entity workers, activity binding converters, and registered blueprints. Configure every worker consistently before invocations; conflicting settings must fail explicitly rather than silently changing encoding.\n\nApplications that do not opt in retain the existing ASCII-escaped encoding. The companion PR now requires `azure-functions>=2.4.0b1`, which contains the released upstream API; documentation and sample requirements use the same minimum. If an incompatible SDK is manually installed despite the dependency requirement, unsupported opt-in configuration fails explicitly. Provider/core package versions remain unchanged.\n\nUpstream tracking: [Azure/azure-functions-python-library#371](https://github.com/Azure/azure-functions-python-library/issues/371).\n\nUpstream implementation: [Azure/azure-functions-python-library#372](https://github.com/Azure/azure-functions-python-library/pull/372), merged and available in [azure-functions 2.4.0b1](https://github.com/Azure/azure-functions-python-library/releases/tag/2.4.0b1).\n\nCompanion implementation: [#292](https://github.com/microsoft/durabletask-python/pull/292), now ready for review after adopting the released shared-library dependency. Validation against the published 2.4.0b1 wheel includes provider unit coverage, focused local Functions-host E2E coverage, and source/built-package type checks. Hosted-backend staging validation is not claimed.\n\n## Compatibility\n\nExisting ASCII-escaped and literal Unicode JSON deserialize to the same values. Local public-provider experiments exercised activity/sub-orchestration replay across encoding upgrades and rollbacks. The Functions/DTS compatibility assessment does not establish live production validation.\n\nBecause the codec belongs to a shared library, evaluation must also cover other consumers of its serialization API. Keeping the default unchanged avoids imposing a new byte representation on consumers that have not opted in. Applications deriving hashes, signatures, IDs, or replay decisions from serialized JSON still need upgrade/rollback guidance when enabling the option.\n\n## Acceptance criteria\n\n- Preserve the shared codec's default and explicit `ensure_ascii=True` output, envelopes, strict/loose rules, whitespace, and separators.\n- Add opt-in round-trip coverage for ASCII, Cyrillic, other scripts, Unicode keys, combining characters, and valid characters outside the basic multilingual plane.\n- Keep surrogate code units and ASCII DEL escaped on the opt-in path so payloads remain transport-safe.\n- Verify existing escaped payloads remain readable and assess persisted/replayed payload compatibility.\n- Expose one consistent, startup-configured provider option across clients, workers, activities, entities, and blueprints, including large-payload externalization paths.\n- Preserve the existing ASCII-escaped encoding when no opt-in is requested; require the released shared-SDK minimum and report unsupported opt-in configuration explicitly.\n- Demonstrate serialized UTF-8 size reduction without attributing unrelated request overhead to escaping.\n- Document configuration scope, byte-dependent compatibility concerns, upstream release coordination, and the need for real-host staging validation.\n\nSee #269 for the full report. This is independent of large-payload externalization: reduced escaping does not eliminate aggregate request limits.\n\r\n","author":{"url":"https://github.com/berndverst","@type":"Person","name":"berndverst"},"datePublished":"2026-10-12T20:12:13Z","url":"https://github.com/microsoft/durabletask-python/issues/291"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:d7f795cd-8caa-d875-4f9c-08044a662046 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 8B86:83EB9:71DED:98322:6AFE2CD6 |
| html-safe-nonce | fe0cccb3d1fb166293d4bc9951e01c200ae15c9f951b1034295067f003c12a19 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4Qjg2OjgzRUI5OjcxREVEOjk4MzIyOjZBRkUyQ0Q2IiwidmlzaXRvcl9pZCI6IjQ0NDg0NDM4NDgxMjg4Njc1NDIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | aee0650286d0b49298e278356485f93476ccaceb00ba8e1f75f23f5818df62c0 |
| hovercard-subject-tag | issue:5819814334 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/durabletask-python/291/issue_layout |
| twitter:image | https://opengraph.githubassets.com/01e4cf1649118516d5e6083891f43a9114e43aab5bc6520b7b2e5868117b39f9/microsoft/durabletask-python/issues/291 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/01e4cf1649118516d5e6083891f43a9114e43aab5bc6520b7b2e5868117b39f9/microsoft/durabletask-python/issues/291 |
| og:image:alt | Parent: #269 — ask 4. Problem The reporter's activity results contained 2.55 MB of UTF-8 data, mostly Cyrillic, while the rejected orchestration completion request was 11.05 MB. The report identifi... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | berndverst |
| hostname | github.com |
| expected-hostname | github.com |
| None | e973f9ec3625f3faad8cac5c2d11a214b7ffe45f3cc563b828459b290ecaeed3 |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/durabletask-python git https://github.com/microsoft/durabletask-python.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 627664515 |
| octolytics-dimension-repository_nwo | microsoft/durabletask-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 627664515 |
| octolytics-dimension-repository_network_root_nwo | microsoft/durabletask-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 1f92c367595b9190e0be293896078033886b1913 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width