René's URL Explorer Experiment


Title: Performance [P2]: Reduce orchestration history replay allocations by berndverst · Pull Request #204 · microsoft/durabletask-python · GitHub

Open Graph Title: Performance [P2]: Reduce orchestration history replay allocations by berndverst · Pull Request #204 · microsoft/durabletask-python

X Title: Performance [P2]: Reduce orchestration history replay allocations by berndverst · Pull Request #204 · microsoft/durabletask-python

Description: Summary Long orchestration histories were copied and rescanned several times per work item, before and during replay. Each of these allocated throwaway lists sized by the full history, which is pure GC pressure and cache churn on every single work item. This PR removes those copies without changing any behavior. Changes 1. TaskHubGrpcWorker._execute_orchestrator() Built list(req.pastEvents) + list(req.newEvents) — three throwaway lists sized by the entire history — purely to locate the first executionStarted event and then break. Now iterates itertools.chain(req.pastEvents, req.newEvents), which allocates a constant-size iterator instead. Iteration order (past events, then new events) and the break on first match are unchanged. 2. _OrchestrationExecutor.execute() Materialized every executionStarted event in the committed history via a list comprehension, then read only element [0]: orchestration_started_events = [e for e in old_events if e.HasField("executionStarted")] if len(orchestration_started_events) >= 1: orchestration_name = orchestration_started_events[0].executionStarted.name Replaced with next(, ""), which short-circuits at the first match instead of scanning and collecting the whole history. The resolved name is identical in both branches: first match when one exists, "" otherwise. 3. _OrchestrationExecutor._build_rewind_result() Built a combined all_events = list(old_events) + list(new_events) to feed its two passes. Both parameters are already typed Sequence[pb.HistoryEvent] (and the function already relies on that via len()/indexing), so each pass now chains them lazily with its own fresh iterator. Order is unchanged (old events first, then new events), and the first pass fully completes before the second begins — no generator-exhaustion hazard. Deliberately not changed The rewind detection in execute() already short-circuits correctly: has_rewind_in_new = any(e.HasField("executionRewound") for e in new_events) if has_rewind_in_new and any(e.HasField("executionCompleted") for e in old_events): Both are lazy generators, and the and means old_events is only scanned when a rewind is actually in flight. Fusing the executionCompleted scan into the executionStarted scan — as the issue's proposed solution hints at — would force a full O(N) walk of the committed history on every normal work item, making the common path measurably worse. Left as-is on purpose. Compatibility No public API signature changes, no renames, no removed symbols, no changed import paths. Replay order, rewind handling, non-determinism detection, log messages, and exception types/messages are all preserved. This is an allocation-only change with no observable behavior difference, so per the repo's changelog policy it does not get a CHANGELOG.md entry. Tests Added two regression tests in tests/durabletask/test_orchestration_executor.py covering both branches of the orchestration-name lookup, which previously had no direct coverage: test_orchestration_name_resolved_from_committed_history — name is taken from the first executionStarted event when it is not the first event in the committed history. test_orchestration_name_unknown_without_committed_history — falls back to when the committed history carries no executionStarted event. Verification All run against a session-local venv with both packages installed editable from this worktree. Check Baseline (before) After pytest tests/durabletask -q --ignore-glob="*_e2e.py" 680 passed, 7 skipped, 1 failed 683 passed, 7 skipped, 0 failed pytest tests/durabletask-azuremanaged/{test_sandboxes_extension,test_durabletask_grpc_interceptor,test_azuremanaged_grpc_resiliency}.py — 45 passed flake8 durabletask/worker.py tests/durabletask/test_orchestration_executor.py — clean pyright durabletask/worker.py 0 errors 0 errors Notes on the baseline: The single baseline failure was test_client.py::test_sync_client_recreate_cooldown_prevents_immediate_repeated_recreation, a timing-sensitive test unrelated to this change; it passes after the change too. tests/durabletask/test_orchestration_executor.py (171 tests incl. the executor and tracing suites) and tests/durabletask/test_build_rewind_result.py (the 13 rewind tests) pass in full — these are the primary guards for replay correctness and rewind behavior. pyright reports 0 errors on durabletask/worker.py both before and after. The new test diagnostics are the same reportPrivateUsage / reportUnknown* kinds the test file already produces 433 times, matching the established convention in test_build_rewind_result.py. The remaining tests/durabletask-azuremanaged files (test_dts_activity_sequence.py, test_dts_batch_actions.py, entities/test_dts_entity_failure_handling.py) require a live Durable Task Scheduler endpoint and fail identically on a clean baseline in this environment. Fixes #185

Open Graph Description: Summary Long orchestration histories were copied and rescanned several times per work item, before and during replay. Each of these allocated throwaway lists sized by the full history, which is pur...

X Description: Summary Long orchestration histories were copied and rescanned several times per work item, before and during replay. Each of these allocated throwaway lists sized by the full history, which is pur...

Opengraph URL: https://github.com/microsoft/durabletask-python/pull/204

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:8d8af330-296d-74cf-34f0-8ae53fb69abb
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idA048:AF0BD:F97A3:146F79:6AFE44BE
html-safe-nonce2cb9e5e214bde2dd2da696d7ee6b45a1478f8fcd78b1719f9de3bda7e60e7b8f
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMDQ4OkFGMEJEOkY5N0EzOjE0NkY3OTo2QUZFNDRCRSIsInZpc2l0b3JfaWQiOiI4MTI1MjgwOTI0MDYyMzM2MTkwIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac572988ef12bdebcfa2aae8e6575bca59e393ea3bb71da8f19ec85bcd80eace26
hovercard-subject-tagpull_request:4139788785
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/microsoft/durabletask-python/pull/204/files
twitter:imagehttps://avatars.githubusercontent.com/u/4535280?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/4535280?s=400&v=4
og:image:altSummary Long orchestration histories were copied and rescanned several times per work item, before and during replay. Each of these allocated throwaway lists sized by the full history, which is pur...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Nonef1a5901e2377b0af2a0b303f9528678f1475d04b9219dca1e993344196eaa07c
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/microsoft/durabletask-python git https://github.com/microsoft/durabletask-python.git
octolytics-dimension-user_id6154722
octolytics-dimension-user_loginmicrosoft
octolytics-dimension-repository_id627664515
octolytics-dimension-repository_nwomicrosoft/durabletask-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id627664515
octolytics-dimension-repository_network_root_nwomicrosoft/durabletask-python
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasece393ea11ad276623b946bc9663d3e7d6f564275
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/microsoft/durabletask-python/pull/204/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fdurabletask-python%2Fpull%2F204%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fdurabletask-python%2Fpull%2F204%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=microsoft%2Fdurabletask-python
Reloadhttps://github.com/microsoft/durabletask-python/pull/204/files
Reloadhttps://github.com/microsoft/durabletask-python/pull/204/files
Reloadhttps://github.com/microsoft/durabletask-python/pull/204/files
microsofthttps://github.com/microsoft
durabletask-pythonhttps://github.com/microsoft/durabletask-python
Codehttps://github.com/microsoft/durabletask-python
Issues15 (15)https://github.com/microsoft/durabletask-python/issues
Pull requests12 (12)https://github.com/microsoft/durabletask-python/pulls
Actionshttps://github.com/microsoft/durabletask-python/actions
Projectshttps://github.com/microsoft/durabletask-python/projects
Security and qualityhttps://github.com/microsoft/durabletask-python/security
Insightshttps://github.com/microsoft/durabletask-python/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fmicrosoft%2Fdurabletask-python%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fmicrosoft%2Fdurabletask-python%2Fissues%2Fnew%2Fchoose
Bernd Verst (berndverst)https://github.com/berndverst
mainhttps://github.com/microsoft/durabletask-python/tree/main
berndverst-reduce-history-replay-allocationshttps://github.com/microsoft/durabletask-python/tree/berndverst-reduce-history-replay-allocations
Conversation 1 https://github.com/microsoft/durabletask-python/pull/204
Commits 1 https://github.com/microsoft/durabletask-python/pull/204/commits
Checks 18 https://github.com/microsoft/durabletask-python/pull/204/checks
Files changed https://github.com/microsoft/durabletask-python/pull/204/files
Please reload this pagehttps://github.com/microsoft/durabletask-python/pull/204/files
Performance [P2]: Reduce orchestration history replay allocations https://github.com/microsoft/durabletask-python/pull/204/files#top
Show all changes 1 commit https://github.com/microsoft/durabletask-python/pull/204/files
8ea44a2 Performance: reduce orchestration history replay allocations Jul 27, 2026 https://github.com/microsoft/durabletask-python/pull/204/commits/8ea44a26b022c43794877fb0419650fb5a97f289
Clear filters https://github.com/microsoft/durabletask-python/pull/204/files
Please reload this pagehttps://github.com/microsoft/durabletask-python/pull/204/files
Please reload this pagehttps://github.com/microsoft/durabletask-python/pull/204/files
worker.py https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
test_orchestration_executor.py https://github.com/microsoft/durabletask-python/pull/204/files#diff-3d6b06ac4db653818f8956dc727c6b691775880aca64ddd2e3342c04b8ec0b8e
durabletask/worker.pyhttps://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
View file https://github.com/microsoft/durabletask-python/blob/8ea44a26b022c43794877fb0419650fb5a97f289/durabletask/worker.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/microsoft/durabletask-python/pull/204/{{ revealButtonHref }}
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
https://github.com/microsoft/durabletask-python/pull/204/files#diff-821cca3b6d2d4c27b83bc2ea2c1dfbce058c46419967498ae26dc12693418899
tests/durabletask/test_orchestration_executor.pyhttps://github.com/microsoft/durabletask-python/pull/204/files#diff-3d6b06ac4db653818f8956dc727c6b691775880aca64ddd2e3342c04b8ec0b8e
View file https://github.com/microsoft/durabletask-python/blob/8ea44a26b022c43794877fb0419650fb5a97f289/tests/durabletask/test_orchestration_executor.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/microsoft/durabletask-python/pull/204/{{ revealButtonHref }}
https://github.com/microsoft/durabletask-python/pull/204/files#diff-3d6b06ac4db653818f8956dc727c6b691775880aca64ddd2e3342c04b8ec0b8e
https://github.com/microsoft/durabletask-python/pull/204/files#diff-3d6b06ac4db653818f8956dc727c6b691775880aca64ddd2e3342c04b8ec0b8e
Please reload this pagehttps://github.com/microsoft/durabletask-python/pull/204/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.