René's URL Explorer Experiment


Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast · GitHub

Open Graph Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast

X Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast

Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for every row's entity key, then pipe.execute() once. Write phase — loop the rows, skip any row whose timestamp is <= the stored one, otherwise queue an hset. Because the read phase completes before the first write is queued, rows in the same batch that share an entity key all compare against the same pre-batch value. None of them can see the others. When nothing is stored yet, prev_event_time is None for all of them, the guard is skipped entirely, and every row gets an hset queued. Redis applies pipelined commands in order, so the last row in list order wins — regardless of its event time. Pushing one entity key with timestamps in descending order therefore leaves the oldest value in the store: test_data = pd.DataFrame({ "customer_id": [1] * 3, "total_purchases": [30, 20, 10], "event_timestamp": [t3, t2, t1], # latest first }) store.push_features(fg_name, test_data, mode="online") # get_online_features returns 10, the value belonging to t1 No error and no warning — just a silently incorrect feature value. The fix Track the latest event timestamp queued so far per entity key within the batch, and compare each row against whichever is newer: the stored timestamp, or an earlier row of the same batch. latest_seen_nanos = max(prev_total_nanos, batch_latest_nanos.get(redis_key_bin, 0)) if latest_seen_nanos and new_total_nanos <= latest_seen_nanos: if progress: progress(1) continue batch_latest_nanos[redis_key_bin] = new_total_nanos redis_key_bin is already computed in the read phase and uniquely identifies (project, entity_key), so it was free to reuse as the dedup key. The guard had to move out of the if prev_event_time: block, with prev_total_nanos initialised to 0 so the max() works when nothing is stored yet — that's the structural part of the diff. online_write_batch_async had the identical defect and gets the same fix; fixing only the sync path would have left half the bug in place. The skip_dedup fast path is deliberately untouched. Its own comment says it is "suitable for initial loads or append-only pipelines where out-of-order writes are not a concern," so losing ordering there is the documented tradeoff of enabling it. One design note worth your input The issue suggests sorting the batch by timestamp, or reducing to one row per key. Either would work, but both change behaviour for batches that are already correct — they alter how many writes get queued in the ascending case and shift the progress() accounting. I went with the running-max approach because it repairs only the broken case and leaves every already-working case byte-identical. Happy to switch to sort-or-reduce if you'd rather have the write-amplification reduction too; it's a small change from here. Behaviour that is intentionally preserved: Rows older than the value already stored are still skipped. Ties within a batch keep the first row, matching the existing <= "older or same instant" semantics. A stored timestamp of epoch 0 still doesn't trigger a skip — the original required a truthy prev_total_nanos, and max(0, ...) preserves that. Note on the labels The wontfix label on #5163 was applied by stale[bot] as its configured staleLabel, not by a maintainer declining the report — the timeline shows the bot's comment as the only activity on the issue. Flagging it so it doesn't read as a prior decision. kind/bug and priority/p2 are the defaults from bug_report.md. If this is out of scope for other reasons, I'm glad to hear it. Which issue(s) this PR fixes: Fixes #5163 Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests Integration tests Manual tests Testing is not required for this change Misc Five tests added to sdk/python/tests/unit/infra/online_store/test_redis.py, reusing the existing MagicMock pipeline idiom, so no Redis server or Docker is needed: Test Before fix After fix keeps_latest_event_time_within_batch[descending] fails passes keeps_latest_event_time_within_batch[unordered] fails passes keeps_latest_event_time_within_batch[ascending] passes passes async_keeps_latest_event_time_within_batch fails passes still_skips_rows_older_than_stored_value passes passes The two that pass before the fix are the load-bearing ones — they're there to show the change repairs the broken orderings without disturbing the ascending case or the existing staleness guard. The pre-fix failure is assert b'\x18\n' == b'\x18\x1e', i.e. int32_val=10 where 30 was expected — the reporter's "returns 10 instead of 30", reproduced deterministically. test_redis.py goes 18 → 23 passing. ruff check, ruff format --check and mypy feast/infra/online_stores/redis.py are all clean. I ran unit tests only, not the integration suite — it needs cloud credentials I don't have. Happy to add an integration test if you'd like one for this path. One small correction to the issue for the record: it describes the code as skipping "records with timestamps older than what's already been processed in the current batch." Nothing intra-batch is tracked today, which is exactly why the bug exists — but the symptom as reported is accurate.

Open Graph Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev...

X Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev...

Opengraph URL: https://github.com/feast-dev/feast/pull/6656

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:7b50ff11-bcbc-9c50-2c62-a48505f24394
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-id87FC:1DD5B3:3078B1:434FF6:6AFE99CF
html-safe-nonce1ebef01ad25747b128b9e82a36eabb1f669e1c27f5dbf0bae2f0b5f6cb532ee8
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4N0ZDOjFERDVCMzozMDc4QjE6NDM0RkY2OjZBRkU5OUNGIiwidmlzaXRvcl9pZCI6IjYwMDMzNTkxMTc1MjE4ODU2NDciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmace6189b8c4590ae3d16b9cffed195d3ff6a7668f06da13c99432cca51e5995547
hovercard-subject-tagpull_request:4159764238
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6656/files
twitter:imagehttps://avatars.githubusercontent.com/u/24850536?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/24850536?s=400&v=4
og:image:altWhat this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/feast-dev/feast/pull/6656/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F6656%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F6656%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/pull/6656/files
Reloadhttps://github.com/feast-dev/feast/pull/6656/files
Reloadhttps://github.com/feast-dev/feast/pull/6656/files
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
Sign up for GitHub https://github.com/signup?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose
ntkatholehttps://github.com/ntkathole
feast-dev:masterhttps://github.com/feast-dev/feast/tree/master
adarshsm:fix/redis-batch-event-time-orderinghttps://github.com/adarshsm/feast/tree/fix/redis-batch-event-time-ordering
Conversation 2 https://github.com/feast-dev/feast/pull/6656
Commits 1 https://github.com/feast-dev/feast/pull/6656/commits
Checks 32 https://github.com/feast-dev/feast/pull/6656/checks
Files changed https://github.com/feast-dev/feast/pull/6656/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6656/files
fix: Preserve event-time ordering within Redis online_write_batch https://github.com/feast-dev/feast/pull/6656/files#top
Show all changes 1 commit https://github.com/feast-dev/feast/pull/6656/files
edc9781 fix: Preserve event-time ordering within Redis online_write_batch adarshsm Jul 29, 2026 https://github.com/feast-dev/feast/pull/6656/commits/edc9781c50969d0b13fa747e9d91d09ee9b02293
Clear filters https://github.com/feast-dev/feast/pull/6656/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6656/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6656/files
redis.py https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
test_redis.py https://github.com/feast-dev/feast/pull/6656/files#diff-c44a5a05d39122961f5a1df8728cafebc6cd7679bdf02a71d42461c238fa637f
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/feast/infra/online_stores/redis.pyhttps://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
View file https://github.com/adarshsm/feast/blob/edc9781c50969d0b13fa747e9d91d09ee9b02293/sdk/python/feast/infra/online_stores/redis.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6656/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/pull/6656/files#diff-c2b25e2070b2be8b73b7574b51fd84470df80368d2746be111c1a3fb8793b2ee
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/tests/unit/infra/online_store/test_redis.pyhttps://github.com/feast-dev/feast/pull/6656/files#diff-c44a5a05d39122961f5a1df8728cafebc6cd7679bdf02a71d42461c238fa637f
View file https://github.com/adarshsm/feast/blob/edc9781c50969d0b13fa747e9d91d09ee9b02293/sdk/python/tests/unit/infra/online_store/test_redis.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6656/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6656/files#diff-c44a5a05d39122961f5a1df8728cafebc6cd7679bdf02a71d42461c238fa637f
https://github.com/feast-dev/feast/pull/6656/files#diff-c44a5a05d39122961f5a1df8728cafebc6cd7679bdf02a71d42461c238fa637f
https://github.com/feast-dev/feast/pull/6656/files#diff-c44a5a05d39122961f5a1df8728cafebc6cd7679bdf02a71d42461c238fa637f
Please reload this pagehttps://github.com/feast-dev/feast/pull/6656/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.