Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast · GitHub
Open Graph Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast
X Title: fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast
Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for every row's entity key, then pipe.execute() once. Write phase — loop the rows, skip any row whose timestamp is <= the stored one, otherwise queue an hset. Because the read phase completes before the first write is queued, rows in the same batch that share an entity key all compare against the same pre-batch value. None of them can see the others. When nothing is stored yet, prev_event_time is None for all of them, the guard is skipped entirely, and every row gets an hset queued. Redis applies pipelined commands in order, so the last row in list order wins — regardless of its event time. Pushing one entity key with timestamps in descending order therefore leaves the oldest value in the store: test_data = pd.DataFrame({ "customer_id": [1] * 3, "total_purchases": [30, 20, 10], "event_timestamp": [t3, t2, t1], # latest first }) store.push_features(fg_name, test_data, mode="online") # get_online_features returns 10, the value belonging to t1 No error and no warning — just a silently incorrect feature value. The fix Track the latest event timestamp queued so far per entity key within the batch, and compare each row against whichever is newer: the stored timestamp, or an earlier row of the same batch. latest_seen_nanos = max(prev_total_nanos, batch_latest_nanos.get(redis_key_bin, 0)) if latest_seen_nanos and new_total_nanos <= latest_seen_nanos: if progress: progress(1) continue batch_latest_nanos[redis_key_bin] = new_total_nanos redis_key_bin is already computed in the read phase and uniquely identifies (project, entity_key), so it was free to reuse as the dedup key. The guard had to move out of the if prev_event_time: block, with prev_total_nanos initialised to 0 so the max() works when nothing is stored yet — that's the structural part of the diff. online_write_batch_async had the identical defect and gets the same fix; fixing only the sync path would have left half the bug in place. The skip_dedup fast path is deliberately untouched. Its own comment says it is "suitable for initial loads or append-only pipelines where out-of-order writes are not a concern," so losing ordering there is the documented tradeoff of enabling it. One design note worth your input The issue suggests sorting the batch by timestamp, or reducing to one row per key. Either would work, but both change behaviour for batches that are already correct — they alter how many writes get queued in the ascending case and shift the progress() accounting. I went with the running-max approach because it repairs only the broken case and leaves every already-working case byte-identical. Happy to switch to sort-or-reduce if you'd rather have the write-amplification reduction too; it's a small change from here. Behaviour that is intentionally preserved: Rows older than the value already stored are still skipped. Ties within a batch keep the first row, matching the existing <= "older or same instant" semantics. A stored timestamp of epoch 0 still doesn't trigger a skip — the original required a truthy prev_total_nanos, and max(0, ...) preserves that. Note on the labels The wontfix label on #5163 was applied by stale[bot] as its configured staleLabel, not by a maintainer declining the report — the timeline shows the bot's comment as the only activity on the issue. Flagging it so it doesn't read as a prior decision. kind/bug and priority/p2 are the defaults from bug_report.md. If this is out of scope for other reasons, I'm glad to hear it. Which issue(s) this PR fixes: Fixes #5163 Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests Integration tests Manual tests Testing is not required for this change Misc Five tests added to sdk/python/tests/unit/infra/online_store/test_redis.py, reusing the existing MagicMock pipeline idiom, so no Redis server or Docker is needed: Test Before fix After fix keeps_latest_event_time_within_batch[descending] fails passes keeps_latest_event_time_within_batch[unordered] fails passes keeps_latest_event_time_within_batch[ascending] passes passes async_keeps_latest_event_time_within_batch fails passes still_skips_rows_older_than_stored_value passes passes The two that pass before the fix are the load-bearing ones — they're there to show the change repairs the broken orderings without disturbing the ascending case or the existing staleness guard. The pre-fix failure is assert b'\x18\n' == b'\x18\x1e', i.e. int32_val=10 where 30 was expected — the reporter's "returns 10 instead of 30", reproduced deterministically. test_redis.py goes 18 → 23 passing. ruff check, ruff format --check and mypy feast/infra/online_stores/redis.py are all clean. I ran unit tests only, not the integration suite — it needs cloud credentials I don't have. Happy to add an integration test if you'd like one for this path. One small correction to the issue for the record: it describes the code as skipping "records with timestamps older than what's already been processed in the current batch." Nothing intra-batch is tracked today, which is exactly why the bug exists — but the symptom as reported is accurate.
Open Graph Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev...
X Description: What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev...
Opengraph URL: https://github.com/feast-dev/feast/pull/6656
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:7b50ff11-bcbc-9c50-2c62-a48505f24394 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 87FC:1DD5B3:3078B1:434FF6:6AFE99CF |
| html-safe-nonce | 1ebef01ad25747b128b9e82a36eabb1f669e1c27f5dbf0bae2f0b5f6cb532ee8 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4N0ZDOjFERDVCMzozMDc4QjE6NDM0RkY2OjZBRkU5OUNGIiwidmlzaXRvcl9pZCI6IjYwMDMzNTkxMTc1MjE4ODU2NDciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | e6189b8c4590ae3d16b9cffed195d3ff6a7668f06da13c99432cca51e5995547 |
| hovercard-subject-tag | pull_request:4159764238 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6656/files |
| twitter:image | https://avatars.githubusercontent.com/u/24850536?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/24850536?s=400&v=4 |
| og:image:alt | What this PR does / why we need it: online_write_batch performs its staleness check against a snapshot that is read before any write is queued: Read phase — hmget the stored event timestamp for ev... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width