Title: Redis sink flushes only rows that have more recent eventTimestamp by pyalex · Pull Request #913 · feast-dev/feast · GitHub
Open Graph Title: Redis sink flushes only rows that have more recent eventTimestamp by pyalex · Pull Request #913 · feast-dev/feast
X Title: Redis sink flushes only rows that have more recent eventTimestamp by pyalex · Pull Request #913 · feast-dev/feast
Description: What this PR does / why we need it: Since we keep only one row per entity in Redis storage previously it was unpredictable which row will be written, especially in batch ingestion. In this PR we read existing value from redis first to compare eventTimestamps and only if row from stream has more recent eventTimestamp it will be written to store. This guarantees that more recent row won't be overwritten with older value. Now redis will have the latest value for entity. Also, I refactored tests for redis sink to remove duplicated parts of code and use @Parametrized tests instead. Which issue(s) this PR fixes: Fixes # Does this PR introduce a user-facing change?: Redis sink has now more predictable order of writes. Value is being overwritten only by more recent row from stream.
Open Graph Description: What this PR does / why we need it: Since we keep only one row per entity in Redis storage previously it was unpredictable which row will be written, especially in batch ingestion. In this PR we re...
X Description: What this PR does / why we need it: Since we keep only one row per entity in Redis storage previously it was unpredictable which row will be written, especially in batch ingestion. In this PR we re...
Opengraph URL: https://github.com/feast-dev/feast/pull/913
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/checks(.:format) |
| route-controller | pull_requests |
| route-action | checks |
| fetch-nonce | v2:538d40e8-c1f1-5e2a-5074-f26050aec055 |
| current-catalog-service-hash | 87dc3bc62d9b466312751bfd5f889726f4f1337bdff4e8be7da7c93d6c00a25a |
| request-id | B2CA:F747F:13EA1:189A1:697DE1DD |
| html-safe-nonce | 173e6674ba1e1fb472d2dda137aee6c6314d427becb73170696767e4d3a47399 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMkNBOkY3NDdGOjEzRUExOjE4OUExOjY5N0RFMUREIiwidmlzaXRvcl9pZCI6IjYyODg1NTg0MzIxNDgzODIxNzMiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 700672816924db8cbbcf258b5e161a94a0e237ff19c2ff82c7beb2f883f45672 |
| hovercard-subject-tag | pull_request:460716776 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,checks,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/913/checks |
| twitter:image | https://avatars.githubusercontent.com/u/1303659?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/1303659?s=400&v=4 |
| og:image:alt | What this PR does / why we need it: Since we keep only one row per entity in Redis storage previously it was unpredictable which row will be written, especially in batch ingestion. In this PR we re... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 60279d4097367e16897439d16d6bbe4180663db828c666eeed2656988ffe59f6 |
| turbo-cache-control | no-preview |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width full-width-p-0 |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 7c85641c598ad130c74f7bcc27f58575cac69551 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width