Title: feat: Add read_from and client_az options to the Redis GLIDE client by NopeCharon · Pull Request #7093 · feast-dev/feast · GitHub
Open Graph Title: feat: Add read_from and client_az options to the Redis GLIDE client by NopeCharon · Pull Request #7093 · feast-dev/feast
X Title: feat: Add read_from and client_az options to the Redis GLIDE client by NopeCharon · Pull Request #7093 · feast-dev/feast
Description: What this PR does / why we need it: The GLIDE client from #6870 always reads from the primary: the Feast config has no way to pass GLIDE's read strategy through, so clusters with replicas can't spread online reads across them. This adds two RedisOnlineStoreConfig options that map directly onto GLIDE's own settings: read_from: primary (default), prefer_replica, az_affinity, az_affinity_replicas_and_primary or all_nodes, translated to glide_sync.ReadFrom. client_az: the client's availability zone, used by the az_affinity strategies. online_store: type: redis redis_type: redis_cluster connection_string: "redis1:6379,redis2:6379" client: glide read_from: az_affinity client_az: us-east-1a I used dedicated options rather than connection_string parameters, as the issue suggests. The reason is that connection_string parameters are also passed as kwargs to redis-py, which still handles writes and async reads, and redis-py would reject them. The config is validated at load time, so a setting is never silently ignored: read_from and client_az require client: glide, since redis-py doesn't use them. A read_from other than primary requires redis_type: redis_cluster. A standalone GLIDE client sends every batch to the primary whatever read_from is, so the batched HMGET read would ignore the strategy. GLIDE's standalone BatchOptions has no routing option. I checked this against a real primary + replica: with PREFER_REPLICA, single hmget calls went to the replica but exec(Batch(...)) still went to the primary. The az_affinity strategies require client_az. GLIDE also enforces this, but only when the client is created. Docs: docs/reference/online-stores/redis.md gets a "Reading from replicas" section and two new rows in the configuration reference table. Which issue(s) this PR fixes: Fixes #7086 Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests Integration tests Manual tests Testing is not required for this change Unit tests (tests/unit/infra/online_store/test_redis.py) cover the default, the three validation rules, and the mapping of every read_from value onto glide_sync.ReadFrom and client_az. Manual test: I ran RedisOnlineStore.online_write_batch + online_read with client: glide against a Valkey 8 cluster with one primary (availability-zone zone-a) and one replica (zone-b). For each config I counted HMGET calls per node from INFO commandstats over one read of 3 entities: config primary replica default (primary) 3 0 prefer_replica 0 3 all_nodes 2 1 az_affinity, client_az: zone-b 0 3 az_affinity, client_az: zone-a (no replica in zone, falls back to other replicas) 0 3 az_affinity_replicas_and_primary, client_az: zone-a 3 0 All reads returned the written values. Misc
Open Graph Description: What this PR does / why we need it: The GLIDE client from #6870 always reads from the primary: the Feast config has no way to pass GLIDE's read strategy through, so clusters with replicas can...
X Description: What this PR does / why we need it: The GLIDE client from #6870 always reads from the primary: the Feast config has no way to pass GLIDE's read strategy through, so clusters with replicas c...
Opengraph URL: https://github.com/feast-dev/feast/pull/7093
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:e99bdb33-e667-8588-e709-ba5f6cdb454b |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 99BE:3DF67A:F65238:143B1C2:6AFE9894 |
| html-safe-nonce | 0535225be1b50f0ba2a0b0b25f9d24299dad43a21a4c0229fcc35538610e5835 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5OUJFOjNERjY3QTpGNjUyMzg6MTQzQjFDMjo2QUZFOTg5NCIsInZpc2l0b3JfaWQiOiI1Nzc5MDAyODA3ODAxNTE4MjI4IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | c354b59fe5b06efdf419dbb1276a8f4756531ae967ac37155ab625723eea7250 |
| hovercard-subject-tag | pull_request:5079472030 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/7093/files |
| twitter:image | https://avatars.githubusercontent.com/u/115794254?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/115794254?s=400&v=4 |
| og:image:alt | What this PR does / why we need it: The GLIDE client from #6870 always reads from the primary: the Feast config has no way to pass GLIDE's read strategy through, so clusters with replicas can... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width