René's URL Explorer Experiment


Title: Redis Online Store adapter is not production-ready · Issue #3596 · feast-dev/feast · GitHub

Open Graph Title: Redis Online Store adapter is not production-ready · Issue #3596 · feast-dev/feast

X Title: Redis Online Store adapter is not production-ready · Issue #3596 · feast-dev/feast

Description: Purpose of this issue I'm opening this issue to bring to light some concerns and potential areas for improvement in the Redis Online Store adapter for the whole community. I understand that the Feast team has other priorities, and the co...

Open Graph Description: Purpose of this issue I'm opening this issue to bring to light some concerns and potential areas for improvement in the Redis Online Store adapter for the whole community. I understand that the Fea...

X Description: Purpose of this issue I'm opening this issue to bring to light some concerns and potential areas for improvement in the Redis Online Store adapter for the whole community. I understand that the...

Opengraph URL: https://github.com/feast-dev/feast/issues/3596

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Redis Online Store adapter is not production-ready","articleBody":"## Purpose of this issue\r\n\r\nI'm opening this issue to bring to light some concerns and potential areas for improvement in the Redis Online Store adapter for the whole community. I understand that the Feast team has other priorities, and the community is also encouraged to contribute. My main goal is to make this information more clear for existing and future users, so they can make informed decisions based on the documentation and avoid potential pitfalls (that my team was not able to avoid prior to start using Feast with Redis).\r\n\r\n## Expected Behavior\r\n\r\nRedis is positioned as a well-supported Online Store within Feast, with a [dedicated blog post](https://redis.com/blog/building-feature-stores-with-redis-introduction-to-feast-with-redis/), being used [in performance assessment](https://feast.dev/blog/feast-benchmarks/), and boasting the [most complete set of functionalities](https://docs.feast.dev/reference/online-stores/overview#functionality-matrix).\r\n\r\n## Current Behavior\r\n\r\nHowever, it seems that the current implementation of Redis' Online Store has some issues and lacks certain basic functionalities, even those mentioned in the functionality matrix.\r\n\r\nHere is a list of concerns:\r\n\r\n1. While, in theory, feature data is \"collocated by entity key\", in practice it works as if it were \"collocated by feature view\". This is due to `get_online_features()` performing [one `read_from_online_store` per feature view in your request](https://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/feature_store.py#L1736). This implies that the more feature views you request, the longer `get_online_features()` will take, regardless of key collocation. Additionally, all database reads are synchronous.\r\n\r\n2. Although, in theory, it \"supports TTL (time to live) at retrieval\", Feature View TTL doesn't seem to matter in practice. Instead, you have to set up [a store-level TTL](https://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/online_stores/redis.py#L72) that is only used to expire whole entities. In complex production feature store, with several feature views for the same entity and different TTL requirements, this is never going to be useful. As a result, you can get features older than TTL during online feature retrieval with Redis. [Here](https://github.com/feast-dev/feast/issues/1802) is a \"closed\" and marked as \"done\" issue mentioning this.\r\n\r\n3. In theory, the online store should \"generate a plan and update infrastructure (e.g. tables)\". However, in practice, renaming or deleting features and feature views will leave them stored in Redis, causing it to grow indefinitely. [Feast will only delete data from Redis when the last features/feature view of a specific entity is not in use anymore](https://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/online_stores/redis.py#L116). [Here](https://github.com/feast-dev/feast/issues/2254) is a \"closed\" issue mentioning this. This is not an issue when using `key_ttl_seconds`, but in complex production-level feature stores, this may not be a viable solution.\r\n\r\n## Possible Solution\r\n\r\nSwitching to another online store (like Dynamo or Cassandra) can solve some of the issues, but not all. The performance of Redis itself is not in question, but the Feast implementation may need improvements.\r\n\r\nSome possible fixes include:\r\n\r\n1. I see no reason for all online stores to not be \"collocated by entity key\". I really like this about Feast's Redis data model. In offline stores, feature views have conflicting timestamps and different historical sources. But in online stores, you want to only keep the latest values (like a common dimension in a warehouse), so all feature views that share the entities can be stored in one table/hash/whatever. It would speed up online feature retrieval across all online stores. Cassandra, for example, could very much implement the same data model for column families. But Feast itself doesn't make use of the better data model in Redis.\r\n\r\n2. Addressing the conflict between [Redis' inability to allow EXPIRE (or ttl) in hash child keys](https://stackoverflow.com/questions/16545321/how-to-expire-the-hset-child-key-in-redis) and the \"collocated by entity key\" design. One basic principle in Feast is that if you request a feature from the online store and its value is older than its TTL, you should get a null. Workarounds may be needed to make Redis a more suitable data store for Online Stores in this regard. Cassandra, for example, [has TTL for both table and individual columns](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useExpireExample.html).\r\n\r\n3. Implementing the use of `HDEL` for infrastructure updates when feature views are renamed/deleted. For individual features, the change might be more substantial, as [`update_infra()` currently only has access to feature views and entities to create/delete](https://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/provider.py#L43). Further exploration of this issue is necessary.\r\n\r\nI hope that addressing these concerns can lead to a more robust and reliable implementation of the Redis Online Store adapter in Feast, or even improve Feast as a whole. As a heavy user, I would prioritize these changes over other integrations, and I believe I'm likely not the only one. Strengthening the Redis integration and enhancing Feast's performance will undoubtedly benefit the wider community.\r\n","author":{"url":"https://github.com/roelschr","@type":"Person","name":"roelschr"},"datePublished":"2023-04-13T10:25:14Z","url":"https://github.com/feast-dev/feast/issues/3596"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:cfd8c299-ee34-7973-1c00-026d5c2c9b2e
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id99BC:79CF9:309C19:409010:6AFE980F
html-safe-nonce5e83252e4405dc9615d93513728cf1fa5fdc74736d9bbe7cfbe89170f086071e
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5OUJDOjc5Q0Y5OjMwOUMxOTo0MDkwMTA6NkFGRTk4MEYiLCJ2aXNpdG9yX2lkIjoiNzQ1NDgwNzEzMDEwNTYxNjM5OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac99de5d60066640e18e617fa8697d767de12122d98d9c3fefc3575fe76496c0c3
hovercard-subject-tagissue:1666145845
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/3596/issue_layout
twitter:imagehttps://opengraph.githubassets.com/ac0bd7a510c9c986178d0b26681810dec9eb7d9c009049e0f1b471f87d46c869/feast-dev/feast/issues/3596
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/ac0bd7a510c9c986178d0b26681810dec9eb7d9c009049e0f1b471f87d46c869/feast-dev/feast/issues/3596
og:image:altPurpose of this issue I'm opening this issue to bring to light some concerns and potential areas for improvement in the Redis Online Store adapter for the whole community. I understand that the Fea...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameroelschr
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/feast-dev/feast/issues/3596#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F3596
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F3596
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/issues/3596
Reloadhttps://github.com/feast-dev/feast/issues/3596
Reloadhttps://github.com/feast-dev/feast/issues/3596
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
https://github.com/feast-dev/feast
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/3596
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/3596
#6337https://github.com/feast-dev/feast/pull/6337
Redis Online Store adapter is not production-readyhttps://github.com/feast-dev/feast/issues/3596#top
#6337https://github.com/feast-dev/feast/pull/6337
https://github.com/DvirDukhan
keep-openhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22keep-open%22
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
https://github.com/roelschr
roelschrhttps://github.com/roelschr
on Apr 13, 2023https://github.com/feast-dev/feast/issues/3596#issue-1666145845
dedicated blog posthttps://redis.com/blog/building-feature-stores-with-redis-introduction-to-feast-with-redis/
in performance assessmenthttps://feast.dev/blog/feast-benchmarks/
most complete set of functionalitieshttps://docs.feast.dev/reference/online-stores/overview#functionality-matrix
one read_from_online_store per feature view in your requesthttps://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/feature_store.py#L1736
a store-level TTLhttps://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/online_stores/redis.py#L72
Herehttps://github.com/feast-dev/feast/issues/1802
Feast will only delete data from Redis when the last features/feature view of a specific entity is not in use anymorehttps://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/online_stores/redis.py#L116
Herehttps://github.com/feast-dev/feast/issues/2254
Redis' inability to allow EXPIRE (or ttl) in hash child keyshttps://stackoverflow.com/questions/16545321/how-to-expire-the-hset-child-key-in-redis
has TTL for both table and individual columnshttps://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useExpireExample.html
update_infra() currently only has access to feature views and entities to create/deletehttps://github.com/feast-dev/feast/blob/v0.30.0/sdk/python/feast/infra/provider.py#L43
roelschrhttps://github.com/roelschr
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
on Apr 13, 2023https://github.com/feast-dev/feast/issues/3596#event-8995263175
https://github.com/adchia
adchiahttps://github.com/adchia
on Apr 21, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1518007176
@DvirDukhanhttps://github.com/DvirDukhan
https://github.com/DvirDukhan
DvirDukhanhttps://github.com/DvirDukhan
on Apr 24, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1520876348
@adchiahttps://github.com/adchia
@roelschrhttps://github.com/roelschr
#2254 (comment)https://github.com/feast-dev/feast/issues/2254#issue-1117083681
https://github.com/judahrand
judahrandhttps://github.com/judahrand
on Apr 25, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1521553010
Redis' Server Side Assisted Client Side Cachinghttps://redis.io/docs/manual/client-side-caching/
https://github.com/rodrigo-christiansen-wonolo
rodrigo-christiansen-wonolohttps://github.com/rodrigo-christiansen-wonolo
on Apr 26, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1524056319
kevjumbahttps://github.com/kevjumba
DvirDukhanhttps://github.com/DvirDukhan
on May 3, 2023https://github.com/feast-dev/feast/issues/3596#event-9157348105
https://github.com/judahrand
judahrandhttps://github.com/judahrand
on May 4, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1534272442
@adchiahttps://github.com/adchia
@roelschrhttps://github.com/roelschr
#2254 (comment)https://github.com/feast-dev/feast/issues/2254#issue-1117083681
https://redis.com/blog/redis-clustering-best-practices-with-keys/https://redis.com/blog/redis-clustering-best-practices-with-keys/
https://github.com/roelschr
roelschrhttps://github.com/roelschr
on May 10, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1542210009
@DvirDukhanhttps://github.com/DvirDukhan
@judahrandhttps://github.com/judahrand
https://github.com/judahrand
judahrandhttps://github.com/judahrand
on May 10, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1542232865
https://github.com/apps/stale
stalehttps://github.com/apps/stale
on Sep 17, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1722489132
stalehttps://github.com/apps/stale
wontfixThis will not be worked onhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22wontfix%22
on Sep 17, 2023https://github.com/feast-dev/feast/issues/3596#event-10391496493
https://github.com/sudohainguyen
sudohainguyenhttps://github.com/sudohainguyen
on Sep 22, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1730890433
stalehttps://github.com/apps/stale
wontfixThis will not be worked onhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22wontfix%22
on Sep 22, 2023https://github.com/feast-dev/feast/issues/3596#event-10443698981
https://github.com/sudohainguyen
sudohainguyenhttps://github.com/sudohainguyen
on Sep 25, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1732862805
@roelschrhttps://github.com/roelschr
https://github.com/rodrigo-christiansen-wonolo
rodrigo-christiansen-wonolohttps://github.com/rodrigo-christiansen-wonolo
on Sep 26, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1735807613
https://github.com/roelschr
roelschrhttps://github.com/roelschr
on Sep 27, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1737014837
@sudohainguyenhttps://github.com/sudohainguyen
through a loop over all feature views in the requesthttps://github.com/feast-dev/feast/blob/2192e6527fa10f1580e4dd8f350e05e45af981b7/sdk/python/feast/feature_store.py#L1727
https://github.com/sudohainguyen
sudohainguyenhttps://github.com/sudohainguyen
on Oct 12, 2023https://github.com/feast-dev/feast/issues/3596#issuecomment-1759381854
through a loop over all feature views in the requesthttps://github.com/feast-dev/feast/blob/2192e6527fa10f1580e4dd8f350e05e45af981b7/sdk/python/feast/feature_store.py#L1727
https://github.com/apps/stale
stalehttps://github.com/apps/stale
on Mar 17, 2024https://github.com/feast-dev/feast/issues/3596#issuecomment-2002413048
Stalehttps://github.com/probot/stale/pull/430
stalehttps://github.com/apps/stale
wontfixThis will not be worked onhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22wontfix%22
on Mar 17, 2024https://github.com/feast-dev/feast/issues/3596#event-12142633527
shuchuhttps://github.com/shuchu
on Apr 24, 2024https://github.com/feast-dev/feast/issues/3596#event-12584465886
stalehttps://github.com/apps/stale
wontfixThis will not be worked onhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22wontfix%22
on Apr 24, 2024https://github.com/feast-dev/feast/issues/3596#event-12584466347
shuchuhttps://github.com/shuchu
on Apr 24, 2024https://github.com/feast-dev/feast/issues/3596#event-12584470191
tokokohttps://github.com/tokoko
keep-openhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22keep-open%22
on Jul 16, 2024https://github.com/feast-dev/feast/issues/3596#event-13515386158
https://github.com/franciscojavierarceo
franciscojavierarceohttps://github.com/franciscojavierarceo
on Jan 14, 2026https://github.com/feast-dev/feast/issues/3596#issuecomment-3750256925
https://github.com/EXPEbdodla
EXPEbdodlahttps://github.com/EXPEbdodla
on Jan 14, 2026https://github.com/feast-dev/feast/issues/3596#issuecomment-3750268452
@franciscojavierarceohttps://github.com/franciscojavierarceo
https://github.com/franciscojavierarceo
franciscojavierarceohttps://github.com/franciscojavierarceo
on Jan 14, 2026https://github.com/feast-dev/feast/issues/3596#issuecomment-3750300313
@EXPEbdodlahttps://github.com/EXPEbdodla
https://github.com/franciscojavierarceo
franciscojavierarceohttps://github.com/franciscojavierarceo
on Jan 14, 2026https://github.com/feast-dev/feast/issues/3596#issuecomment-3750301249
@ntkatholehttps://github.com/ntkathole
ntkatholehttps://github.com/ntkathole
completedhttps://github.com/feast-dev/feast/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#6337https://github.com/feast-dev/feast/pull/6337
on Apr 30, 2026https://github.com/feast-dev/feast/issues/3596#event-25043829962
Sign up for freehttps://github.com/signup?return_to=https://github.com/feast-dev/feast/issues/3596
Sign in to commenthttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/3596
DvirDukhanhttps://github.com/DvirDukhan
keep-openhttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22keep-open%22
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.