Title: TTL for entities in the Online stores other Than Redis · Issue #1988 · feast-dev/feast · GitHub
Open Graph Title: TTL for entities in the Online stores other Than Redis · Issue #1988 · feast-dev/feast
X Title: TTL for entities in the Online stores other Than Redis · Issue #1988 · feast-dev/feast
Description: Is your feature request related to a problem? Please describe. (for context: we are using the Redis online store) At the moment, once an entity is written to the Online store, it's there until it's manually cleaned up. In most of our bus...
Open Graph Description: Is your feature request related to a problem? Please describe. (for context: we are using the Redis online store) At the moment, once an entity is written to the Online store, it's there until it's...
X Description: Is your feature request related to a problem? Please describe. (for context: we are using the Redis online store) At the moment, once an entity is written to the Online store, it's there until ...
Opengraph URL: https://github.com/feast-dev/feast/issues/1988
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"TTL for entities in the Online stores other Than Redis","articleBody":"**Is your feature request related to a problem? Please describe.**\r\n(for context: we are using the Redis online store)\r\nAt the moment, once an entity is written to the Online store, it's there until it's manually cleaned up.\r\n\r\nIn most of our business cases, the Entity will have a natural expiration time (when it's no longer relevant for the use case).\r\n\r\nIt would be nice to be able to expire the entity in the online store, with some kind of TTL setting.\r\n\r\n**Describe the solution you'd like**\r\n1. in our local setup we end up expiring the key bin in Redis (combination of entity + project)\r\n```\r\ndef online_write_batch(...):\r\n redis_key_bin = _redis_key(project, entity_key)\r\n ...\r\n if table.ttl:\r\n redis_client_pipeline.expire(name=redis_key_bin, time=table.ttl)\r\n```\r\nthe issue with this is that the `ttl` param in `feature_view/table` is overloaded (initially designed for a use case in the offline store)\r\nAlso, in this type of setup and use case, the TTL should be on the entity level not `feature_view` because the Redis key bin is based on entity (feature_view ttl is designed for something else also)\r\n\r\nso the proposal:\r\nadd `online_store_ttl` param to the Entity object\r\nexample:\r\n```\r\ndriver_entity = Entity(\r\n name=\"driver\",\r\n description=\"Driver Identifier\", \r\n value_type=ValueType.INT64, \r\n online_store_ttl=Duration(seconds=86400)\r\n)\r\n```\r\n\r\n```\r\ndef online_write_batch(...):\r\n redis_key_bin = _redis_key(project, entity_key)\r\n ...\r\n if entity.online_store_ttl:\r\n redis_client_pipeline.expire(name=redis_key_bin, time=entity.online_store_ttl)\r\n```\r\n\r\nNote: implementation details may vary a bit based on the Online store type","author":{"url":"https://github.com/vas28r13","@type":"Person","name":"vas28r13"},"datePublished":"2021-11-01T18:18:08.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/1988/feast/issues/1988"}
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:ac365836-0ebc-b9cb-bd09-c1636923f8e4 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 85F8:33929D:2256778:30F1400:6978FCD3 |
| html-safe-nonce | 2231ea42b075f345f04b15730aed7bc08384a0daa6b58c99bc2e71ed46426f43 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4NUY4OjMzOTI5RDoyMjU2Nzc4OjMwRjE0MDA6Njk3OEZDRDMiLCJ2aXNpdG9yX2lkIjoiNDMwOTI0NDQxNzYwMTUwMjQxOSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 9b1b3f3501eedc4e54d3e6a74f616988558302e7fc782b8ee752713d710f7d59 |
| hovercard-subject-tag | issue:1041457450 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/1988/issue_layout |
| twitter:image | https://opengraph.githubassets.com/97c07b0fcb92d554766b6397a7a8290b1235b2d5c5adf269b0aeb84b22f95f09/feast-dev/feast/issues/1988 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/97c07b0fcb92d554766b6397a7a8290b1235b2d5c5adf269b0aeb84b22f95f09/feast-dev/feast/issues/1988 |
| og:image:alt | Is your feature request related to a problem? Please describe. (for context: we are using the Redis online store) At the moment, once an entity is written to the Online store, it's there until it's... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | vas28r13 |
| hostname | github.com |
| expected-hostname | github.com |
| None | 6d9b384ce19e0ac9756580c70eae9a3359939e77ff05c731561090d13085b2b7 |
| turbo-cache-control | no-preview |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 56ad33f6fdc0d0fb49c96b3c46ed74c55d926471 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width