René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Supporting multiple online feature stores","articleBody":"**Is your feature request related to a problem? Please describe.**\r\nWe are seeing that ML feature teams want to own their feature projections and not rely on a single global online feature store with the most common reason being reduction of blast radius.\r\n\r\n**Describe the solution you'd like**\r\nA way to create projections of a subset of feature views to specific online stores.\r\n\r\n**Describe alternatives you've considered**\r\nI was pointed towards attempting to implement the requested feature using a custom provider which I managed to get working, albeit rather hacky. I've only focused on fanning out the writing of features. \r\n\r\n```python\r\nclass ManyOnlineStoresPassthroughConfig(RepoConfig):\r\n    online_stores: List[dict]\r\n\r\nclass ManyOnlineStoresPassthroughProvider(PassthroughProvider):\r\n    def online_write_batch(self, config: ManyOnlineStoresPassthroughConfig, table: FeatureView, *args, **kwargs) -\u003e None:\r\n        \"\"\"\r\n        Writes feature view to all subscribers\r\n        \"\"\"\r\n        feature_view_subscribers = [store for store in config.online_stores if table.name in store[\"view_subscriptions\"]]\r\n\r\n        for store in feature_view_subscribers:\r\n            print(f\"{table.name} online write to subscriber {store['name']}\")\r\n\r\n            config.online_store = get_online_config_from_type(store[\"type\"])(**store)\r\n            self.online_store = get_online_store_from_config(config.online_store)\r\n\r\n            return super().online_write_batch(config, table, *args, **kwargs)\r\n```\r\n\r\nWith the config update of\r\n\r\n```yaml\r\nproject: aaaa\r\nregistry: gs://temp-feast-dev/dev/registry.db\r\nprovider: providers.custom_provider.ManyOnlineStoresPassthroughProvider\r\n\r\noffline_store:\r\n  type: bigquery\r\n  dataset: temp_feast-dev\r\n  project_id: abc\r\n  location: EU\r\n\r\nonline_stores:\r\n  - name: \"lovely\"\r\n    type: redis\r\n    redis_type: redis\r\n    connection_string: \"blah\"\r\n    view_subscriptions: [\"feature_view_1\"]\r\n\r\n  - name: \"lovely\"\r\n    type: redis\r\n    redis_type: redis\r\n    connection_string: \"abc\"\r\n    view_subscriptions: [\"feature_view_1\", \"feature_view_2\"]\r\n```\r\n\r\nThere's still some unknowns around retrieving the online features as we'd need a way of knowing what projection should be used. \r\n\r\nHappy to help wherever I can! ","author":{"url":"https://github.com/alexjajones","@type":"Person","name":"alexjajones"},"datePublished":"2022-03-07T22:27:01.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/2383/feast/issues/2383"}