René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`FeatureStore.materialize_incremental()` does not initially materialize data for `FeatureView` with `ttl=0`","articleBody":"## Expected Behavior \r\nWhen executing `FeatureStore.materialize_incremental()` on a `FeatureView` that has a `ttl=0` and has never yet been materialized, I would expect all historical data to be materialized since a `ttl=0` indicates that features live forever according to the [docs](https://rtd.feast.dev/en/master/index.html#feast.feature_view.FeatureView.ttl).\r\n\r\nFor an instance of `FeatureView` that has not explicitly defined `ttl`, it currently defaults to `timedelta(days=0)`, which hints that this is indeed an expected value for `ttl`.\r\n\r\n## Current Behavior\r\nCurrently, the `start_date` for the materialization defaults to `datetime.utcnow() - ttl` if the feature view has not yet been materialized (i.e. `FeatureView.most_recent_end_time is None`). This means that `start_date = datetime.utcnow() - 0s`, thus `start_date = datetime.utcnow()`, resulting in no data being materialized.\r\n\r\n[feast/sdk/python/feast/feature_store.py](https://github.com/feast-dev/feast/blob/5b4b07f999eb38542eb5518cf0db54a98baeeb40/sdk/python/feast/feature_store.py#L1073)\r\n```python\r\n    def materialize_incremental(\r\n        self, end_date: datetime, feature_views: Optional[List[str]] = None,\r\n    ) -\u003e None:\r\n        ...\r\n        for feature_view in feature_views_to_materialize:\r\n            start_date = feature_view.most_recent_end_time\r\n            if start_date is None:\r\n                if feature_view.ttl is None:\r\n                    raise Exception(\r\n                        f\"No start time found for feature view {feature_view.name}. materialize_incremental() requires\"\r\n                        f\" either a ttl to be set or for materialize() to have been run at least once.\"\r\n                    )\r\n                \u003e\u003e\u003e start_date = datetime.utcnow() - feature_view.ttl \u003c\u003c\u003c\r\n        ...\r\n\r\n```\r\n\r\n## Steps to reproduce\r\n\r\n```python\r\nfrom feast import FeatureView, FeatureStore\r\nfrom datetime import datetime \r\n\r\nmy_feature_view = FeatureView(\r\n    name=\"my_feature_view\",\r\n    entities=[\"my_entity\"],\r\n    schema=[\r\n        Field(name=\"my_feature\", dtype=types.Bool)\r\n    ],\r\n    # ttl=timedelta(seconds=0) ; if not defined, TTL defaults to 0s\r\n    source=sources.my_source\r\n)\r\n\r\nfs = FeatureStore(\".\")\r\nfs.materialize_incremental(end_date=datetime.utcnow(), views=[\"my_feature_view\"])\r\n```\r\n\r\n### Specifications\r\n\r\n- Version: 0.20.2\r\n- Platform: macOS\r\n- Subsystem:\r\n\r\n## Possible Solution\r\n[feast/sdk/python/feast/feature_store.py](https://github.com/feast-dev/feast/blob/5b4b07f999eb38542eb5518cf0db54a98baeeb40/sdk/python/feast/feature_store.py#L1073)\r\n```python\r\n    def materialize_incremental(\r\n        self, end_date: datetime, feature_views: Optional[List[str]] = None,\r\n    ) -\u003e None:\r\n        ...\r\n        for feature_view in feature_views_to_materialize:\r\n            start_date = feature_view.most_recent_end_time\r\n            if start_date is None:\r\n                if feature_view.ttl is None:\r\n                    raise Exception(\r\n                        f\"No start time found for feature view {feature_view.name}. materialize_incremental() requires\"\r\n                        f\" either a ttl to be set or for materialize() to have been run at least once.\"\r\n                    )\r\n                start_date = datetime.utcnow() - feature_view.ttl\r\n\r\n                if feature_view.ttl == timedelta(days=0):\r\n                    # what is the lower boundary for \"forever\" (as defined in the docs for ttl=0)?\r\n                    # the smallest UNIX epoch timestamp could be a good candidate\r\n                    start_date = datetime(1970,  1, 1)\r\n        ...\r\n```\r\n","author":{"url":"https://github.com/chasleslr","@type":"Person","name":"chasleslr"},"datePublished":"2022-05-09T15:43:31.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/2651/feast/issues/2651"}