René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Running `feast materialize-incremental` for an end date far in future breaks incremental materializations up to that date","articleBody":"## Expected Behavior\r\n\r\nWhen running `feast materialize-incremental` it should not be possible for a materialize to run starting from a date that hasn't occurred yet. Would expect `materialize-incremental` to choose `min(now, most_recent_materialization_date)`\r\n\r\n## Current Behavior\r\n\r\nWhen running `feast materialize-incremental` for a future date, say `2025-04-08T00:00:00`, it will set that date as the next start date. This breaks all future `materialize-incremental` commands up until that date, if you run daily materializations they will no longer function until the actual date goes beyond this erroneously entered start date. Because the data is stored as a serialized protobuf in the SQL-based registry this is also non-trivial to change. \r\n\r\nYou can see code [here](https://github.com/feast-dev/feast/blob/1f17b9bcd635d1344e1f9be4dcb745d8e206e071/sdk/python/feast/feature_view.py#L444) showing that there is no validation checks for the start date, merely the latest date that has ever run.\r\n\r\n## Steps to reproduce\r\n\r\nRun a `materialize-incremental` command for a date far in the future. \r\n\r\nRun a second `materialize-incremental` command for a date less than the last start date. Watch Feast always find no date to materialize. \r\n\r\n### Specifications\r\n\r\n- Version:  0.34.1\r\n- Platform: Mac OSX\r\n- Subsystem: Sonoma 14.5\r\n\r\n## Possible Solution\r\n\r\nRefactor the above function to choose `min(now, most_recent_materialization_date)`. Happy to contribute to this fix. \r\n","author":{"url":"https://github.com/samhallam-reverb","@type":"Person","name":"samhallam-reverb"},"datePublished":"2024-05-23T19:54:21.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":10},"url":"https://github.com/4222/feast/issues/4222"}