René's URL Explorer Experiment

Title: feat: Implemented Tiling Support for Time-Windowed Aggregations by ntkathole · Pull Request #5724 · feast-dev/feast · GitHub

Open Graph Title: feat: Implemented Tiling Support for Time-Windowed Aggregations by ntkathole · Pull Request #5724 · feast-dev/feast

X Title: feat: Implemented Tiling Support for Time-Windowed Aggregations by ntkathole · Pull Request #5724 · feast-dev/feast

Description: What this PR does / why we need it: This PR implements tiling for computing time-windowed aggregations efficiently by pre-aggregating data into small, time-bucketed units (tiles) that can be reused across multiple queries. Instead of scanning all raw events for each window, it: Pre-aggregates data into small time buckets (tiles) Computes windowed aggregations by subtracting tiles: window_at_T = tile_at_T - tile_at_(T-window_size) Stores intermediate representations (IRs) for holistic aggregations (avg, std, var) to enable correct merging Integrates tiling into Spark and Ray compute engines for StreamFeatureView Sawtooth Window Tiling Tile Generation: Events bucketed into hop-sized intervals (e.g., 1-minute tiles) Cumulative Aggregation: Each tile contains cumulative sum/count from start Window Computation: Windowed aggregation = tile_at_T - tile_at_(T-window_size) IR Handling: For holistic aggs, IRs subtracted separately, then final value recomputed Enable Tiling in StreamFeatureView sales_features = StreamFeatureView( name="sales_features", entities=[customer], source=kafka_source, aggregations=[ Aggregation( column="amount", function="avg", time_window=timedelta(minutes=5), ), Aggregation( column="amount", function="sum", time_window=timedelta(minutes=5), ), ], timestamp_field="event_timestamp", enable_tiling=True, # Enable tiling optimization tiling_hop_size=timedelta(minutes=1), # Generate tiles every 1 minute ) Architecture ┌─────────────────────────────────────────────────────────────┐ │ StreamFeatureView (enable_tiling=True) │ └────────────────────────┬────────────────────────────────────┘ │ ┌───────────────┴───────────────┐ │ │ ┌────▼────┐ ┌────▼────┐ │ Spark │ │ Ray │ │ Node │ │ Node │ └────┬────┘ └────┬────┘ │ │ │ Convert to pandas │ ▼ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Tiling Logic. │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ orchestrator.apply_sawtooth_window_tiling() │ │ │ │ - Group by entity + hop interval │ │ │ │ - Compute cumulative aggregations (tiles) │ │ │ │ - Output: Cumulative tiles at each hop │ │ │ └──────────────────────┬───────────────────────────────┘ │ │ │ │ │ ┌──────────────────────▼───────────────────────────────┐ │ │ │ tile_subtraction.convert_cumulative_to_windowed() │ │ │ │ - Subtract tiles: window = tile_T - tile_(T-W) │ │ │ │ - Recompute holistic aggs from IRs │ │ │ │ - Output: Windowed aggregations │ │ │ └──────────────────────┬───────────────────────────────┘ │ │ │ │ │ ┌──────────────────────▼───────────────────────────────┐ │ │ │ tile_subtraction.deduplicate_keep_latest() │ │ │ │ - Keep latest window per entity │ │ │ └──────────────────────────────────────────────────────┘ │ └─────────────────────────┬───────────────────────────────────┘ │ Convert back to engine format ▼ ┌───────────────┐ │ Online Store │ │ │ └───────────────┘

Open Graph Description: What this PR does / why we need it: This PR implements tiling for computing time-windowed aggregations efficiently by pre-aggregating data into small, time-bucketed units (tiles) that can be reused...

X Description: What this PR does / why we need it: This PR implements tiling for computing time-windowed aggregations efficiently by pre-aggregating data into small, time-bucketed units (tiles) that can be reused...

Opengraph URL: https://github.com/feast-dev/feast/pull/5724

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width