René's URL Explorer Experiment

Title: feat: [Backend] Data Quality Monitoring with native compute, multi-backend support, REST API, CLI by jyejare · Pull Request #6202 · feast-dev/feast · GitHub

Open Graph Title: feat: [Backend] Data Quality Monitoring with native compute, multi-backend support, REST API, CLI by jyejare · Pull Request #6202 · feast-dev/feast

X Title: feat: [Backend] Data Quality Monitoring with native compute, multi-backend support, REST API, CLI by jyejare · Pull Request #6202 · feast-dev/feast

Description: To check real UI monitoring: Visit PR #6422, see Demo. What this PR does / why we need it: This PR introduces comprehensive feature quality monitoring capabilities to Feast, enabling proactive tracking of feature distributions and data quality metrics. Currently, Feast has no built-in tools for monitoring feature health in production — ML teams must build custom solutions to detect issues like distribution shifts, elevated null rates, or degraded data quality before they silently impact model performance. What it adds: Core Monitoring Engine Hybrid computation engine — SQL push-down on the native OfflineStore as the primary compute path, with a Python-based (PyArrow/NumPy) fallback for backends that don't implement native compute. This leverages the offline store as a compute engine (same architecture as Feast materialization). Fully native storage — Monitoring metrics are stored within the configured OfflineStore backend itself (no separate monitoring database). Six static methods on the OfflineStore base class (compute_monitoring_metrics, get_monitoring_max_timestamp, ensure_monitoring_tables, save_monitoring_metrics, query_monitoring_metrics, clear_monitoring_baseline) handle compute and storage. PyArrow-based metrics computation (MetricsCalculator) — Backend-agnostic statistical computation as fallback, supporting: Numeric features: mean, stddev, min/max, percentiles (p50/p75/p90/p95/p99), null rate, histograms Categorical features: top-N value counts with other/unique counts Automatic feature type classification from Feast's PrimitiveFeastType and ValueType Multi-Backend Support (8 Offline Stores) All 6 native monitoring methods implemented for each backend with dialect-specific SQL: Backend Compute Storage Dialect highlights PostgreSQL SQL push-down INSERT ON CONFLICT PERCENTILE_CONT, WIDTH_BUCKET Snowflake SQL push-down MERGE with VARIANT JSON APPROX_PERCENTILE, WIDTH_BUCKET BigQuery SQL push-down MERGE into BQ tables APPROX_QUANTILES, parameterized queries Redshift SQL push-down MERGE via Data API APPROXIMATE PERCENTILE_DISC Spark SparkSQL push-down Parquet tables PERCENTILE_APPROX, spark.sql() Oracle SQL via Ibis MERGE FROM DUAL PERCENTILE_CONT WITHIN GROUP DuckDB In-memory SQL Parquet files QUANTILE_CONT, HISTOGRAM Dask PyArrow compute Parquet files pyarrow.compute + numpy Multi-Granularity Time-Series Metrics 5 granularities: daily, weekly, biweekly, monthly, quarterly Auto-compute mode: Detects latest event timestamp and computes all granularities in one job Pre-computed metrics stored per date + granularity for fast retrieval On-demand transient compute: Fresh statistics for arbitrary date ranges (not stored) Batch + Log Data Source Support Batch source: Reads from the feature view's batch_source via OfflineStore.pull_all_from_table_or_query() Log source: Reads from feature serving logs via FeatureService.logging_config destination, using __log_timestamp as event timestamp Feature name normalization: Prefixed log column names (driver_stats__conv_rate) are parsed back to their original feature_view_name + feature_name for storage compatibility and drift detection data_source_type column (batch / log) differentiates metrics in storage Orchestration Service (MonitoringService) Ties registry, offline store, calculator, and storage together Computes and aggregates metrics at feature, feature view, and feature service levels Cached OfflineStore instance for performance Unified compute/timestamp methods handling both batch and log paths with SQL push-down + fallback NaN/Inf Sanitization Multi-layered protection against NaN/Inf float values that break JSON serialization: opt_float() in monitoring_utils.py — sanitizes at SQL result parsing _sanitize_floats() in monitoring_service.py — final safety net on all API read paths Applied in PyArrow compute paths (MetricsCalculator, Dask backend) Prevents HTTP 422 errors from Out of range float values are not JSON compliant: nan Shared Utilities (monitoring_utils.py) Centralized table name constants, column lists, PK definitions monitoring_table_meta(), opt_float(), empty_numeric_metric(), empty_categorical_metric(), normalize_monitoring_row(), build_view_aggregate() Used by all 8 backends — eliminates duplication and ensures consistency DQM Job Engine (DQMJobManager) Asynchronous job abstraction for metric computation (compute, baseline, auto_compute) Job status tracking in feast_monitoring_jobs table Forwards all parameters including set_baseline to the compute engine Supports future integration with Ray/Spark job runners REST API (/monitoring/) Method Endpoint Description POST /monitoring/compute Submit batch DQM job POST /monitoring/auto_compute Auto-detect dates, all granularities POST /monitoring/compute/transient On-demand compute (not stored) POST /monitoring/compute/log Compute from serving logs POST /monitoring/auto_compute/log Auto-detect log dates, all granularities GET /monitoring/jobs/{job_id} DQM job status GET /monitoring/metrics/features Per-feature metrics GET /monitoring/metrics/feature_views Per-view aggregates GET /monitoring/metrics/feature_services Per-service aggregates GET /monitoring/metrics/baseline Baseline distribution retrieval GET /monitoring/metrics/timeseries Time-series data for trend analysis All endpoints support cascading filters: project, feature_service_name, feature_view_name, feature_name, granularity, data_source_type, date range. RBAC enforced using existing AuthzedAction.DESCRIBE (read) and AuthzedAction.UPDATE (compute). CLI (feast monitor run) Options: --feature-view TEXT Feature view name (omit for all) --feature-service TEXT Feature service name (required for --source-type log with explicit dates) --feature-name TEXT Feature name(s), repeatable --start-date TEXT Start date YYYY-MM-DD (omit for auto-detect) --end-date TEXT End date YYYY-MM-DD (omit for auto-detect) --granularity TEXT daily | weekly | biweekly | monthly | quarterly --set-baseline Mark this computation as baseline --source-type TEXT batch | log | all (default: batch) Auto-Baseline on feast apply Automatically queues baseline metric computation for new features on feast apply Non-blocking (async DQM job), idempotent (skips existing baselines) Configurable — can be disabled via feature_store.yaml: data_quality_monitoring: auto_baseline: false Feast Operator Support New CRD type: DataQualityMonitoringConfig added to FeatureStoreSpec Operator generates data_quality_monitoring section in feature_store.yaml when config is set DeepCopy methods auto-generated via make generate Disabling auto-baseline from operator CR: apiVersion: feast.dev/v1 kind: FeatureStore spec: feastProject: my_project dataQualityMonitoring: autoBaseline: false Documentation How-to guide: docs/how-to-guides/feature-monitoring.md — Production setup, CLI usage, REST API reference, orchestrator integration (Airflow, KFP, cron, K8s CronJob), backend compatibility table Quickstart notebook: examples/monitoring/monitoring-quickstart.ipynb — 12-step hands-on walkthrough with visualization examples docs/SUMMARY.md updated with links to both Design decisions: Native OfflineStore compute + storage — Each backend implements its own SQL push-down for metrics calculation and uses its native UPSERT/MERGE for storage. No separate monitoring database needed. Hybrid fallback — Backends that don't implement native compute fall back to Python/PyArrow, ensuring all offline stores are supported. Separate /monitoring/ route rather than extending existing /metrics/ — The existing metrics route serves registry inventory metadata; monitoring serves statistical feature quality data with a different data path. DQM Job Engine for async computation — Supports future Ray/Spark integration for distributed metric computation. Top-level data_quality_monitoring config — Sits alongside materialization and openlineage in RepoConfig, reflecting that it spans offline store compute/storage + apply trigger + server API. Which issue(s) this PR fixes: Partially Fixes #5919 Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests Integration tests Operator unit tests (Ginkgo) Test coverage (all passing): Test Suite Count Covers test_metrics_calculator.py 30+ Numeric/categorical computation, edge cases (empty, all-null, single value, high cardinality), type classification, PyArrow type classification, NaN/Inf sanitization, JSON serializability test_compute_correctness.py 40+ Per-backend metric accuracy for all 8 offline stores (DuckDB, Dask, PostgreSQL, Snowflake, BigQuery, Redshift, Oracle, Spark) — numeric stats, histograms, categorical counts test_monitoring_integration.py 16+ End-to-end batch/log computation, baseline flow, view/service aggregation, native storage dispatch, log feature name normalization, REST API endpoints, CLI, RBAC enforcement repo_config_test.go 92 Operator repo config generation including DataQualityMonitoring config with auto_baseline disabled, YAML serialization verification Snyk SAST scan: 0 vulnerabilities across all new files.

Open Graph Description: To check real UI monitoring: Visit PR #6422, see Demo. What this PR does / why we need it: This PR introduces comprehensive feature quality monitoring capabilities to Feast, enabling proactive trac...

X Description: To check real UI monitoring: Visit PR #6422, see Demo. What this PR does / why we need it: This PR introduces comprehensive feature quality monitoring capabilities to Feast, enabling proactive trac...

Opengraph URL: https://github.com/feast-dev/feast/pull/6202

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width