René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Feast Control Plane UI","articleBody":"## Summary\n\nFeast has a read-only UI that visualizes registry state. This proposal outlines evolving it into an interactive control plane where users can browse, edit, validate, and apply changes visually — and where AI agents can propose changes that humans verify on screen before applying.\n\nThis builds on the Feast UI (#2353), the REST API migration (#5411), CRUD requests (#5301, #5533, #5443), the context engine vision (#5761), and the existing MCP server integration.\n\n## Problem\n\n**The CLI + YAML workflow is powerful but exclusionary.**\n\nNot every Feast user is a data engineer comfortable writing Python definitions. Data scientists and ML engineers who consume features often learn better through visualization than reading YAML. Today, creating a feature view requires knowing table names, column names, and types ahead of time — context-switching between a database client and the feature repo. The edit → `feast apply` → read terminal → fix → repeat loop is slow, and `feast plan` (dry-run) only works on `local` + `sqlite` (`_should_use_plan()` in `feature_store.py`). In managed environments (OpenShift, air-gapped clusters), terminal-inside-pod is often the only option — and those changes don't survive pod restarts.\n\n**The existing UI shows state but cannot change it.**\n\nThe React UI (`ui/`) has pages for data sources, entities, feature views, feature services, permissions, and lineage. But:\n\n- **Entirely read-only.** Zero `POST`/`PUT`/`DELETE` calls or `useMutation` hooks in the React app. `useLoadRegistry` fetches a single protobuf blob from `/registry` and derives everything from that snapshot.\n- **REST registry API is GET-only.** All routes in `api/registry/rest/` are `@router.get`. The gRPC registry server has `ApplyEntity`, `ApplyFeatureView`, `ApplyPermission`, etc. — but these are not exposed via REST.\n- **Lineage is visual but static.** The React Flow graph renders relationships but cannot be edited.\n- **Permissions page says \"manage\" but only displays.** Fixing a group-based access mistake requires: edit Python → commit → `feast apply` → verify — 5-6 terminal steps for what should be one click.\n\n**No verification surface for agentic workflows.**\n\nFeast already has an MCP server (`fastapi_mcp` at `/mcp`) and OpenLineage integration. But without a visual layer, agents are black boxes — users can't see what an agent proposes, verify its choices, or catch mistakes before they reach the registry. Trust requires transparency, and transparency requires a screen.\n\n## Proposed Solution\n\nAn interactive **Feast Control Plane UI** — not replacing CLI/YAML, but complementing it. Phased approach:\n\n### Phase 1 — Live Registry + Basic Mutations (MVP)\n\n- **Migrate UI from protobuf dump to REST API.** The REST GET endpoints already exist with pagination, sorting, filtering, and relationship inclusion — the UI just doesn't consume them yet (#5411).\n- **Add REST write endpoints** wrapping the existing gRPC `Apply*`/`Delete*` RPCs.\n- **Add `/plan` endpoint** calling `FeatureStore.plan()` for structured diff output (currently gated to local+sqlite — registry diffs don't depend on provider and could be generalized).\n- **Create/edit forms** for entities, data sources, feature views with inline validation.\n- **Preview → Apply workflow** with diff view and real-time status.\n\n### Phase 2 — Schema Discovery + Interactive Lineage + Permissions\n\n- **Schema discovery:** browse data source tables/columns, create feature views by selecting columns interactively (eliminates the database-client context-switch).\n- **Interactive lineage:** click nodes to edit definitions inline; build on existing React Flow graph + permission overlays (`permissionUtils.ts`).\n- **Permissions editor:** transform the display-only page into a real editor — select resource types, actions, policy type, save. Run `feast permissions check` logic to surface coverage gaps.\n\n### Phase 3 — Agentic Workflows + Observability\n\n- **Agent proposals as pending diffs:** an AI agent (via MCP) proposes a feature view → it appears in the UI as a visual diff → user reviews, modifies, or rejects → approved changes go through the standard validated apply path.\n- **Agent-assisted debugging:** agent analyzes materialization failures or schema drift, surfaces findings in UI with suggested fixes.\n- **Observability:** feature freshness, materialization history, data source health, usage metrics. Builds on #5920 and existing OpenLineage emitter in `FeatureStore`.\n\nThe control plane becomes the **shared workspace** where human intent and agent execution meet — the agent proposes, the UI displays, the human decides.\n\n## Current State vs What Is Needed\n\n| Capability | Today | Needed |\n|------------|-------|--------|\n| Registry browsing | Protobuf dump via `useLoadRegistry` | Migrate to existing REST API endpoints |\n| Registry mutations | gRPC `Apply*/Delete*`; REST is GET-only | REST write wrappers |\n| Apply via HTTP | None — only `FeatureStore.apply()` Python API | REST `/apply` calling `FeatureStore.apply()` |\n| Plan / diff | `FeatureStore.plan()` — gated to local+sqlite | REST `/plan`; lift provider gate for registry diffs |\n| Schema discovery | `provider.validate_data_source()` for validation | New endpoint to enumerate tables/columns |\n| Lineage | React Flow graph, read-only | Editable nodes, click-to-edit definitions |\n| Permissions | Display-only page | Write API + edit forms |\n| MCP / agentic | `fastapi_mcp` at `/mcp` (serving endpoints only) | Registry mutation tools + approval workflow |\n\n## Technical Notes\n\n- **Backend:** REST write layer must call `FeatureStore.apply()` / `FeatureStore.plan()` — not bypass to raw gRPC Apply (which skips validation, inference, and infra updates). `plan()` already accepts `desired_repo_contents: RepoContents` — the entry point for UI-driven diffs without generating Python files.\n- **Frontend:** Extend existing React UI (Elastic UI, React Query, React Flow). Add mutation hooks (currently zero).\n- **Safety:** All mutations go through `FeatureStore` validation. Destructive ops require confirmation. Diff preview before apply.\n- **Auth:** Reuse existing OIDC + K8s auth. UI authenticates through the same mechanism as all other Feast clients.\n\n## Alternatives Considered\n\n- **CLI/YAML only.** Works for experienced engineers. Does not address discoverability, onboarding, or agentic verification.\n- **Internal tools instead of upstream.** Fragments the ecosystem. Community demand (#5301, #5533, #5443) shows this is a shared need.\n- **Wait for agentic maturity.** MCP and OpenLineage already exist. Building the control plane now provides the human-in-the-loop layer that responsible agent workflows need.\n\n## Open Questions\n\n1. **Where should this live?** Extend `ui/` (simplest), separate repo, or plugin?\n2. **Write API: gRPC wrappers vs full `FeatureStore.apply()` path?** The latter is safer but requires constructing `RepoContents` from API input.\n3. **Should `FeatureStore.plan()` be generalized?** Registry diffs don't depend on provider — only infra diffs do.\n4. **Code-vs-registry source of truth?** If teams use Git, UI-only edits could diverge. Should the UI generate exportable definitions?\n5. **How far should MCP go?** Direct mutation tools, or approval-gated proposals only?\n\n## References\n\n- #2353 — Feast Web UI\n- #5411 — Migrate UI to REST API\n- #5301, #5533, #5443 — CRUD UI requests\n- #5920 — Feature Server Observability\n- #5761 — Feast as Context Engine for AI Agents\n\n## Authors\nAniket Paluskar, Cursor (Claude)","author":{"url":"https://github.com/aniketpalu","@type":"Person","name":"aniketpalu"},"datePublished":"2026-03-29T11:22:46.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/6192/feast/issues/6192"}