Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast · GitHub
Open Graph Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast
X Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast
Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect is that a feature view can declare vecto...
Open Graph Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ...
X Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ...
Opengraph URL: https://github.com/feast-dev/feast/issues/6907
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default","articleBody":"## Summary\n\n`vector_length` is declared on `Field` and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect is that a feature view can declare `vector_length=160`, be materialized with 200-dimensional vectors, and Feast will not complain.\n\nAll line numbers are against `master` as of filing.\n\n## 1. Validation only runs on the online-write path\n\n`_validate_vector_features` is defined at [`feature_store.py:3618`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3618) and has exactly one call site, [`feature_store.py:3665`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3665), inside `_get_feature_view_and_df_for_online_write`.\n\nSo:\n\n| entry point | validated? |\n|---|---|\n| `write_to_online_store` ([L3677](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3677)) | yes |\n| `materialize` ([L2835](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2835)) | no |\n| `materialize_incremental` ([L2592](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2592)) | no |\n| `get_historical_features` ([L1988](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L1988)) | no |\n\nEmbedding workloads are overwhelmingly batch. The path that actually carries vectors at volume is the unvalidated one.\n\n## 2. The check assumes the vector is the first feature\n\n[`feature_store.py:3629`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3629):\n\n```python\nif feature_view.features and feature_view.features[0].vector_index:\n```\n\nIf the vector field is not at index 0, `features[0].vector_index` is `False` and the whole check is skipped. Declaring `Field(name=\"id\", ...)` before `Field(name=\"embedding\", vector_index=True, vector_length=160)` is enough to disable validation, with no warning.\n\nThere is already a helper that does this correctly — [`utils.py:1959`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1959) `_get_feature_view_vector_field_metadata()`, which scans `feature_view.schema` for `vector_index` and raises if there is more than one ([L1965](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1965)). Several online stores already use it. The validator should too.\n\n## 3. `vector_length=0` is the default, and 0 means \"skip\"\n\n[`feature_store.py:3631`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3631):\n\n```python\nif feature_view.features[0].vector_length != 0:\n```\n\nand the default is 0 ([`field.py:60`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L60)). So `Field(name=\"embedding\", dtype=Array(Float32), vector_index=True)` — a perfectly natural declaration — validates nothing. Opt-in-by-accident is the wrong default for a correctness check.\n\nSuggestion: when `vector_index=True`, require `vector_length`, or infer it once from the first batch and then enforce it consistently. Either is better than silently doing nothing.\n\n## 4. `df.iterrows()` does not scale\n\n[`feature_store.py:3632`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3632) validates with a per-row Python loop. For embedding tables this is the dominant cost and effectively forces users to turn validation off. A vectorized length check over the column (or an Arrow-level check) is equivalent and ~free.\n\n## 5. `Field.__eq__` ignores two of the three vector attributes\n\n[`field.py:91-93`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L91-L93):\n\n```python\nor self.vector_length != other.vector_length\n# or self.vector_index != other.vector_index\n# or self.vector_search_metric != other.vector_search_metric\n```\n\nOnly `vector_length` participates in equality. Changing `vector_search_metric` from `COSINE` to `L2`, or flipping `vector_index`, is therefore invisible to `feast apply` / `feast plan` — a change that alters retrieval semantics produces no diff.\n\nIf these are commented out deliberately (e.g. to avoid churn on registries written before the fields existed), that is worth a comment saying so, because as written it reads like an oversight.\n\n## Why this matters\n\nReviewing an unrelated production embedding pipeline recently, I found the same class of bug with a worse outcome: instead of raising, it silently truncated vectors to the declared width (`vector.take(dimension)`), and the declared width itself fell back to \"length of the first row\" when absent from config. Nothing downstream verified it, and the resulting recall loss was invisible because no quality metric was attached to the artifact.\n\nFeast gets the hard part right — it raises rather than truncates. The problem is only that the raise is unreachable from the batch path, skipped when the vector is not the first field, and off by default. Those are cheap to fix and they are what makes the guarantee real.\n\n## Proposed changes\n\n1. Call `_validate_vector_features` from the materialize / historical-retrieval paths, not only online write.\n2. Resolve the vector field via `_get_feature_view_vector_field_metadata()` instead of `features[0]`.\n3. Require `vector_length` when `vector_index=True`, or enforce a consistent inferred value.\n4. Replace `iterrows()` with a vectorized length check.\n5. Decide `Field.__eq__` intent for `vector_index` / `vector_search_metric` — include them, or document why not.\n\nHappy to send this as a PR; filing first since (3) and (5) are behaviour decisions rather than clear-cut fixes.\n","author":{"url":"https://github.com/haoxu0","@type":"Person","name":"haoxu0"},"datePublished":"2026-09-30T17:13:48Z","url":"https://github.com/feast-dev/feast/issues/6907"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:0bf568b0-65bc-b50a-6e17-85d6747c3138 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 800A:C843D:ED89CF:1376057:6AFE951C |
| html-safe-nonce | 3fc581202de13851207f9289d32726be5bd941a990363e2be67747498009e49b |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4MDBBOkM4NDNEOkVEODlDRjoxMzc2MDU3OjZBRkU5NTFDIiwidmlzaXRvcl9pZCI6Ijg4MTczMjIzNjg3NDIzNjQ0NDQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 21cdc01a7f000a4b4879d94ac626ad344015979e61bbcd57f096d8b5a3b97df2 |
| hovercard-subject-tag | issue:5650256341 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/6907/issue_layout |
| twitter:image | https://opengraph.githubassets.com/071f5fb203884360c0e6436b9503966e4ab213006d1395ca593a25aa5af25f00/feast-dev/feast/issues/6907 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/071f5fb203884360c0e6436b9503966e4ab213006d1395ca593a25aa5af25f00/feast-dev/feast/issues/6907 |
| og:image:alt | Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | haoxu0 |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width