René's URL Explorer Experiment


Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast · GitHub

Open Graph Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast

X Title: Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default · Issue #6907 · feast-dev/feast

Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect is that a feature view can declare vecto...

Open Graph Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ...

X Description: Summary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ...

Opengraph URL: https://github.com/feast-dev/feast/issues/6907

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by default","articleBody":"## Summary\n\n`vector_length` is declared on `Field` and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect is that a feature view can declare `vector_length=160`, be materialized with 200-dimensional vectors, and Feast will not complain.\n\nAll line numbers are against `master` as of filing.\n\n## 1. Validation only runs on the online-write path\n\n`_validate_vector_features` is defined at [`feature_store.py:3618`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3618) and has exactly one call site, [`feature_store.py:3665`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3665), inside `_get_feature_view_and_df_for_online_write`.\n\nSo:\n\n| entry point | validated? |\n|---|---|\n| `write_to_online_store` ([L3677](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3677)) | yes |\n| `materialize` ([L2835](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2835)) | no |\n| `materialize_incremental` ([L2592](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2592)) | no |\n| `get_historical_features` ([L1988](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L1988)) | no |\n\nEmbedding workloads are overwhelmingly batch. The path that actually carries vectors at volume is the unvalidated one.\n\n## 2. The check assumes the vector is the first feature\n\n[`feature_store.py:3629`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3629):\n\n```python\nif feature_view.features and feature_view.features[0].vector_index:\n```\n\nIf the vector field is not at index 0, `features[0].vector_index` is `False` and the whole check is skipped. Declaring `Field(name=\"id\", ...)` before `Field(name=\"embedding\", vector_index=True, vector_length=160)` is enough to disable validation, with no warning.\n\nThere is already a helper that does this correctly — [`utils.py:1959`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1959) `_get_feature_view_vector_field_metadata()`, which scans `feature_view.schema` for `vector_index` and raises if there is more than one ([L1965](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1965)). Several online stores already use it. The validator should too.\n\n## 3. `vector_length=0` is the default, and 0 means \"skip\"\n\n[`feature_store.py:3631`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3631):\n\n```python\nif feature_view.features[0].vector_length != 0:\n```\n\nand the default is 0 ([`field.py:60`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L60)). So `Field(name=\"embedding\", dtype=Array(Float32), vector_index=True)` — a perfectly natural declaration — validates nothing. Opt-in-by-accident is the wrong default for a correctness check.\n\nSuggestion: when `vector_index=True`, require `vector_length`, or infer it once from the first batch and then enforce it consistently. Either is better than silently doing nothing.\n\n## 4. `df.iterrows()` does not scale\n\n[`feature_store.py:3632`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3632) validates with a per-row Python loop. For embedding tables this is the dominant cost and effectively forces users to turn validation off. A vectorized length check over the column (or an Arrow-level check) is equivalent and ~free.\n\n## 5. `Field.__eq__` ignores two of the three vector attributes\n\n[`field.py:91-93`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L91-L93):\n\n```python\nor self.vector_length != other.vector_length\n# or self.vector_index != other.vector_index\n# or self.vector_search_metric != other.vector_search_metric\n```\n\nOnly `vector_length` participates in equality. Changing `vector_search_metric` from `COSINE` to `L2`, or flipping `vector_index`, is therefore invisible to `feast apply` / `feast plan` — a change that alters retrieval semantics produces no diff.\n\nIf these are commented out deliberately (e.g. to avoid churn on registries written before the fields existed), that is worth a comment saying so, because as written it reads like an oversight.\n\n## Why this matters\n\nReviewing an unrelated production embedding pipeline recently, I found the same class of bug with a worse outcome: instead of raising, it silently truncated vectors to the declared width (`vector.take(dimension)`), and the declared width itself fell back to \"length of the first row\" when absent from config. Nothing downstream verified it, and the resulting recall loss was invisible because no quality metric was attached to the artifact.\n\nFeast gets the hard part right — it raises rather than truncates. The problem is only that the raise is unreachable from the batch path, skipped when the vector is not the first field, and off by default. Those are cheap to fix and they are what makes the guarantee real.\n\n## Proposed changes\n\n1. Call `_validate_vector_features` from the materialize / historical-retrieval paths, not only online write.\n2. Resolve the vector field via `_get_feature_view_vector_field_metadata()` instead of `features[0]`.\n3. Require `vector_length` when `vector_index=True`, or enforce a consistent inferred value.\n4. Replace `iterrows()` with a vectorized length check.\n5. Decide `Field.__eq__` intent for `vector_index` / `vector_search_metric` — include them, or document why not.\n\nHappy to send this as a PR; filing first since (3) and (5) are behaviour decisions rather than clear-cut fixes.\n","author":{"url":"https://github.com/haoxu0","@type":"Person","name":"haoxu0"},"datePublished":"2026-09-30T17:13:48Z","url":"https://github.com/feast-dev/feast/issues/6907"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:0bf568b0-65bc-b50a-6e17-85d6747c3138
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id800A:C843D:ED89CF:1376057:6AFE951C
html-safe-nonce3fc581202de13851207f9289d32726be5bd941a990363e2be67747498009e49b
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4MDBBOkM4NDNEOkVEODlDRjoxMzc2MDU3OjZBRkU5NTFDIiwidmlzaXRvcl9pZCI6Ijg4MTczMjIzNjg3NDIzNjQ0NDQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac21cdc01a7f000a4b4879d94ac626ad344015979e61bbcd57f096d8b5a3b97df2
hovercard-subject-tagissue:5650256341
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/6907/issue_layout
twitter:imagehttps://opengraph.githubassets.com/071f5fb203884360c0e6436b9503966e4ab213006d1395ca593a25aa5af25f00/feast-dev/feast/issues/6907
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/071f5fb203884360c0e6436b9503966e4ab213006d1395ca593a25aa5af25f00/feast-dev/feast/issues/6907
og:image:altSummary vector_length is declared on Field and there is a validator for it, but the validator is reachable from only one code path and has three conditions that silently disable it. The net effect ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamehaoxu0
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/feast-dev/feast/issues/6907#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6907
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6907
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/issues/6907
Reloadhttps://github.com/feast-dev/feast/issues/6907
Reloadhttps://github.com/feast-dev/feast/issues/6907
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
https://github.com/feast-dev/feast
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6907
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6907
Vector length validation is unreachable from the batch path, skipped unless the vector is the first field, and off by defaulthttps://github.com/feast-dev/feast/issues/6907#top
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
https://github.com/haoxu0
haoxu0https://github.com/haoxu0
on Sep 30, 2026https://github.com/feast-dev/feast/issues/6907#issue-5650256341
feature_store.py:3618https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3618
feature_store.py:3665https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3665
L3677https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3677
L2835https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2835
L2592https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L2592
L1988https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L1988
feature_store.py:3629https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3629
utils.py:1959https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1959
L1965https://github.com/feast-dev/feast/blob/master/sdk/python/feast/utils.py#L1965
feature_store.py:3631https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3631
field.py:60https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L60
feature_store.py:3632https://github.com/feast-dev/feast/blob/master/sdk/python/feast/feature_store.py#L3632
field.py:91-93https://github.com/feast-dev/feast/blob/master/sdk/python/feast/field.py#L91-L93
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
on Sep 30, 2026https://github.com/feast-dev/feast/issues/6907#event-32185864510
on Sep 30, 2026https://github.com/feast-dev/feast/issues/6907#event-32186224509
on Sep 30, 2026https://github.com/feast-dev/feast/issues/6907#event-32188705050
fix: make vector length validation reachable, schema-driven and vecto…https://github.com/haoxu0/feast/commit/db6d34a2a66cdbe1e98e6bc10d23a728ff34c9bf
db6d34ahttps://github.com/haoxu0/feast/commit/db6d34a2a66cdbe1e98e6bc10d23a728ff34c9bf
on Sep 30, 2026https://github.com/feast-dev/feast/issues/6907#event-32190264266
fix: Make vector length validation reachable, schema-driven and vecto…https://github.com/haoxu0/feast/commit/1d9439b526934472c39d8933961834725130244c
1d9439bhttps://github.com/haoxu0/feast/commit/1d9439b526934472c39d8933961834725130244c
on Oct 1, 2026https://github.com/feast-dev/feast/issues/6907#event-32240521885
fix: Make vector length validation reachable, schema-driven and vecto…https://github.com/haoxu0/feast/commit/c2cbfeb7aaa53113c0457bf3898cd6680ed59009
c2cbfebhttps://github.com/haoxu0/feast/commit/c2cbfeb7aaa53113c0457bf3898cd6680ed59009
on Oct 1, 2026https://github.com/feast-dev/feast/issues/6907#event-32240536100
fix: Make vector length validation reachable, schema-driven and vecto…https://github.com/feast-dev/feast/commit/29af8c0ab71b8684adb8e57a7fca064fc0a20c7d
29af8c0https://github.com/feast-dev/feast/commit/29af8c0ab71b8684adb8e57a7fca064fc0a20c7d
https://github.com/haoxu0
haoxu0https://github.com/haoxu0
on Oct 3, 2026https://github.com/feast-dev/feast/issues/6907#issuecomment-5963877817
#6909https://github.com/feast-dev/feast/pull/6909
#6909https://github.com/feast-dev/feast/pull/6909
#6909https://github.com/feast-dev/feast/pull/6909
#6934https://github.com/feast-dev/feast/pull/6934
#6935https://github.com/feast-dev/feast/pull/6935
#4855https://github.com/feast-dev/feast/pull/4855
#6934https://github.com/feast-dev/feast/pull/6934
#6935https://github.com/feast-dev/feast/pull/6935
#6934https://github.com/feast-dev/feast/pull/6934
#6935https://github.com/feast-dev/feast/pull/6935
on Oct 4, 2026https://github.com/feast-dev/feast/issues/6907#event-32433650568
fix: Include vector attributes in Field equality (#6934)https://github.com/feast-dev/feast/commit/298c3f6a480b6088eba3498194d6e7b0210e8eaa
#6934https://github.com/feast-dev/feast/pull/6934
298c3f6https://github.com/feast-dev/feast/commit/298c3f6a480b6088eba3498194d6e7b0210e8eaa
on Oct 4, 2026https://github.com/feast-dev/feast/issues/6907#event-32444176680
fix: Infer vector width when vector_length is undeclaredhttps://github.com/haoxu0/feast/commit/bb0c5a1243253336df675ffb0a26a97b07e248bd
bb0c5a1https://github.com/haoxu0/feast/commit/bb0c5a1243253336df675ffb0a26a97b07e248bd
on Oct 4, 2026https://github.com/feast-dev/feast/issues/6907#event-32444192818
fix: Infer vector width when vector_length is undeclaredhttps://github.com/feast-dev/feast/commit/005656522f8110b65a5b9d18e6ca449d7997526b
0056565https://github.com/feast-dev/feast/commit/005656522f8110b65a5b9d18e6ca449d7997526b
https://github.com/haoxu0
haoxu0https://github.com/haoxu0
on Oct 4, 2026https://github.com/feast-dev/feast/issues/6907#issuecomment-5982050076
#6909https://github.com/feast-dev/feast/pull/6909
#6909https://github.com/feast-dev/feast/pull/6909
#6909https://github.com/feast-dev/feast/pull/6909
#6934https://github.com/feast-dev/feast/pull/6934
#6935https://github.com/feast-dev/feast/pull/6935
#4855https://github.com/feast-dev/feast/pull/4855
#6934https://github.com/feast-dev/feast/pull/6934
#6935https://github.com/feast-dev/feast/pull/6935
completedhttps://github.com/feast-dev/feast/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Oct 4, 2026https://github.com/feast-dev/feast/issues/6907#event-32451589579
on Oct 13, 2026https://github.com/feast-dev/feast/issues/6907#event-33136589898
chore(release): release 0.67.0https://github.com/feast-dev/feast/commit/2dfb9e6d1a37719fd11028d916094050f1cdd552
2dfb9e6https://github.com/feast-dev/feast/commit/2dfb9e6d1a37719fd11028d916094050f1cdd552
on Oct 14, 2026https://github.com/feast-dev/feast/issues/6907#event-33213940627
fix: Merge upstream feast-dev/feast master - resolve CSV conflict (#247)https://github.com/opendatahub-io/feast/commit/2b0b7d75acd0a1fbee53a0e77a81010905a82fbc
#247https://github.com/opendatahub-io/feast/pull/247
2b0b7d7https://github.com/opendatahub-io/feast/commit/2b0b7d75acd0a1fbee53a0e77a81010905a82fbc
Sign up for freehttps://github.com/signup?return_to=https://github.com/feast-dev/feast/issues/6907
Sign in to commenthttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6907
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.