René's URL Explorer Experiment


Title: feat(spark): SparkSource query+path and pre-computed offline read for BatchFeatureView by abhijeet-dhumal · Pull Request #6440 · feast-dev/feast · GitHub

Open Graph Title: feat(spark): SparkSource query+path and pre-computed offline read for BatchFeatureView by abhijeet-dhumal · Pull Request #6440 · feast-dev/feast

X Title: feat(spark): SparkSource query+path and pre-computed offline read for BatchFeatureView by abhijeet-dhumal · Pull Request #6440 · feast-dev/feast

Description: What this PR does / why we need it get_historical_features() on a BatchFeatureView re-runs the full UDF on raw data every call. For embedding pipelines, that's 20–40 min of compute per training run even though features already exist from the last materialize. Fix: Route get_historical_features() to read pre-computed parquet from batch_source.path instead of re-executing the UDF. To support this, SparkSource now accepts query + path together: query — raw data read during materialize() path — write-back target and pre-computed read source for get_historical_features() SparkSource( query="SELECT id, text, event_timestamp FROM bronze.documents", path="s3://my-bucket/feast/features/document_embeddings/", ) Also allows BatchFeatureView with online=False, offline=True (offline-only) to skip the online validation check in get_historical_features(), so it can be used purely for training data without configuring an online store. Falls back to live query if path doesn't exist yet (first run before any materialization). Which issue(s) this PR fixes N/A. Enables efficient training data retrieval for BatchFeatureView embedding pipelines without re-running UDFs. Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests — offline path routing, SparkSource constraint, graceful fallback Manual tests — get_historical_features() reads from parquet, not UDF, after materialization

Open Graph Description: What this PR does / why we need it get_historical_features() on a BatchFeatureView re-runs the full UDF on raw data every call. For embedding pipelines, that's 20–40 min of compute per training...

X Description: What this PR does / why we need it get_historical_features() on a BatchFeatureView re-runs the full UDF on raw data every call. For embedding pipelines, that's 20–40 min of compute per trai...

Opengraph URL: https://github.com/feast-dev/feast/pull/6440

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:69b9a5cc-3171-7d3e-bd0f-599180e2a5c7
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idDEF4:16CCCC:276AD34:36C7C72:6A4F5529
html-safe-nonce333ff20dfd23d14ba484df2cb6d57fd96f8bb8dd9e22094708ce16647cf906d0
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJERUY0OjE2Q0NDQzoyNzZBRDM0OjM2QzdDNzI6NkE0RjU1MjkiLCJ2aXNpdG9yX2lkIjoiNDE2NjI2NTY0NzgyMTQ0NDM5MyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmaca2e4e55528a0ed30847a5d9612cdfeb1293b338915e91f5e739cfbdf650a845b
hovercard-subject-tagpull_request:3754894421
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6440/files
twitter:imagehttps://avatars.githubusercontent.com/u/84722973?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/84722973?s=400&v=4
og:image:altWhat this PR does / why we need it get_historical_features() on a BatchFeatureView re-runs the full UDF on raw data every call. For embedding pipelines, that's 20–40 min of compute per training...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneb92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4b249b445842943ed31549e027f57a8ade9881ed
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/feast-dev/feast/pull/6440/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F6440%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F6440%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/pull/6440/files
Reloadhttps://github.com/feast-dev/feast/pull/6440/files
Reloadhttps://github.com/feast-dev/feast/pull/6440/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
feast-dev https://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Notifications https://github.com/login?return_to=%2Ffeast-dev%2Ffeast
Fork 1.4k https://github.com/login?return_to=%2Ffeast-dev%2Ffeast
Star 7.1k https://github.com/login?return_to=%2Ffeast-dev%2Ffeast
Code https://github.com/feast-dev/feast
Issues 212 https://github.com/feast-dev/feast/issues
Pull requests 173 https://github.com/feast-dev/feast/pulls
Discussions https://github.com/feast-dev/feast/discussions
Actions https://github.com/feast-dev/feast/actions
Security and quality 1 https://github.com/feast-dev/feast/security
Insights https://github.com/feast-dev/feast/pulse
Code https://github.com/feast-dev/feast
Issues https://github.com/feast-dev/feast/issues
Pull requests https://github.com/feast-dev/feast/pulls
Discussions https://github.com/feast-dev/feast/discussions
Actions https://github.com/feast-dev/feast/actions
Security and quality https://github.com/feast-dev/feast/security
Insights https://github.com/feast-dev/feast/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose
abhijeet-dhumalhttps://github.com/abhijeet-dhumal
feast-dev:masterhttps://github.com/feast-dev/feast/tree/master
abhijeet-dhumal:feat/spark-bfv-offline-historical-featureshttps://github.com/abhijeet-dhumal/feast/tree/feat/spark-bfv-offline-historical-features
Conversation 16 https://github.com/feast-dev/feast/pull/6440
Commits 8 https://github.com/feast-dev/feast/pull/6440/commits
Checks 28 https://github.com/feast-dev/feast/pull/6440/checks
Files changed https://github.com/feast-dev/feast/pull/6440/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
feat(spark): SparkSource query+path and pre-computed offline read for BatchFeatureView https://github.com/feast-dev/feast/pull/6440/files#top
Show all changes 8 commits https://github.com/feast-dev/feast/pull/6440/files
f6ff08a feat: allow query + path in SparkSource for offline materialization abhijeet-dhumal May 7, 2026 https://github.com/feast-dev/feast/pull/6440/commits/f6ff08a6d8a682d42bd013e43eec277cb15ba0ea
67f647c feat: read from offline path in get_historical_features for BFVs abhijeet-dhumal May 7, 2026 https://github.com/feast-dev/feast/pull/6440/commits/67f647cf4c663cf8be421a080021af6535042fd7
cafab0a fix: graceful fallback when offline path is not readable abhijeet-dhumal May 7, 2026 https://github.com/feast-dev/feast/pull/6440/commits/cafab0a3c553106fbe138b688900551879a2e96d
7d14b5c style: ruff format spark.py and spark_source.py abhijeet-dhumal May 27, 2026 https://github.com/feast-dev/feast/pull/6440/commits/7d14b5cfe4ba1efc24d17d32b8b176d29149fd95
5312075 fix: allow offline-only BatchFeatureView to skip online validation in… abhijeet-dhumal May 27, 2026 https://github.com/feast-dev/feast/pull/6440/commits/5312075970128750e8a559dd04f15a6976b321b1
6794ef2 ci: retrigger CI abhijeet-dhumal Jun 2, 2026 https://github.com/feast-dev/feast/pull/6440/commits/6794ef23df9926b531c0b6f9e07aefc5dd2bb2e7
caecfe4 fix(spark): narrow exception handling in offline path fallback abhijeet-dhumal Jun 2, 2026 https://github.com/feast-dev/feast/pull/6440/commits/caecfe4f94e2be8d4401613bf3c0e1cb7971328b
e7fc883 fix(test): lower performance benchmark threshold from 1.5x to 1.2x abhijeet-dhumal Jun 9, 2026 https://github.com/feast-dev/feast/pull/6440/commits/e7fc883fee4b5881c1964161064c93f568d5ff05
Clear filters https://github.com/feast-dev/feast/pull/6440/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
feature_store.py https://github.com/feast-dev/feast/pull/6440/files#diff-ff4e2a65c8c29b7971edc77a61a7e970d410d77d9f173f72714d1bb49e2509a2
spark.py https://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
spark_source.py https://github.com/feast-dev/feast/pull/6440/files#diff-5adf14693cee066990c26d08bac61fed1fd030a9e02b3f4ad16803b6be0df622
test_feature_server_utils.py https://github.com/feast-dev/feast/pull/6440/files#diff-a69939c75c6c8fab6dca227011b4c12b6e0aa70c7507d3e46d59e03904b09156
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/feast/feature_store.pyhttps://github.com/feast-dev/feast/pull/6440/files#diff-ff4e2a65c8c29b7971edc77a61a7e970d410d77d9f173f72714d1bb49e2509a2
View file https://github.com/feast-dev/feast/blob/e7fc883fee4b5881c1964161064c93f568d5ff05/sdk/python/feast/feature_store.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6440/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6440/files#diff-ff4e2a65c8c29b7971edc77a61a7e970d410d77d9f173f72714d1bb49e2509a2
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
https://github.com/feast-dev/feast/pull/6440/files#diff-ff4e2a65c8c29b7971edc77a61a7e970d410d77d9f173f72714d1bb49e2509a2
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark.pyhttps://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
View file https://github.com/feast-dev/feast/blob/e7fc883fee4b5881c1964161064c93f568d5ff05/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6440/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
ntkatholehttps://github.com/ntkathole
Jun 3, 2026https://github.com/feast-dev/feast/pull/6440/files#r3346952777
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
jyejarehttps://github.com/jyejare
Jun 1, 2026https://github.com/feast-dev/feast/pull/6440/files#r3334153095
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
abhijeet-dhumalhttps://github.com/abhijeet-dhumal
Jun 2, 2026https://github.com/feast-dev/feast/pull/6440/files#r3341447959
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
abhijeet-dhumalhttps://github.com/abhijeet-dhumal
Jun 2, 2026https://github.com/feast-dev/feast/pull/6440/files#r3341449589
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
jyejarehttps://github.com/jyejare
Jun 2, 2026https://github.com/feast-dev/feast/pull/6440/files#r3341712106
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
jyejarehttps://github.com/jyejare
Jun 1, 2026https://github.com/feast-dev/feast/pull/6440/files#r3334153126
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
abhijeet-dhumalhttps://github.com/abhijeet-dhumal
Jun 2, 2026https://github.com/feast-dev/feast/pull/6440/files#r3341588128
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
ntkatholehttps://github.com/ntkathole
Jun 3, 2026https://github.com/feast-dev/feast/pull/6440/files#r3346923453
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
jyejarehttps://github.com/jyejare
Jun 1, 2026https://github.com/feast-dev/feast/pull/6440/files#r3334153150
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
abhijeet-dhumalhttps://github.com/abhijeet-dhumal
Jun 2, 2026https://github.com/feast-dev/feast/pull/6440/files#r3341589482
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
https://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
https://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
ntkatholehttps://github.com/ntkathole
Jun 3, 2026https://github.com/feast-dev/feast/pull/6440/files#r3346916291
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
https://github.com/feast-dev/feast/pull/6440/files#diff-ef3fce408c702d00aa891015376e83ff1084907c69c95cafacf5fae657fca486
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.pyhttps://github.com/feast-dev/feast/pull/6440/files#diff-5adf14693cee066990c26d08bac61fed1fd030a9e02b3f4ad16803b6be0df622
View file https://github.com/feast-dev/feast/blob/e7fc883fee4b5881c1964161064c93f568d5ff05/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6440/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6440/files#diff-5adf14693cee066990c26d08bac61fed1fd030a9e02b3f4ad16803b6be0df622
https://github.com/feast-dev/feast/pull/6440/files#diff-5adf14693cee066990c26d08bac61fed1fd030a9e02b3f4ad16803b6be0df622
https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10
sdk/python/tests/unit/test_feature_server_utils.pyhttps://github.com/feast-dev/feast/pull/6440/files#diff-a69939c75c6c8fab6dca227011b4c12b6e0aa70c7507d3e46d59e03904b09156
View file https://github.com/feast-dev/feast/blob/e7fc883fee4b5881c1964161064c93f568d5ff05/sdk/python/tests/unit/test_feature_server_utils.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/feast-dev/feast/pull/6440/{{ revealButtonHref }}
https://github.com/feast-dev/feast/pull/6440/files#diff-a69939c75c6c8fab6dca227011b4c12b6e0aa70c7507d3e46d59e03904b09156
https://github.com/feast-dev/feast/pull/6440/files#diff-a69939c75c6c8fab6dca227011b4c12b6e0aa70c7507d3e46d59e03904b09156
Please reload this pagehttps://github.com/feast-dev/feast/pull/6440/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.