René's URL Explorer Experiment


Title: Ray compute engine reports a failed online-store write as a successful materialization · Issue #7024 · feast-dev/feast · GitHub

Open Graph Title: Ray compute engine reports a failed online-store write as a successful materialization · Issue #7024 · feast-dev/feast

X Title: Ray compute engine reports a failed online-store write as a successful materialization · Issue #7024 · feast-dev/feast

Description: Expected Behavior When the online store refuses a write during feast materialize with the Ray compute engine, the materialization fails (job status ERROR and FeatureStore.materialize raises), the way it does with the Local engine on the ...

Open Graph Description: Expected Behavior When the online store refuses a write during feast materialize with the Ray compute engine, the materialization fails (job status ERROR and FeatureStore.materialize raises), the w...

X Description: Expected Behavior When the online store refuses a write during feast materialize with the Ray compute engine, the materialization fails (job status ERROR and FeatureStore.materialize raises), the w...

Opengraph URL: https://github.com/feast-dev/feast/issues/7024

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Ray compute engine reports a failed online-store write as a successful materialization","articleBody":"## Expected Behavior\n\nWhen the online store refuses a write during `feast materialize` with the Ray compute engine, the materialization fails (job status ERROR and `FeatureStore.materialize` raises), the way it does with the Local engine on the same setup.\n\n## Current Behavior\n\nWith `batch_engine: ray.engine`, the job reports SUCCEEDED, `materialize` returns normally, and `get_online_features` keeps serving the previous values. Three handlers absorb the error:\n\n- `RayWriteNode.execute` (`sdk/python/feast/infra/compute_engines/ray/nodes.py`, 864): the batch writer is decorated with `@safe_batch_processor`, which logs and returns the batch on any exception (`ray/utils.py` 210-212). This is the path `FeatureStore.materialize` takes.\n- `write_to_online_store` (`ray/utils.py`, handler 82-83) and `write_to_online_store_from_ray_ds` (177-180) log the error and do not raise. These are on the legacy `from_offline_store` path.\n\nWhen Feast starts Ray itself, `log_to_driver` is off unless `enable_ray_logging` is set (`sdk/python/feast/infra/ray_initializer.py` 668-682), so the error line does not reach the console either; it only appears in a Ray worker log under `/tmp/ray`.\n\n`RayReadNode` in the same file re-raises after logging, and the Local, Spark and Flink engines let the error through.\n\n### Something this is hiding\n\n`tests/component/ray/test_compute.py::test_ray_compute_engine_materialize` passes `online_store=MagicMock()` and asserts SUCCEEDED. Run as-is with a real SQLite online store in place of the mock, it still reports SUCCEEDED, but the online table holds 0 rows: the aggregated BatchFeatureView's write raises `ValueError: Column conv_rate not found in table` inside `_convert_arrow_to_proto`, and the handlers above absorb it. With the aggregate column names in the schema instead, the next error is `KeyError: 'Field \"event_timestamp\" does not exist in schema'`. So it looks like the Ray engine cannot currently write an aggregated BatchFeatureView to the online store at all, which matches the `TODO` at `feast/utils.py` 577.\n\n## Steps to reproduce\n\nOn master (386759c), a feature repo with a parquet `FileSource`, the file offline store, the SQLite online store and `batch_engine: {type: ray.engine, max_workers: 1}`:\n\n1. `materialize` once with the online database writable: values 0.8 / 0.7 are served.\n2. Make the online database file read-only (`chmod 0444`), add newer rows (0.1 / 0.2) to the source, and `materialize` again. SQLite raises `attempt to write a readonly database` inside the worker.\n3. The Ray job status is SUCCEEDED, `materialize` returns normally, and `get_online_features` still returns 0.8 / 0.7.\n4. The Local engine on the same read-only database reports ERROR and `materialize` raises.\n\nThe read-only file is a deliberate stand-in for an online store that refuses writes; I have not seen this in a production deployment.\n\n### Specifications\n\n- Version: master at 386759c (15 Oct 2026)\n- Platform: macOS, Python 3.10.7, Ray 2.58.0\n- Subsystem: Ray compute engine (`infra/compute_engines/ray`)\n\n## Possible Solution\n\nA small change (+9/-2 across `ray/utils.py` and `ray/nodes.py`): re-raise after the existing log in the two write functions, and take the write node's batch function out of `safe_batch_processor`, keeping its empty-batch early return. `safe_batch_processor` itself is unchanged (its swallow is pinned by `test_exception_in_func_returns_original_batch`, and other nodes use it). With it, five new tests fail on master and pass, four controls pass both ways, and the existing Ray component tests stay at 75 passed (non-integration) and 13 of 14 (integration).\n\nThe one integration test that turns red is `test_ray_compute_engine_materialize`, for the reason above. That seems like your call, so I have not changed it. A few options:\n\n1. assert ERROR there and track aggregated BatchFeatureView online writes on Ray as their own issue,\n2. drop the aggregations from that test so it tests a plain materialization, or\n3. fix the aggregated write first.\n\nThe intended behaviour change is that a failed online write fails the materialization instead of reporting success, so a pipeline that has been writing nothing will start failing. Happy to open a PR with the change and tests once you have a preference on the test.\n\nThis issue was written with AI assistance (Claude Code); the tests above were run before filing.\n","author":{"url":"https://github.com/ericdelorefice","@type":"Person","name":"ericdelorefice"},"datePublished":"2026-10-15T21:45:11Z","url":"https://github.com/feast-dev/feast/issues/7024"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:3d355d8f-4673-0a32-344f-bfea6ca57c37
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id9AE6:3C86C1:D5EF79:119026D:6AFE8F79
html-safe-noncec5531a7d4fdcf9883958403aa7b80087ee3d9647928ad448c6efbb6d31f38cd3
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5QUU2OjNDODZDMTpENUVGNzk6MTE5MDI2RDo2QUZFOEY3OSIsInZpc2l0b3JfaWQiOiI0MDc1NzQ4NDU2MTU1NDg0MDI1IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac94cc5091b582b15de7e3deb37730658ffe740e8da90fa204c5449eeeb8f55ed2
hovercard-subject-tagissue:5874430095
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/7024/issue_layout
twitter:imagehttps://opengraph.githubassets.com/3744b01a46b5eaee5344e357dc08b9e6760d9a03425a6fffbf68a6a9505940f8/feast-dev/feast/issues/7024
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/3744b01a46b5eaee5344e357dc08b9e6760d9a03425a6fffbf68a6a9505940f8/feast-dev/feast/issues/7024
og:image:altExpected Behavior When the online store refuses a write during feast materialize with the Ray compute engine, the materialization fails (job status ERROR and FeatureStore.materialize raises), the w...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameericdelorefice
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetcanary-1
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/feast-dev/feast/issues/7024#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F7024
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F7024
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/issues/7024
Reloadhttps://github.com/feast-dev/feast/issues/7024
Reloadhttps://github.com/feast-dev/feast/issues/7024
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
https://github.com/feast-dev/feast
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/7024
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/7024
#7026https://github.com/feast-dev/feast/pull/7026
Ray compute engine reports a failed online-store write as a successful materializationhttps://github.com/feast-dev/feast/issues/7024#top
#7026https://github.com/feast-dev/feast/pull/7026
https://github.com/ericdelorefice
ericdeloreficehttps://github.com/ericdelorefice
on Oct 15, 2026https://github.com/feast-dev/feast/issues/7024#issue-5874430095
386759chttps://github.com/feast-dev/feast/commit/386759c1787b1a16a2e1ad7783db00e709cf4b26
386759chttps://github.com/feast-dev/feast/commit/386759c1787b1a16a2e1ad7783db00e709cf4b26
patelchaitanyhttps://github.com/patelchaitany
on Oct 17, 2026https://github.com/feast-dev/feast/issues/7024#event-33523705012
fix: Fail Ray materialization when the online store write failshttps://github.com/patelchaitany/feast/commit/ab5b730438bfd50ef6ddd0aac29c25e0fb1400a0
https://docs.github.com/articles/closing-issues-via-commit-messages
ab5b730https://github.com/patelchaitany/feast/commit/ab5b730438bfd50ef6ddd0aac29c25e0fb1400a0
fix: Fail Ray materialization when the online store write failshttps://github.com/patelchaitany/feast/commit/d492a1d375dc5bc72aac8ef3e576be429a9557e3
https://docs.github.com/articles/closing-issues-via-commit-messages
d492a1dhttps://github.com/patelchaitany/feast/commit/d492a1d375dc5bc72aac8ef3e576be429a9557e3
ntkatholehttps://github.com/ntkathole
on Oct 18, 2026https://github.com/feast-dev/feast/issues/7024#event-33575885096
fix: Fail Ray materialization when the online store write failshttps://github.com/patelchaitany/feast/commit/e0e51f76adcd9703d7f10de9fbf96b823b7109e7
https://docs.github.com/articles/closing-issues-via-commit-messages
e0e51f7https://github.com/patelchaitany/feast/commit/e0e51f76adcd9703d7f10de9fbf96b823b7109e7
ntkatholehttps://github.com/ntkathole
completedhttps://github.com/feast-dev/feast/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#7026https://github.com/feast-dev/feast/pull/7026
on Oct 18, 2026https://github.com/feast-dev/feast/issues/7024#event-33575903608
odh-devops-apphttps://github.com/apps/odh-devops-app
on Oct 18, 2026https://github.com/feast-dev/feast/issues/7024#event-33579931581
fix: Stop the Ray engine from hiding materialization errors (feast-de…https://github.com/opendatahub-io/feast/commit/7903e2edc615b4dbbcdf158a8c2921da06867d1b
feast-de…https://github.com/feast-dev/feast/pull/7026
https://docs.github.com/articles/closing-issues-via-commit-messages
7903e2ehttps://github.com/opendatahub-io/feast/commit/7903e2edc615b4dbbcdf158a8c2921da06867d1b
franciscojavierarceohttps://github.com/franciscojavierarceo
on Nov 10, 2026https://github.com/feast-dev/feast/issues/7024#event-35734995530
chore(release): release 0.68.0https://github.com/feast-dev/feast/commit/fb5e5d5e1df1903f41ac738cd49adac55ef44b88
fb5e5d5https://github.com/feast-dev/feast/commit/fb5e5d5e1df1903f41ac738cd49adac55ef44b88
shuchuhttps://github.com/shuchu
on Nov 11, 2026https://github.com/feast-dev/feast/issues/7024#event-35759493044
chore(release): release 0.68.0https://github.com/shuchu/feast/commit/5b94c9eb145d3fe7b84782133961639dc87ee951
5b94c9ehttps://github.com/shuchu/feast/commit/5b94c9eb145d3fe7b84782133961639dc87ee951
Sign up for freehttps://github.com/signup?return_to=https://github.com/feast-dev/feast/issues/7024
Sign in to commenthttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/7024
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.