René's URL Explorer Experiment


Title: spark_application engine reports successful single-feature-view materializations as failure · Issue #6673 · feast-dev/feast · GitHub

Open Graph Title: spark_application engine reports successful single-feature-view materializations as failure · Issue #6673 · feast-dev/feast

X Title: spark_application engine reports successful single-feature-view materializations as failure · Issue #6673 · feast-dev/feast

Description: Expected Behavior When a SparkApplication reaches COMPLETED, Feast should report the materialization as successful. Current Behavior Every successful single-feature-view materialization using the spark_application batch engine is reporte...

Open Graph Description: Expected Behavior When a SparkApplication reaches COMPLETED, Feast should report the materialization as successful. Current Behavior Every successful single-feature-view materialization using the s...

X Description: Expected Behavior When a SparkApplication reaches COMPLETED, Feast should report the materialization as successful. Current Behavior Every successful single-feature-view materialization using the s...

Opengraph URL: https://github.com/feast-dev/feast/issues/6673

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"spark_application engine reports successful single-feature-view materializations as failure","articleBody":"## Expected Behavior\n\nWhen a SparkApplication reaches `COMPLETED`, Feast should report the materialization as successful.\n\n## Current Behavior\n\nEvery successful single-feature-view materialization using the `spark_application` batch engine is reported as a failure:\n\n```text\nException: SparkApplication feast-sa-\u003cjob-id> not found\n```\n\nThis is deterministic: Feast deletes the SparkApplication CR before performing the status check that fails.\n\n`SparkApplicationComputeEngine.materialize()` waits for the job to reach a terminal status and then unconditionally deletes the CR:\n\n```python\ntry:\n    self._wait_for_completion(job)\n    return self._build_per_fv_jobs(registry, tasks, job_id, job)\nfinally:\n    self._cleanup(job_id)\n```\n\nAfter `materialize()` returns, `FeatureStore._submit_and_process_materialization_jobs()` calls `job.status()` again. For a single feature view, `_build_per_fv_jobs()` returns the live polling job:\n\n```python\nif len(tasks) \u003c= 1:\n    return [job for _ in tasks]\n```\n\nThe second status check therefore queries the CR that `_cleanup()` just deleted. The resulting 404 sets `_error`, changes the status to `ERROR`, and is subsequently raised by `FeatureStore`.\n\nBy this point, the Spark driver has already materialized the features and updated the registry. Automation calling `feast materialize` or the feature server's `/materialize` endpoint therefore sees a failure and may retry work that already succeeded.\n\nThis is not caused by the Spark Operator's `timeToLiveSeconds`; Feast explicitly deletes the CR immediately after `_wait_for_completion()` returns.\n\nThe relevant flow is still present on `master`:\n\n- [Cleanup after completion](https://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/compute.py#L224-L228)\n- [Single-FV path returns the polling job](https://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/compute.py#L266-L267)\n- [Status re-queries the Kubernetes API](https://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/job.py#L94-L115)\n- [404 changes the job to an error](https://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/job.py#L129-L134)\n- [FeatureStore performs the second status check](https://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/feature_store.py#L563-L569)\n\n## Steps to reproduce\n\n1. Configure a Feast project with `batch_engine.type: spark_application`.\n2. Apply the feature definitions.\n3. Materialize exactly one feature view.\n4. Wait for the SparkApplication to reach `COMPLETED`.\n5. Observe Feast delete the CR and report:\n\n   ```text\n   SparkApplication feast-sa-\u003cjob-id> not found\n   ```\n\nThe feature-view count matters. For multiple feature views, `_build_per_fv_jobs()` returns `CompletedMaterializationJob` instances for successful views. Their `status()` method returns `SUCCEEDED` without querying Kubernetes, so the successful multi-FV path does not exhibit this bug.\n\nThe failure can also be reproduced without a cluster:\n\n```python\nfrom kubernetes.client.exceptions import ApiException\n\nfrom feast.infra.compute_engines.spark_application.job import (\n    SparkApplicationMaterializationJob,\n)\n\n\nclass Api:\n    \"\"\"Return COMPLETED once, then simulate the CR having been deleted.\"\"\"\n\n    def __init__(self):\n        self.calls = 0\n\n    def get_namespaced_custom_object(self, **kwargs):\n        self.calls += 1\n        if self.calls == 1:\n            return {\n                \"status\": {\n                    \"applicationState\": {\n                        \"state\": \"COMPLETED\",\n                    }\n                }\n            }\n        raise ApiException(status=404, reason=\"Not Found\")\n\n\njob = SparkApplicationMaterializationJob(\"abc123\", \"default\", Api())\n\nprint(job.status())  # SUCCEEDED: observed by _wait_for_completion\nprint(job.status())  # ERROR: observed by FeatureStore after cleanup\nprint(job.error())   # SparkApplication feast-sa-abc123 not found\n```\n\n### Specifications\n\n- Version: Feast v0.65.0; also present on `master` at `ca355cb`\n- Platform: Kubernetes/EKS with the Kubeflow Spark Operator (`sparkoperator.k8s.io/v1beta2`)\n- Subsystem: `spark_application` batch compute engine and materialization\n\n## Possible Solution\n\nCache the first terminal status (`SUCCEEDED` or `ERROR`) in `SparkApplicationMaterializationJob` and return it on subsequent calls instead of polling Kubernetes again. Non-terminal states should continue polling normally.\n\nAn alternative is to return a `CompletedMaterializationJob` from the successful single-FV path, matching the existing multi-FV behavior. However, caching terminal state is more robust because it prevents any future caller from losing a known result after cleanup.\n\nRegression tests should verify that:\n\n- `COMPLETED` followed by a 404 remains `SUCCEEDED`, with `error()` returning `None`.\n- `FAILED` remains `ERROR` and preserves its `errorMessage`.\n- Non-terminal states continue polling.\n\nI have the terminal-status caching fix and regression tests working locally and can open a PR","author":{"url":"https://github.com/dbbvitor","@type":"Person","name":"dbbvitor"},"datePublished":"2026-07-30T14:33:43Z","url":"https://github.com/feast-dev/feast/issues/6673"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:04ecce1d-1711-f873-9bbc-a48b8cc01605
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id9D8C:E40BB:EE686B:145E4F2:6AFE9812
html-safe-noncea5224567513d9cb1be83b15658d530cd3816d1df86807692bb347e63e85590e5
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5RDhDOkU0MEJCOkVFNjg2QjoxNDVFNEYyOjZBRkU5ODEyIiwidmlzaXRvcl9pZCI6IjMyNjQ1NTkyMTEzMjA2MTI4ODIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac6040fbd6cad5bc7f8a452c998de3838cf32c32d3f78ee767f2d3f0465ae4fa7a
hovercard-subject-tagissue:5020832465
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/6673/issue_layout
twitter:imagehttps://opengraph.githubassets.com/62059d8982f0452336f503c90943bd449172bd6e3113d540b4d61f330c9b8812/feast-dev/feast/issues/6673
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/62059d8982f0452336f503c90943bd449172bd6e3113d540b4d61f330c9b8812/feast-dev/feast/issues/6673
og:image:altExpected Behavior When a SparkApplication reaches COMPLETED, Feast should report the materialization as successful. Current Behavior Every successful single-feature-view materialization using the s...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamedbbvitor
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/feast-dev/feast/issues/6673#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6673
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6673
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/issues/6673
Reloadhttps://github.com/feast-dev/feast/issues/6673
Reloadhttps://github.com/feast-dev/feast/issues/6673
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
https://github.com/feast-dev/feast
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6673
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6673
#6700https://github.com/feast-dev/feast/pull/6700
spark_application engine reports successful single-feature-view materializations as failurehttps://github.com/feast-dev/feast/issues/6673#top
#6700https://github.com/feast-dev/feast/pull/6700
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
https://github.com/dbbvitor
dbbvitorhttps://github.com/dbbvitor
on Jul 30, 2026https://github.com/feast-dev/feast/issues/6673#issue-5020832465
Cleanup after completionhttps://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/compute.py#L224-L228
Single-FV path returns the polling jobhttps://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/compute.py#L266-L267
Status re-queries the Kubernetes APIhttps://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/job.py#L94-L115
404 changes the job to an errorhttps://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/infra/compute_engines/spark_application/job.py#L129-L134
FeatureStore performs the second status checkhttps://github.com/feast-dev/feast/blob/ca355cb73fdd2e8d955274b9e7e65ada060ddfe3/sdk/python/feast/feature_store.py#L563-L569
dbbvitorhttps://github.com/dbbvitor
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
on Jul 30, 2026https://github.com/feast-dev/feast/issues/6673#event-28726492241
adarshsmhttps://github.com/adarshsm
on Aug 12, 2026https://github.com/feast-dev/feast/issues/6673#event-29333658536
fix: Report single-feature-view spark_application materialization suc…https://github.com/adarshsm/feast/commit/abf7dc5162cbf6bb51fa5ba7e9bdad51b3b55328
https://docs.github.com/articles/closing-issues-via-commit-messages
abf7dc5https://github.com/adarshsm/feast/commit/abf7dc5162cbf6bb51fa5ba7e9bdad51b3b55328
ntkatholehttps://github.com/ntkathole
on Aug 13, 2026https://github.com/feast-dev/feast/issues/6673#event-29396956083
fix: Report single-feature-view spark_application materialization suc…https://github.com/feast-dev/feast/commit/a9219d9672848b6d62c6a44d1001e4487656fe9f
https://docs.github.com/articles/closing-issues-via-commit-messages
a9219d9https://github.com/feast-dev/feast/commit/a9219d9672848b6d62c6a44d1001e4487656fe9f
ntkatholehttps://github.com/ntkathole
completedhttps://github.com/feast-dev/feast/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#6700https://github.com/feast-dev/feast/pull/6700
on Aug 13, 2026https://github.com/feast-dev/feast/issues/6673#event-29396956117
franciscojavierarceohttps://github.com/franciscojavierarceo
on Aug 21, 2026https://github.com/feast-dev/feast/issues/6673#event-29781434385
chore(release): release 0.66.0https://github.com/feast-dev/feast/commit/1d5be950cf718a674cb01eaf0fd0ad6f0a4c6335
1d5be95https://github.com/feast-dev/feast/commit/1d5be950cf718a674cb01eaf0fd0ad6f0a4c6335
HSarwathttps://github.com/HSarwat
on Aug 22, 2026https://github.com/feast-dev/feast/issues/6673#event-29844848763
fix: Report single-feature-view spark_application materialization suc…https://github.com/HSarwat/feast/commit/2af536586983b4f4c78d2c0b96ceec76aa61e4e1
https://docs.github.com/articles/closing-issues-via-commit-messages
2af5365https://github.com/HSarwat/feast/commit/2af536586983b4f4c78d2c0b96ceec76aa61e4e1
Michie-hubhttps://github.com/Michie-hub
on Aug 31, 2026https://github.com/feast-dev/feast/issues/6673#event-30282803508
fix: Report single-feature-view spark_application materialization suc…https://github.com/Michie-hub/feast/commit/204653c93e50888fc3393a12c217a2715fad4841
https://docs.github.com/articles/closing-issues-via-commit-messages
204653chttps://github.com/Michie-hub/feast/commit/204653c93e50888fc3393a12c217a2715fad4841
chore(release): release 0.66.0https://github.com/Michie-hub/feast/commit/a29c9231976fd7c68e51570047f9517f06579041
https://docs.github.com/articles/closing-issues-via-commit-messages
a29c923https://github.com/Michie-hub/feast/commit/a29c9231976fd7c68e51570047f9517f06579041
Sign up for freehttps://github.com/signup?return_to=https://github.com/feast-dev/feast/issues/6673
Sign in to commenthttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6673
kind/bughttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Fbug%22
priority/p2https://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22priority%2Fp2%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.