René's URL Explorer Experiment


Title: Add upstream FeatureView lineage support for PushSource · Issue #6839 · feast-dev/feast · GitHub

Open Graph Title: Add upstream FeatureView lineage support for PushSource · Issue #6839 · feast-dev/feast

X Title: Add upstream FeatureView lineage support for PushSource · Issue #6839 · feast-dev/feast

Description: Is your feature request related to a problem? Please describe. In many real-world production ML architectures, derived features are computed outside of Feast by external stream or batch computation engines. These external pipelines read ...

Open Graph Description: Is your feature request related to a problem? Please describe. In many real-world production ML architectures, derived features are computed outside of Feast by external stream or batch computation...

X Description: Is your feature request related to a problem? Please describe. In many real-world production ML architectures, derived features are computed outside of Feast by external stream or batch computation...

Opengraph URL: https://github.com/feast-dev/feast/issues/6839

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add upstream FeatureView lineage support for PushSource","articleBody":"**Is your feature request related to a problem? Please describe.**\n\nIn many real-world production ML architectures, derived features are computed outside of Feast by external stream or batch computation engines. These external pipelines read upstream features from existing Feast `FeatureView`s and push the transformed features into Feast via a `PushSource` backed by an offline batch store.\n\nCurrently, Feast's static registry lineage only tracks:\n```\nPushSource --> Target FeatureView\n```\nBecause `PushSource` does not support declaring the upstream `FeatureView`s it was computed from, the relationship between the upstream feature views and the push source is lost in both the Feast UI Lineage graph and the registry metadata.\n\n**Describe the solution you'd like**\n\nWe would like `PushSource` to support declaring upstream `FeatureView` dependencies via a `source_views` (or `upstream_feature_views`) parameter:\n\n```python\n# 1. Upstream feature views\nuser_tx_fv = FeatureView(name=\"user_transaction_stats\", ...)\nuser_credit_fv = FeatureView(name=\"user_credit_profile\", ...)\n# 2. PushSource declaring upstream views and offline batch source\nrisk_push_source = PushSource(\n    name=\"risk_calc_pipeline\",\n    batch_source=FileSource(path=\"data/risk_offline.parquet\"),\n    source_views=[user_tx_fv, user_credit_fv],  # \u003c-- Upstream dependencies\n    description=\"External Spark job computing risk scores from transaction & credit features\",\n)\n# 3. Target feature view receiving the pushed features\nuser_risk_fv = FeatureView(\n    name=\"user_risk_target_fv\",\n    entities=[user_entity],\n    ttl=timedelta(days=30),\n    source=risk_push_source,\n    schema=[Field(name=\"risk_score\", dtype=Float32)],\n)\n```\n\nWith this metadata, Feast can render the complete directed graph in the Feast UI Lineage view:\n\n```\n[ user_transaction_stats ] (FeatureView) ──┐\n                                           ├──→ [ risk_calc_pipeline ] (PushSource) ──→ [ user_risk_target_fv ] (FeatureView)\n[ user_credit_profile ]    (FeatureView) ──┘\n```\n\nProposed Implementation Details\n\n1. Protobuf (protos/feast/core/DataSource.proto): Add upstream_feature_views to PushOptions:\n```\nmessage PushOptions {\n  reserved 1;\n  repeated string upstream_feature_views = 2;\n}\n```\n2. Python SDK (sdk/python/feast/data_source.py): Update PushSource constructor, `to_proto()`, and `from_proto()` to accept and serialize `source_views: Optional[List[Union[BaseFeatureView, str]]]`.\n3. Registry Lineage (sdk/python/feast/lineage/registry_lineage.py): Update `RegistryLineageGenerator._parse_direct_relationships()` to emit `EntityRelation(source=FeatureView, target=DataSource)` for each upstream feature view defined on a PushSource.\n4. UI Lineage Parser (ui/src/parsers/parseEntityRelationships.ts): Extract `upstreamFeatureViews` from `dataSources` to draw the corresponding edges in React Flow.\n5. OpenLineage Emitter (sdk/python/feast/openlineage/mappers.py): Ensure `emit_apply()` creates the corresponding input dataset connections for OpenLineage producers.\n\n**Describe alternatives you've considered**\n\n1. Using OnDemandFeatureView (ODFV): ODFV handles on-the-fly transformations at request/serving time, but is not intended for pre-computed, stored, push-ingested features requiring offline batch storage.\n2. Adding metadata in tags: Storing `tags={\"depends_on\": \"user_tx_fv,user_credit_fv\"}` records metadata, but is non-standard and does not render visual lineage relationships in the Feast UI graph.\n3. Emitting manual OpenLineage events via CI/CD: Works if an OpenLineage backend is configured, but does not solve native Feast static registry lineage or out-of-the-box UI visualization.\n\n**Additional context**\nI am happy to contribute this change and submit a PR covering the proto update, Python SDK, registry lineage generator, and UI relationship parser if the community agrees with this direction!","author":{"url":"https://github.com/mao-liu","@type":"Person","name":"mao-liu"},"datePublished":"2026-09-15T10:50:47Z","url":"https://github.com/feast-dev/feast/issues/6839"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:2837d401-244b-4384-83c4-088f16ab3794
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idAFDE:79CF9:1C1C7A:256CB1:6AFE9229
html-safe-nonceaa4dc3c341fcc42a60e15de5b2d87d85d0a508234f07fa0959385baf3d956e67
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBRkRFOjc5Q0Y5OjFDMUM3QToyNTZDQjE6NkFGRTkyMjkiLCJ2aXNpdG9yX2lkIjoiNDQ4OTIyNzI4MTMxODcxMTg0OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacc506ac996488cf86880da4b014beb3771005e09c5a63fd95311c7c2bb790e83f
hovercard-subject-tagissue:5460931111
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/6839/issue_layout
twitter:imagehttps://opengraph.githubassets.com/e82cb4952a9dbc45d5f5ac8a30cc45a0dc42f74ea60e22233cd3bbe1575d6089/feast-dev/feast/issues/6839
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/e82cb4952a9dbc45d5f5ac8a30cc45a0dc42f74ea60e22233cd3bbe1575d6089/feast-dev/feast/issues/6839
og:image:altIs your feature request related to a problem? Please describe. In many real-world production ML architectures, derived features are computed outside of Feast by external stream or batch computation...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamemao-liu
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/feast-dev/feast git https://github.com/feast-dev/feast.git
octolytics-dimension-user_id57027613
octolytics-dimension-user_loginfeast-dev
octolytics-dimension-repository_id161133770
octolytics-dimension-repository_nwofeast-dev/feast
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id161133770
octolytics-dimension-repository_network_root_nwofeast-dev/feast
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/feast-dev/feast/issues/6839#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6839
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fissues%2F6839
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=feast-dev%2Ffeast
Reloadhttps://github.com/feast-dev/feast/issues/6839
Reloadhttps://github.com/feast-dev/feast/issues/6839
Reloadhttps://github.com/feast-dev/feast/issues/6839
feast-devhttps://github.com/feast-dev
feasthttps://github.com/feast-dev/feast
Codehttps://github.com/feast-dev/feast
Issues223 (223)https://github.com/feast-dev/feast/issues
Pull requests143 (143)https://github.com/feast-dev/feast/pulls
Discussionshttps://github.com/feast-dev/feast/discussions
Actionshttps://github.com/feast-dev/feast/actions
Security and quality1 (1)https://github.com/feast-dev/feast/security
Insightshttps://github.com/feast-dev/feast/insights
https://github.com/feast-dev/feast
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6839
New issuehttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6839
#6853https://github.com/feast-dev/feast/pull/6853
Add upstream FeatureView lineage support for PushSourcehttps://github.com/feast-dev/feast/issues/6839#top
#6853https://github.com/feast-dev/feast/pull/6853
https://github.com/mao-liu
kind/featureNew feature or requesthttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Ffeature%22
https://github.com/mao-liu
mao-liuhttps://github.com/mao-liu
on Sep 15, 2026https://github.com/feast-dev/feast/issues/6839#issue-5460931111
mao-liuhttps://github.com/mao-liu
kind/featureNew feature or requesthttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Ffeature%22
on Sep 15, 2026https://github.com/feast-dev/feast/issues/6839#event-31166431067
https://github.com/ntkathole
ntkatholehttps://github.com/ntkathole
on Sep 15, 2026https://github.com/feast-dev/feast/issues/6839#issuecomment-5679283194
@mao-liuhttps://github.com/mao-liu
https://github.com/mao-liu
mao-liuhttps://github.com/mao-liu
on Sep 17, 2026https://github.com/feast-dev/feast/issues/6839#issuecomment-5721630769
@ntkatholehttps://github.com/ntkathole
OpenLineageProcessorhttps://github.com/feast-dev/feast/blob/fa8f06b1f41fd4518a788fcc2fd9e50791643b0d/sdk/python/feast/openlineage/processor.py#L315
https://github.com/ntkathole
ntkatholehttps://github.com/ntkathole
on Sep 18, 2026https://github.com/feast-dev/feast/issues/6839#issuecomment-5725961997
@mao-liuhttps://github.com/mao-liu
ntkatholehttps://github.com/ntkathole
mao-liuhttps://github.com/mao-liu
on Sep 18, 2026https://github.com/feast-dev/feast/issues/6839#event-31371035788
https://github.com/mao-liu
mao-liuhttps://github.com/mao-liu
on Sep 20, 2026https://github.com/feast-dev/feast/issues/6839#issuecomment-5753495861
#6853https://github.com/feast-dev/feast/pull/6853
#6852https://github.com/feast-dev/feast/issues/6852
haoxu0https://github.com/haoxu0
completedhttps://github.com/feast-dev/feast/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#6853https://github.com/feast-dev/feast/pull/6853
on Oct 16, 2026https://github.com/feast-dev/feast/issues/6839#event-33491419140
Sign up for freehttps://github.com/signup?return_to=https://github.com/feast-dev/feast/issues/6839
Sign in to commenthttps://github.com/login?return_to=https://github.com/feast-dev/feast/issues/6839
mao-liuhttps://github.com/mao-liu
kind/featureNew feature or requesthttps://github.com/feast-dev/feast/issues?q=state%3Aopen%20label%3A%22kind%2Ffeature%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.