Title: feat: Integrate Chronon backfills with scalable historical retrieval · Issue #6983 · feast-dev/feast · GitHub
Open Graph Title: feat: Integrate Chronon backfills with scalable historical retrieval · Issue #6983 · feast-dev/feast
X Title: feat: Integrate Chronon backfills with scalable historical retrieval · Issue #6983 · feast-dev/feast
Description: Problem Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved datasets. The current adapter still loads p...
Open Graph Description: Problem Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved dat...
X Description: Problem Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved dat...
Opengraph URL: https://github.com/feast-dev/feast/issues/6983
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"feat: Integrate Chronon backfills with scalable historical retrieval","articleBody":"## Problem\n\nFeast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved datasets. The current adapter still loads projected data into pandas and performs joins locally. Backfill execution and connecting its output to Feast remain manual.\n\nML platform teams should be able to use Feast's historical retrieval APIs with Chronon-backed computation at a scale that does not require the full source dataset in the Feast client's memory.\n\n## Proposed solution\n\nImplement this in stages:\n\n1. Define an execution contract mapping a Feast feature selection to a versioned Chronon configuration, entity keys/timestamps, execution target, and output location. Distinguish reading an existing materialization from explicitly requesting a new backfill.\n2. Add an optional backend that submits or attaches to a job through a supported Chronon execution integration. Expose job status, failures, timeout/cancellation behavior, and output location through the retrieval workflow.\n3. Support registering or persisting remotely produced results without downloading the entire dataset. Make local dataframe collection explicit, and reject unsupported transformations instead of silently falling back to unbounded local execution.\n\nKeep the existing local Parquet path compatible. Chronon retains ownership of feature computation and job execution; registration alone must not launch jobs.\n\n## Acceptance criteria\n\n- [ ] A runnable integration example executes a real Chronon backfill and consumes its output through `get_historical_features()` and Feast dataset APIs.\n- [ ] The execution contract supports a documented remote entity-input format and records configuration/input versions, job identity, and output provenance.\n- [ ] Regression tests preserve documented point-in-time, TTL, timestamp, field-mapping, feature-name, missing-value, and entity-row semantics; unsupported combinations fail clearly.\n- [ ] A reproducible workload exceeding the configured client memory budget completes through the distributed path without loading the full source or result into pandas.\n- [ ] Retries do not duplicate completed work or expose partial outputs; job failure, timeout, and cancellation behavior is tested.\n- [ ] Document backend setup, credentials, execution ownership, supported on-demand transformations, operational limits, and migration from the current workflow.\n\n## Alternatives considered\n\nContinue externally orchestrating Chronon and manually publishing Parquet for local retrieval, or hand-wire Chronon outputs to another Feast offline store. These remain useful options, but leave users to maintain the handoff and execution semantics.\n\n## Context\n\nFollow-up to #6188; related definition/metadata work: #6982.\n\nStarting points: [Chronon offline adapter](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/contrib/chronon_offline_store/chronon.py), [current historical retrieval documentation](https://github.com/feast-dev/feast/blob/master/docs/reference/offline-stores/chronon.md), and [retrieval regression tests](https://github.com/feast-dev/feast/tree/master/sdk/python/tests/unit/infra/offline_stores/contrib/chronon_offline_store).\n\nBackend choice and the submission API need a design proposal before implementation.\n","author":{"url":"https://github.com/franciscojavierarceo","@type":"Person","name":"franciscojavierarceo"},"datePublished":"2026-10-09T04:44:39Z","url":"https://github.com/feast-dev/feast/issues/6983"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:3a41e303-67db-fd60-bf88-b70ed3be9936 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D8FC:252B41:28CC80:38A91B:6AFE9813 |
| html-safe-nonce | 2462414d4f59c180bfb97ef0f94f8c557f4f3dfd945dff218e360fcde9b1267d |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEOEZDOjI1MkI0MToyOENDODA6MzhBOTFCOjZBRkU5ODEzIiwidmlzaXRvcl9pZCI6IjQxMjEyNzA1NzQwMDc2MjE2NTEiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 46f2b22ecb29230f4da128023e9f1fe18b74bf2225bc3070fc5454c3927f9fa0 |
| hovercard-subject-tag | issue:5772899952 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/feast-dev/feast/6983/issue_layout |
| twitter:image | https://opengraph.githubassets.com/ba622925f8eb82b30a389ffe175a0c5ac77dd224653d5eb8e19e7a37095a3654/feast-dev/feast/issues/6983 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/ba622925f8eb82b30a389ffe175a0c5ac77dd224653d5eb8e19e7a37095a3654/feast-dev/feast/issues/6983 |
| og:image:alt | Problem Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved dat... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | franciscojavierarceo |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width