René's URL Explorer Experiment
Title: fix: Keep the source event timestamp separate from request data in Dask retrieval by iammanoj807 · Pull Request #7177 · feast-dev/feast · GitHub
Open Graph Title: fix: Keep the source event timestamp separate from request data in Dask retrieval by iammanoj807 · Pull Request #7177 · feast-dev/feast
X Title: fix: Keep the source event timestamp separate from request data in Dask retrieval by iammanoj807 · Pull Request #7177 · feast-dev/feast
Description: What this PR does / why we need it:
In the file (Dask) offline store, get_historical_features merges each feature view's source data into the entity dataframe with suffixes=("", "__"). When the entity dataframe already has a column named like the source's timestamp_field, the source column becomes __, but the TTL filter, the created-timestamp cutoff, deduplication and the final column drop all keep using , which is now the request's column. As a result:
feature values come back missing, or from after the event timestamp;
the request's own column is dropped, and the source timestamp comes back as __;
a request column of that name holding strings, integers or floats raises AttributeError: 'str' object has no attribute 'tzinfo';
a request with both ts and ts__ raises Duplicate column names found, or, with two feature views using ts, keeps allocating memory.
#7071 fixed this for the source's created_timestamp_column by keeping it under a temporary name. This PR does the same for timestamp_field, through a small _unused_column_name helper now shared by both renames. Because the check runs after _field_mapping, it also covers its __ alias when the request has a column with that name.
Adds test_dask_timestamp_field_collision.py (22 tests, through the public FeatureStore API with local Parquet files, like #7071's tests):
a request column named like the source timestamp field, holding a later, earlier or in-between timestamp, a string or an integer, with both settings of full_feature_names and filter_by_created_timestamp, next to request columns that take the temporary names;
a request column named like the __event_timestamp alias;
two feature views sharing a colliding timestamp field, with ts and ts__ in the request.
On master, the first 21 fail (9 missing values, 4 values from after the event timestamp, 8 errors) and the last one runs out of memory. All pass with this change.
Also adds test_historical_features_with_request_column_named_like_timestamp_field to the universal historical retrieval tests, so the case runs on every universal offline store. The source's timestamp_field is feature_timestamp, and the entity dataframe has its own feature_timestamp request column (later than, earlier than and between the source rows). It checks the latest eligible feature value for each request and that the request columns come back unchanged, with and without full feature names.
Which issue(s) this PR fixes:
Fixes #7176
Checks
I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format
Testing Strategy
Unit tests
Integration tests
Manual tests
Testing is not required for this change
Misc
Run locally:
Offline store unit tests, plus the other unit test files that call get_historical_features or the Dask store: the same results as master, plus the 22 new tests. The failures and errors are the same on master (optional dependencies not installed locally: Spark, Trino, Postgres, MLflow, Lance).
File and DuckDB universal offline store integration tests (sdk/python/tests/integration/offline_store): 94 passed and 5 skipped, which is the 90 that pass without the new test plus its 4 cases. Without the fix, the new test fails on the file store (the request column is dropped and feature_timestamp__ is added) and passes on DuckDB. test_dqm_validation.py is not collected locally.
ruff 0.16.10 passes on the changed files; mypy reports no new errors in dask.py.
Manual check: the reproducer from the issue, on master, 0.68.0 and this branch.
Randomised comparison against master, DuckDB and a brute-force point-in-time reference: 800 get_historical_features calls on the file store, with 1–3 feature views per call (timestamp fields ts, feature_ts and event_timestamp; with and without created columns; TTL set, 0 and unset), both full_feature_names settings, both filter_by_created_timestamp settings, and request columns named like the source timestamp and created columns, their temporary names, feature names and other labels, holding UTC, naive, string, integer and float values.
147 calls had a request column named like a source timestamp field. Master got all 147 wrong (63 raised, 84 returned wrong or missing values or dropped the request column). This branch matched the reference on every value and returned the request columns unchanged in 143. Each of the other 4 is a separate, existing behaviour that is the same on master: 2 have a request column named like a requested feature (DuckDB raises Duplicate column name there), and 2 have request rows sharing a join key and timestamp (#6786).
The other 653 calls gave identical results on master and this branch.
DuckDB matched the reference on every call with a timestamp name collision.
Open Graph Description: What this PR does / why we need it:
In the file (Dask) offline store, get_historical_features merges each feature view's source data into the entity dataframe with suffixes=("", "...
X Description: What this PR does / why we need it:
In the file (Dask) offline store, get_historical_features merges each feature view's source data into the entity dataframe with suffixes=("&...
Opengraph URL: https://github.com/feast-dev/feast/pull/7177
X: @github
direct link
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:b7cca4d9-70b3-7017-ae3b-3228161cf2a2 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | EA68:30C434:D4D18C:124CE6F:6AFE9276 |
| html-safe-nonce | 95ad42f8b58f86efc6d2533f38d75676208194214b632b3b556cfa41337d2d54 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQTY4OjMwQzQzNDpENEQxOEM6MTI0Q0U2Rjo2QUZFOTI3NiIsInZpc2l0b3JfaWQiOiI2MTU0NTcyMDU2ODgxOTU5NTQyIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | cbbb75e246192a73fa4cc267444c0fe891f97ad3900ff15aa93834da49f9f6b2 |
| hovercard-subject-tag | pull_request:5371820033 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | ///pull_requests/show/files |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/7177/files |
| twitter:image | https://avatars.githubusercontent.com/u/207680204?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/207680204?s=400&v=4 |
| og:image:alt | What this PR does / why we need it:
In the file (Dask) offline store, get_historical_features merges each feature view's source data into the entity dataframe with suffixes=("", "... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
| Skip to content | https://github.com/feast-dev/feast/pull/7177/files#start-of-content |
| https://github.com/ |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F7177%2Ffiles |
| GitHubRepository hosting, code review, and collaboration | https://github.com/platform |
| CopilotAI coding agent for tasks, projects, and automated work | https://github.com/features/copilot |
| ActionsCI/CD and automation | https://github.com/features/actions |
| Code QualityQuality analysis, fixes, and enforcement | https://github.com/features/code-quality |
| CodespacesCloud development environments | https://github.com/features/codespaces |
| Code SecurityVulnerability detection and remediation | https://github.com/security/advanced-security/code-security |
| Secret ProtectionSecret scanning and leak prevention | https://github.com/security/advanced-security/secret-protection |
| All Features | https://github.com/features |
| Documentation | https://docs.github.com/en |
| Training | https://learn.github.com/skills |
| Certifications | https://learn.github.com/credentials |
| Build with Copilot | https://github.com/features/ai |
| Customize Copilot | https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview |
| Integrations | https://github.com/marketplace |
| Trending repos | https://github.com/trending |
| GitHub Sponsors | https://github.com/open-source/sponsors |
| Developer articles | https://github.com/resources/articles |
| ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place. | https://github.blog/changelog/ |
| Enterprises | https://github.com/enterprise |
| Team | https://github.com/team |
| Startups | https://github.com/enterprise/startups |
| Nonprofits | https://github.com/solutions/industry/nonprofits |
| Healthcare | https://github.com/solutions/industry/healthcare |
| Financial services | https://github.com/solutions/industry/financial-services |
| Manufacturing | https://github.com/solutions/industry/manufacturing |
| Government | https://github.com/solutions/industry/government |
| See all industries | https://github.com/solutions/industry |
| Why GitHub | https://github.com/why-github |
| Customer Stories | https://github.com/customer-stories |
| Business insights | https://github.com/solutions/executive-insights |
| Trust Center | https://github.com/trust-center |
| GitHub Advanced SecurityEnterprise-grade code security | https://github.com/security/advanced-security |
| Copilot for businessEnterprise-grade AI features | https://github.com/features/copilot/copilot-business |
| Premium SupportEnterprise-grade 24/7 support | https://github.com/enterprise/premium-support |
| Documentation | https://docs.github.com/en |
| Resource center | https://github.com/resources |
| Blog | https://github.blog/ |
| Events & webinars | https://github.com/resources/events |
| Community Forum | https://github.com/orgs/community/discussions |
| Customer Stories | https://github.com/customer-stories |
| Integrations | https://github.com/marketplace |
| Customer support | https://help.github.com/ |
| Training | https://learn.github.com/skills |
| GitHub Status | https://www.githubstatus.com/ |
| About | https://github.com/about |
| Partners | https://github.com/partners |
| Changelog | https://github.blog/changelog/ |
| GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub. | https://githubuniverse.com/ |
| Pricing | https://github.com/pricing |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ffeast-dev%2Ffeast%2Fpull%2F7177%2Ffiles |
| Sign up | https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=feast-dev%2Ffeast |
| Reload | https://github.com/feast-dev/feast/pull/7177/files |
| Reload | https://github.com/feast-dev/feast/pull/7177/files |
| Reload | https://github.com/feast-dev/feast/pull/7177/files |
| feast-dev | https://github.com/feast-dev |
| feast | https://github.com/feast-dev/feast |
| Code | https://github.com/feast-dev/feast |
| Issues223 (223) | https://github.com/feast-dev/feast/issues |
| Pull requests143 (143) | https://github.com/feast-dev/feast/pulls |
| Discussions | https://github.com/feast-dev/feast/discussions |
| Actions | https://github.com/feast-dev/feast/actions |
| Security and quality1 (1) | https://github.com/feast-dev/feast/security |
| Insights | https://github.com/feast-dev/feast/insights |
| Sign up for GitHub
| https://github.com/signup?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose |
| terms of service | https://docs.github.com/terms |
| privacy statement | https://docs.github.com/privacy |
| Sign in | https://github.com/login?return_to=%2Ffeast-dev%2Ffeast%2Fissues%2Fnew%2Fchoose |
| iammanoj807 | https://github.com/iammanoj807 |
| feast-dev:master | https://github.com/feast-dev/feast/tree/master |
| iammanoj807:fix/dask-timestamp-field-request-column | https://github.com/iammanoj807/feast/tree/fix/dask-timestamp-field-request-column |
|
Conversation
5
| https://github.com/feast-dev/feast/pull/7177 |
|
Commits
2
| https://github.com/feast-dev/feast/pull/7177/commits |
|
Checks
24
| https://github.com/feast-dev/feast/pull/7177/checks |
|
Files changed
| https://github.com/feast-dev/feast/pull/7177/files |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
|
fix: Keep the source event timestamp separate from request data in Dask retrieval
| https://github.com/feast-dev/feast/pull/7177/files#top |
|
Show all changes
2 commits
| https://github.com/feast-dev/feast/pull/7177/files |
|
3b15684
fix: Keep the source event timestamp separate from request data in Da…
iammanoj807 Nov 18, 2026
| https://github.com/feast-dev/feast/pull/7177/commits/3b156840c717b001e4ac50f56036e630782ca153 |
|
bb0ca85
test: Cover request columns named like the source timestamp field in …
iammanoj807 Nov 18, 2026
| https://github.com/feast-dev/feast/pull/7177/commits/bb0ca8595a16f99f916dd227360ce1ffffca5284 |
|
Clear filters
| https://github.com/feast-dev/feast/pull/7177/files |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
|
dask.py
| https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
test_universal_historical_retrieval.py
| https://github.com/feast-dev/feast/pull/7177/files#diff-3bb5de0560a06ec0e2cc747437f20195a574512498e0e628382bf74134e8542b |
|
test_dask_timestamp_field_collision.py
| https://github.com/feast-dev/feast/pull/7177/files#diff-bcc290ba7318c645557ba6dac033ef44a6221931616ed6a6e32bff7dd38debbb |
|
| https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10 |
| sdk/python/feast/infra/offline_stores/dask.py | https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
View file
| https://github.com/feast-dev/feast/blob/bb0ca8595a16f99f916dd227360ce1ffffca5284/sdk/python/feast/infra/offline_stores/dask.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/feast-dev/feast/pull/7177/{{ revealButtonHref }} |
|
| https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
| https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
| https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
| https://github.com/feast-dev/feast/pull/7177/files#diff-3de9c84e29516229969edd4207d459ab105da1919b3cd1869a6289c316b5575e |
|
| https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10 |
| sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py | https://github.com/feast-dev/feast/pull/7177/files#diff-3bb5de0560a06ec0e2cc747437f20195a574512498e0e628382bf74134e8542b |
|
View file
| https://github.com/feast-dev/feast/blob/bb0ca8595a16f99f916dd227360ce1ffffca5284/sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/feast-dev/feast/pull/7177/{{ revealButtonHref }} |
|
| https://github.com/feast-dev/feast/pull/7177/files#diff-3bb5de0560a06ec0e2cc747437f20195a574512498e0e628382bf74134e8542b |
|
| https://github.com/feast-dev/feast/blob/master/CODEOWNERS#L10 |
| sdk/python/tests/unit/infra/offline_stores/test_dask_timestamp_field_collision.py | https://github.com/feast-dev/feast/pull/7177/files#diff-bcc290ba7318c645557ba6dac033ef44a6221931616ed6a6e32bff7dd38debbb |
|
View file
| https://github.com/feast-dev/feast/blob/bb0ca8595a16f99f916dd227360ce1ffffca5284/sdk/python/tests/unit/infra/offline_stores/test_dask_timestamp_field_collision.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/feast-dev/feast/pull/7177/{{ revealButtonHref }} |
| franciscojavierarceo | https://github.com/franciscojavierarceo |
| Nov 18, 2026 | https://github.com/feast-dev/feast/pull/7177/files#r4612936714 |
| Learn more | https://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
| iammanoj807 | https://github.com/iammanoj807 |
| Nov 18, 2026 | https://github.com/feast-dev/feast/pull/7177/files#r4613884090 |
| Learn more | https://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment |
| bb0ca85 | https://github.com/feast-dev/feast/commit/bb0ca8595a16f99f916dd227360ce1ffffca5284 |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
| Please reload this page | https://github.com/feast-dev/feast/pull/7177/files |
|
| https://github.com |
| Terms | https://docs.github.com/site-policy/github-terms/github-terms-of-service |
| Privacy | https://docs.github.com/site-policy/privacy-policies/github-privacy-statement |
| Security | https://github.com/security |
| Status | https://www.githubstatus.com/ |
| Community | https://github.community/ |
| Docs | https://docs.github.com/ |
| Contact | https://support.github.com?tags=dotcom-footer |
Viewport: width=device-width
URLs of crawlers that visited me.