Title: fix(compute-engine/local): Honor field_mapping on join keys in dedup + join nodes by 1fanwang · Pull Request #6395 · feast-dev/feast · GitHub
Open Graph Title: fix(compute-engine/local): Honor field_mapping on join keys in dedup + join nodes by 1fanwang · Pull Request #6395 · feast-dev/feast
X Title: fix(compute-engine/local): Honor field_mapping on join keys in dedup + join nodes by 1fanwang · Pull Request #6395 · feast-dev/feast
Description: What this PR does Fixes #5942. The bug isn't actually Snowflake-specific despite the issue title — it lives in the local compute engine's DAG nodes. Path: _get_column_names() returns the reverse-mapped join keys (e.g. ["USERID"]). pull_latest_from_table_or_query is called with those raw names — correct, the offline store needs them. LocalSourceReadNode.execute renames the result columns via field_mapping.get(col, col) — the table now has user_id. LocalDedupNode and LocalJoinNode then look up column_info.join_keys (still ["USERID"]) on a df whose columns are user_id. pandas raises KeyError: Index(['USERID'], dtype='object') — the exact error in the issue. The existing ColumnInfo class already has mapped properties for timestamp_column and created_timestamp_column (precedent: PR #4886, which fixed the analogous bug for get_historical_features). This PR adds the missing join_keys_columns mapped property and uses it in LocalDedupNode and LocalJoinNode. How was this tested? New regression test test_local_dedup_node_with_field_mapping_on_join_key in tests/unit/infra/compute_engines/local/test_nodes.py. Reproduces the exact KeyError(['USERID']) on master; passes with this PR. All 9 existing tests in test_nodes.py still pass. All 62 tests under tests/unit/infra/compute_engines/ still pass. ruff check, ruff format --check, mypy all clean. Scope Local-engine only. The Spark, Ray, AWS Lambda, k8s, and Snowflake compute engines have the same column_info.join_keys lookup pattern in their nodes and likely the same latent bug. Each is a separate <50 LoC follow-up — happy to file those if maintainers want them in this PR or as stacked.
Open Graph Description: What this PR does Fixes #5942. The bug isn't actually Snowflake-specific despite the issue title — it lives in the local compute engine's DAG nodes. Path: _get_column_names() returns the r...
X Description: What this PR does Fixes #5942. The bug isn't actually Snowflake-specific despite the issue title — it lives in the local compute engine's DAG nodes. Path: _get_column_names() retur...
Opengraph URL: https://github.com/feast-dev/feast/pull/6395
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:6caff467-6346-a4e8-9ba1-a40542f639ad |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | DE48:4F8A:1350BEC:1A27E36:6A4EEC0B |
| html-safe-nonce | 0766a5352975fb089edc290405b1bb3117c55c75255b8262dc5b08dbf7efc350 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJERTQ4OjRGOEE6MTM1MEJFQzoxQTI3RTM2OjZBNEVFQzBCIiwidmlzaXRvcl9pZCI6IjYxMzg5MTcxMDcyMzQ5NTgzNDciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | e4e8990bdefe0adbb8e3961cbef821a33bd24acf8d3c501f800987b5b6e34dab |
| hovercard-subject-tag | pull_request:3667339023 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6395/files |
| twitter:image | https://avatars.githubusercontent.com/u/21269208?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/21269208?s=400&v=4 |
| og:image:alt | What this PR does Fixes #5942. The bug isn't actually Snowflake-specific despite the issue title — it lives in the local compute engine's DAG nodes. Path: _get_column_names() returns the r... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 00d04d6b867d2f16ab50bebae8ccc56e07c8988b |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width