Title: fix: Fall back to the serialized body when UDF source misses globals by Daksha1611 · Pull Request #6816 · feast-dev/feast · GitHub
Open Graph Title: fix: Fall back to the serialized body when UDF source misses globals by Daksha1611 · Pull Request #6816 · feast-dev/feast
X Title: fix: Fall back to the serialized body when UDF source misses globals by Daksha1611 · Pull Request #6816 · feast-dev/feast
Description: What this PR does / why we need it: Source-first UDF rehydration execs only the UDF's own source into a namespace seeded with pandas/numpy. A UDF that reads a helper, constant, or aliased import from its defining module cannot resolve those names — but the exec still succeeds, because a function body's free variables are only looked up when the function is called. resolve_udf therefore treated rehydration as successful and never fell back to the dill body, even when that body carried the captured globals and would have run. The failure surfaced later as a NameError during retrieval, so an on-demand feature view that served correctly before started raising once it was loaded from the registry: NameError: name 'scaled' is not defined This validates the rebuilt callable before accepting it: walk its LOAD_GLOBAL operands, plus those of nested code objects (inner functions, comprehensions), and return None when a name resolves in neither the exec namespace nor builtins. resolve_udf then falls back to the serialized body exactly as it did before. Only LOAD_GLOBAL operands are inspected. co_names also contains attribute names (df.columns), so checking that instead would flag working UDFs and push them onto the dill path for no reason — there's a regression test covering that. Self-contained UDFs are unaffected and still take the source path, so the Spark segfault and cross-Python-version benefits of source-first rehydration are kept intact. Which issue(s) this PR fixes: Fixes #6815 Checks I've made sure the tests are passing. My commits are signed off (git commit -s) My PR title follows conventional commits format Testing Strategy Unit tests Integration tests Manual tests Testing is not required for this change Four regression tests added to sdk/python/tests/unit/transformation/test_udf_rehydrate.py. Three of them fail on master and pass here; the fourth (test_attribute_access_is_not_ mistaken_for_a_missing_global) passes both ways by design, guarding the co_names false-positive trap described above. Verified end to end as well: an ODFV applied from a script, then read back in a separate process holding only the registry, returned NameError before this change and the correct value after. sdk/python/tests/unit filtered to the transformation / on-demand / feature-view areas: 314 passed, 5 skipped. The 9 errors in that run are MongoDB testcontainers failing to start locally and are present on master too. ruff check, ruff format --check and mypy are clean on both changed files. Misc The pre-existing test_resolve_udf_prefers_source_over_garbage_dill_body still passes, which confirms self-contained UDFs continue to prefer source over the body.
Open Graph Description: What this PR does / why we need it: Source-first UDF rehydration execs only the UDF's own source into a namespace seeded with pandas/numpy. A UDF that reads a helper, constant, or aliased impor...
X Description: What this PR does / why we need it: Source-first UDF rehydration execs only the UDF's own source into a namespace seeded with pandas/numpy. A UDF that reads a helper, constant, or aliased i...
Opengraph URL: https://github.com/feast-dev/feast/pull/6816
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:c72c502f-6510-1e7f-ec7c-38539b3caff4 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 9676:29B1E5:D7CE9:120CCF:6AFE9269 |
| html-safe-nonce | 28ed40eccab930e0d7fab35792920ca768de0bfeecadb2b609b1370a352b3744 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5Njc2OjI5QjFFNTpEN0NFOToxMjBDQ0Y6NkFGRTkyNjkiLCJ2aXNpdG9yX2lkIjoiNTkwNjc4NjMzOTc4NDY1OTU2MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 408a066fc96c4856baa732eaa53473a253634c0b72bd7c323427b2d09379de82 |
| hovercard-subject-tag | pull_request:4457365818 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6816/files |
| twitter:image | https://avatars.githubusercontent.com/u/113719658?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/113719658?s=400&v=4 |
| og:image:alt | What this PR does / why we need it: Source-first UDF rehydration execs only the UDF's own source into a namespace seeded with pandas/numpy. A UDF that reads a helper, constant, or aliased impor... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width