Title: fix: Handle numpy.ndarray from Arrow/Athena in Array(String) materialisation by Abhishek8108 · Pull Request #6388 · feast-dev/feast · GitHub
Open Graph Title: fix: Handle numpy.ndarray from Arrow/Athena in Array(String) materialisation by Abhishek8108 · Pull Request #6388 · feast-dev/feast
X Title: fix: Handle numpy.ndarray from Arrow/Athena in Array(String) materialisation by Abhishek8108 · Pull Request #6388 · feast-dev/feast
Description: Summary Closes #6325 Materialising feature views with Array(String) columns via the Athena offline store fails with TypeError or ValueError. Arrow/Athena deserialises these columns as numpy.ndarray with object dtype instead of plain Python lists, and two code paths in type_map.py do not handle this. Root causes 1. _validate_collection_item_types — TypeError on None elements inside ndarrays Nullable Arrow columns embed None elements inside the ndarray. The validator iterates over the ndarray directly and checks type(item) in valid_types. For STRING_LIST, valid_types = [np.str_, str], and type(None) is not in that set, so it raises TypeError: bad argument type for built-in operation. 2. _convert_list_values_to_proto generic path — TypeError from protobuf The generic list conversion calls StringList(val=value) where value is a numpy.ndarray. Protobuf rejects non-list inputs with TypeError: bad argument type for built-in operation. Fix In _validate_collection_item_types: coerce ndarray to a plain Python list via .tolist() before element-level checks, and skip None elements (nullable elements are valid in Arrow columns). In the generic conversion path of _convert_list_values_to_proto: apply the same .tolist() coercion so protobuf always receives a plain Python list. None elements are stripped (protobuf fixed-type lists cannot hold nulls). The fix covers all three failure modes from #6325: np.array(['a', 'b', 'c']) → ['a', 'b', 'c'] ✓ np.array(['a', None, 'c']) → ['a', 'c'] (None stripped) ✓ np.array([]) → [] ✓ Changes sdk/python/feast/type_map.py — two targeted edits, no API surface change sdk/python/tests/unit/test_type_map.py — four new unit tests in TestAthenaArrayStringConversion Test plan TestAthenaArrayStringConversion::test_string_list_from_ndarray — plain ndarray converts correctly TestAthenaArrayStringConversion::test_string_list_from_ndarray_with_none_elements — None elements stripped, no TypeError TestAthenaArrayStringConversion::test_string_list_from_empty_ndarray — empty ndarray yields empty list, no ValueError TestAthenaArrayStringConversion::test_string_list_mixed_null_and_ndarray_rows — mix of null rows and ndarray rows ruff format --check and ruff check — clean on both modified files Existing test_type_map.py tests unaffected
Open Graph Description: Summary Closes #6325 Materialising feature views with Array(String) columns via the Athena offline store fails with TypeError or ValueError. Arrow/Athena deserialises these columns as numpy.ndarray...
X Description: Summary Closes #6325 Materialising feature views with Array(String) columns via the Athena offline store fails with TypeError or ValueError. Arrow/Athena deserialises these columns as numpy.ndarray...
Opengraph URL: https://github.com/feast-dev/feast/pull/6388
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:e40b15df-9a51-2f6c-8197-34df5654dd4c |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D948:1EA4:42450B0:5DCE2F9:6A50050E |
| html-safe-nonce | eec5cab7d444c8ad7e36f684a5bfacabe21f37effd2de052f3242eae3bd49fc7 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEOTQ4OjFFQTQ6NDI0NTBCMDo1RENFMkY5OjZBNTAwNTBFIiwidmlzaXRvcl9pZCI6IjQ0MTY3MzM4OTQxMTc2ODg1OTAiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 4a7ee97da38eb00b3a9babe594e0e89c774a2d0e772fa402d21effad92ba65b5 |
| hovercard-subject-tag | pull_request:3654391747 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/6388/files |
| twitter:image | https://avatars.githubusercontent.com/u/87538407?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/87538407?s=400&v=4 |
| og:image:alt | Summary Closes #6325 Materialising feature views with Array(String) columns via the Athena offline store fails with TypeError or ValueError. Arrow/Athena deserialises these columns as numpy.ndarray... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 56eec25a3430492dd6ec2a480eb33256a0da5192d5714b0db4f5ab5f33203d94 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | e155b8a71f9284454f73118dfe5e634e74f7afb3 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width