René's URL Explorer Experiment

Title: feat: Add Set as feature type by Copilot · Pull Request #5888 · feast-dev/feast · GitHub

Open Graph Title: feat: Add Set as feature type by Copilot · Pull Request #5888 · feast-dev/feast

X Title: feat: Add Set as feature type by Copilot · Pull Request #5888 · feast-dev/feast

Description: What this PR does / why we need it: This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values with automatic deduplication. Changes Made 1. Protobuf Definitions (Value.proto) Added 8 new SET type variants for primitive types: INT32_SET, INT64_SET, FLOAT_SET, DOUBLE_SET STRING_SET, BYTES_SET, BOOL_SET, UNIX_TIMESTAMP_SET Added corresponding protobuf message types (Int32Set, Int64Set, etc.) Updated the Value message to include set fields Compiled with protobuf version 4.25.1 and mypy-protobuf 3.3.0 (matching CI environment) 2. Python SDK Updates value_type.py: Added set types to the ValueType enum and defined SetType union types.py: Created new Set class (ComplexFeastType) similar to Array Added set type mappings in VALUE_TYPES_TO_FEAST_TYPES Set types support all primitive types except Map type_map.py: Added PYTHON_SET_VALUE_TYPE_TO_PROTO_VALUE mapping Implemented Python → Proto conversions for sets with automatic duplicate removal Implemented Proto → Python conversions returning native Python sets Updated PROTO_VALUE_TO_VALUE_TYPE_MAP with set field mappings Added SET types to pandas type mapping (returns "object" type) Added set type inference in python_type_to_feast_value_type to properly handle Python sets passed in entity_rows Fixed None value handling in all set and list value conversions to prevent "TypeError: 'NoneType' object is not iterable" errors 3. Tests Added unit tests in test_types.py for Set type creation and round-trip conversion Added comprehensive tests in test_type_map.py for: Set conversion (string, int, float, bool sets) Duplicate removal from lists when converting to sets Empty and null set handling Multiple set value conversions Added test_python_transformation_with_set_types in test_on_demand_python_transformation.py for: Set types in On Demand Feature Views Automatic deduplication of set values Conversion from lists to sets Online feature retrieval with Set types 4. Documentation Updated docs/reference/type-system.md with: Set type table showing all supported set variants Usage examples demonstrating duplicate removal Updated complete feature view example with set fields Usage Example from feast.types import Set, String, Int32 from feast import Field # Define fields with set types schema = [ Field(name="visited_pages", dtype=Set(String)), Field(name="unique_categories", dtype=Set(Int32)), ] # Data with duplicates data = { "visited_pages": {"home", "products", "checkout", "products"}, # duplicate removed "unique_categories": {1, 2, 3, 2, 1} # duplicates removed } # Feast stores: {"home", "products", "checkout"} and {1, 2, 3} Benefits Automatic deduplication: Sets automatically remove duplicate values Type safety: Full type checking and validation Consistent API: Set types follow the same patterns as Array types Comprehensive support: All primitive types (except Map) have set variants Misc Protobuf changes are backward compatible. Existing LIST and MAP types are unaffected. Generated protobuf files use protobuf 4.25.1 and mypy-protobuf 3.3.0, matching the versions specified in CI requirements. All linting checks pass (make format-python and make lint-python) Unit tests pass for set type functionality Fixed multiple None value handling bugs in type conversion logic to ensure robustness Original prompt This section details on the original issue you should resolve Add set as feature type Is your feature request related to a problem? Please describe. Support for set features as feast feature types is missing. Describe the solution you'd like Add support for set type features in feast. Describe alternatives you've considered We have considered storing set features as list or maps but then burden is on calling client to covert back. please add this to the feast types and update the docs Comments on the Issue (you are @copilot in this section) Fixes #5886 ✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Open Graph Description: What this PR does / why we need it: This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values with automatic deduplication. Changes Made 1. Proto...

X Description: What this PR does / why we need it: This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values with automatic deduplication. Changes Made 1. Proto...

Opengraph URL: https://github.com/feast-dev/feast/pull/5888

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width