René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"FeatureStore.get_online_features is broken with int64 vals","articleBody":"## Expected Behavior \r\nI should be able to use an int64 value as the join key value for an entity, when the value type is `value_type=ValueType.INT64`.\r\n\r\n## Current Behavior\r\nOnly values -2147483648 \u003c= number \u003c= 2147483647 can be used as the value. This is equivalent to -2^31 \u003c= number \u003c= 2^31 - 1\r\n\r\n\r\n## Steps to reproduce\r\nSpecify a value greater than 2^31 - 1 as a the join key value when looking up values from the feature store.\r\n\r\n```\r\n\u003e\u003e\u003e fs.get_online_features(features=['driver_hourly_stats:conv_rate'],  entity_rows=[{'driver_id': 2147483648}]).to_df()\r\nTraceback (most recent call last):\r\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/usage.py\", line 202, in exception_logging_wrapper\r\n    result = func(*args, **kwargs)\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/feature_store.py\", line 897, in get_online_features\r\n    union_of_entity_keys,\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/feature_store.py\", line 924, in _populate_result_rows_from_feature_view\r\n    requested_features=requested_features,\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/infra/passthrough_provider.py\", line 81, in online_read\r\n    result = self.online_store.online_read(config, table, entity_keys)\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/infra/online_stores/redis.py\", line 190, in online_read\r\n    redis_key_bin = _redis_key(project, entity_key)\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/infra/online_stores/helpers.py\", line 40, in _redis_key\r\n    key: List[bytes] = [serialize_entity_key(entity_key), project.encode(\"utf-8\")]\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/infra/key_encoding_utils.py\", line 41, in serialize_entity_key\r\n    val_bytes, value_type = _serialize_val(v.WhichOneof(\"val\"), v)\r\n  File \"/Users/achal/tecton/feast/sdk/python/feast/infra/key_encoding_utils.py\", line 17, in _serialize_val\r\n    return struct.pack(\"\u003cl\", v.int64_val), ValueType.INT64\r\nstruct.error: 'l' format requires -2147483648 \u003c= number \u003c= 2147483647\r\n```\r\n\r\n\r\n## Possible Solution\r\n\r\nThe trivial fix is to make [this line](https://github.com/feast-dev/feast/blob/dcae1606f53028ce5413567fb8b66f92cfef0f8e/sdk/python/feast/infra/key_encoding_utils.py#L17) use `\"q\"` as the format string, which refers to a `signed long long` value which is 8 bytes in size.\r\n\r\nChanging this behavior is a backwards incompatible change, since it would change the bytes encoded for an entity key which takes on int64 values. This means that materialized records oh that value type would would be unreadable.\r\n\r\nThe \"correct\" way of fixing this would require us to version the encoding scheme in the feature store.","author":{"url":"https://github.com/achals","@type":"Person","name":"achals"},"datePublished":"2021-09-25T02:50:28.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":9},"url":"https://github.com/1908/feast/issues/1908"}