René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Failed on fetching historical feature values with get_historical_features python API","articleBody":"## Expected Behavior \r\nget_historical_features should return a joined result with feature view or views as expected but I am unable to see the results.\r\n\r\n## Current Behavior\r\nget_historical_features throws out an error \r\n\"ProgrammingError: (pyodbc.ProgrammingError) ('42000', \"[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]String or binary data would be truncated in table 'feature-store.dbo.feast_entity_df_3dada85cfda44050b3d22b896579ba2f', column 'customer_id'. Truncated value: 'D'. (2628) (SQLExecDirectW)\")\"\r\n\r\n## Steps to reproduce\r\n1. Create an entity with \"customer_id\" as a string type.\r\n2. Create a feature view that links to the above entity.\r\n3. `Feast apply` in a shell script.\r\n4. Execute `get_historical_features`.\r\n\r\n### Specifications\r\n- Version: Python==3.8, feast==0.28.0\r\n- Platform: Azure\r\n- Subsystem: Both registry and offline store are Azure MSSQL and the online store is Azure cache for Redis\r\n\r\n## Possible Solution\r\n1. I looked into the source code and if I understand the backend correctly, when a user triggers \"get_historical_features\", at a very high level, Feast executes three steps: 1) create a temp table 2) insert row(s) based on what was submitted in one's entity_df parameter 3) ready for point-in-time correctness join with feature view data.\r\n2. Root cause was found post the temp table formation from step 1. For a string type of data or entity in my case, Feast transfers the data type into \"varchar\" without inferring its length. Therefore, from the DDL pov, SQL server would interpret it as varchar(1) and this is why steps 2 and 3 both failed afterward.\r\n3. Proposed resolutions: In the file \"type_map.py\", function \"pa_to_mssql_type\" line 601 is the root cause. I would propose two solutions, either make a large number within varchar like varchar(max) or perform some dynamic length checks upfront. I kinda prefer the former as performing some additional checks would take more time and further delay the following operation.\r\n","author":{"url":"https://github.com/tsenhungwu","@type":"Person","name":"tsenhungwu"},"datePublished":"2023-02-15T14:50:35.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/3495/feast/issues/3495"}