René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add DynamoDB in-place list update support for array-based features","articleBody":"**Is your feature request related to a problem? Please describe.**\n\nWhen working with array-based features that represent sequences of past events per entity (e.g., last N transactions, recent interactions, rolling window of measurements), the current `write_to_online_store` implementation requires completely replacing the entire feature set for each entity. This is inefficient for use cases where you want to append new events to existing arrays, as it requires:\n\n1. Reading the current array via `get_online_features`\n2. Modifying it in application code (append, prepend, or maintain sliding window)\n3. Writing the entire updated array back via `write_to_online_store`\n\nFor DynamoDB specifically, this doesn't leverage the native `UpdateItem` operation with `list_append` expressions, which would be more efficient and atomic.\n\n**Describe the solution you'd like**\n\nAdd optional partial update support to the DynamoDB online store that allows in-place list operations. This could be implemented as:\n\n1. **New method in DynamoDB online store**: `update_online_store` or extend `write_to_online_store` with an `update_mode` parameter\n2. **Support for update expressions**: Allow specifying DynamoDB update expressions like:\n   - `list_append(feature_array, :new_values)` - append to end\n   - `list_append(:new_values, feature_array)` - prepend to beginning  \n   - Custom expressions for more complex operations\n\n**Example API:**\n\n```python\n# Option 1: New method\nstore.update_online_store(\n    feature_view_name=\"user_events\",\n    entity_keys=[{\"user_id\": \"123\"}],\n    updates={\n        \"recent_transactions\": {\n            \"operation\": \"list_append\",\n            \"values\": [new_transaction_dict]\n        }\n    }\n)\n\n# Option 2: Extended write_to_online_store\nstore.write_to_online_store(\n    feature_view_name=\"user_events\",\n    df=new_events_df,\n    write_mode=\"append\"  # or \"replace\" (default), \"prepend\"\n)\n```\n\n**Implementation Approach:**\n\nSince this is DynamoDB-specific functionality, it could be:\n- Added as an optional method in the `DynamoDBOnlineStore` class\n- Kept separate from the core online store interface to avoid breaking changes\n- Potentially exposed through a DynamoDB-specific utility or extension\n\n**Key changes needed:**\n\n1. Modify `sdk/python/feast/infra/online_stores/dynamodb.py` to add update operation support\n2. Use boto3's `update_item` with `UpdateExpression` instead of `put_item`\n3. Handle the serialization/deserialization of list values appropriately\n4. Maintain backward compatibility with existing `write_to_online_store` behavior\n\n**Describe alternatives you've considered**\n\n1. **Read-Modify-Write pattern** (current workaround): Inefficient and not atomic\n2. **Maintain arrays upstream**: Requires duplicating windowing logic in batch/stream processing\n3. **Use OnDemandFeatureView**: Doesn't solve the storage efficiency problem\n4. **Custom online store implementation**: Requires maintaining a fork\n\n**Additional context**\n\nThis feature would be particularly valuable for:\n- Time-series features with sliding windows\n- User interaction histories\n- Event sequences for sequential models\n- Any use case requiring efficient array/list manipulation at the storage layer\n\nRelated Slack discussion: https://feastopensource.slack.com/archives/C01M2GYP0UC/p1761052622312959\n\n**Technical References:**\n- Current DynamoDB implementation: `sdk/python/feast/infra/online_stores/dynamodb.py:531` (uses `batch_writer` with full replacement)\n- Async write path: `sdk/python/feast/infra/utils/aws_utils.py:1125` (uses `PutRequest`)\n- DynamoDB UpdateItem documentation: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html\n","author":{"url":"https://github.com/devin-ai-integration[bot]","@type":"Person","name":"devin-ai-integration[bot]"},"datePublished":"2025-10-21T17:58:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/5687/feast/issues/5687"}