René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Redis Cluster implementation is broken","articleBody":"## Expected Behavior \r\n\r\n## Current Behavior\r\n\r\nRecently a Redis cluster bug was fixed in #2311 and tests were added in #2317. However, these tests revealed other issues with our Redis cluster implementation.\r\n\r\nOne issue is the way we currently scan for rows to delete:\r\n```\r\nclient.scan_iter(b\"\".join([prefix, b\"*\", config.project.encode(\"utf8\")]))\r\n```\r\nFirst, `scan_iter` only hits a single master node by default, so this line should instead be something like\r\n```\r\nclient.scan_iter(b\"\".join([prefix, b\"*\", config.project.encode(\"utf8\")]), target_nodes=RedisCluster.PRIMARIES)\r\n```\r\nHowever, this is still broken. After some digging, it seems that `scan_iter` relies on `scan`, which is itself broken - I believe this is an issue with `redis-py`. I filed an issue [here](https://github.com/redis/redis-py/issues/2003).\r\n\r\nAnother issue seems to be with our current pipelining logic. We have `with client.pipeline(transaction=False) as pipe:`, but this leads to the following error:\r\n```\r\nredis.exceptions.ResponseError: Command # 1 (HMGET drivertest_universal_cli_26a064e4 _ts:driver_locations) of pipeline caused error: MOVED 12671 127.0.0.1:6003\r\n```\r\n\r\n## Steps to reproduce\r\n\r\n### Specifications\r\n\r\n- Version:\r\n- Platform:\r\n- Subsystem:\r\n\r\n## Possible Solution\r\n\r\nThis issue tracks a resolution to the existing Redis cluster bugs. After they've been fixed, we should reenable Redis cluster tests, which were disabled in #2327 to unblock other PRs.\r\n","author":{"url":"https://github.com/felixwang9817","@type":"Person","name":"felixwang9817"},"datePublished":"2022-02-20T01:01:47.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/2328/feast/issues/2328"}