René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add Kubernetes Deployment Options to the Bytewax Materialization Engine","articleBody":"**Is your feature request related to a problem? Please describe.**\r\nThe Bytewax materialization engine needs to support configuration options for more advanced Kubernetes deployments (EKS, GCP, etc) to make it usable at scale.\r\n\r\n**Describe the solution you'd like**\r\nThe main configuration options that are needed for the Bytewax materialization job are:\r\n\r\n* setting explicit resource requests and limits (rather than relying on platform defaults which may not be enough)\r\n* supporting service accounts and IAM roles\r\n* specifying an image pull secret to support pulling Docker images from Dockerhub, Artifactory, etc\r\n\r\n**Describe alternatives you've considered**\r\nThe Kubernetes job that runs is dynamically generated by the bytewax code. Existing configuration options are insufficient.\r\n\r\n**Additional context**\r\nI'd really like to test the Bytewax materialization engine on our instance of EKS. In its current implementation, it's operationally too simple and not usable outside of minikube.\r\n","author":{"url":"https://github.com/adamschmidt","@type":"Person","name":"adamschmidt"},"datePublished":"2023-03-03T00:27:57.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/3517/feast/issues/3517"}