René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bytewax materialization engine fails when loading feature_store.yaml","articleBody":"## Expected Behavior \r\n\r\nLoading the `feature_store.yaml` file from within a Bytewax pod should work.\r\n\r\n## Current Behavior\r\n\r\n`yaml.safe_load()` raises an error while trying to reconstruct the object below:\r\n\r\n- pathlib.PosixPath\r\n\r\nThe error occurs while running materialization using Bytewax at the point where the `feature_store.yaml` is loaded. The code where this happens is in [sdk/python/feast/infra/materialization/contrib/bytewax/dataflow.py](https://github.com/feast-dev/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/feast/infra/materialization/contrib/bytewax/dataflow.py#L14). Below is an excerpt:\r\n\r\n```python\r\n# ...\r\n with open(\"/var/feast/feature_store.yaml\") as f:\r\n        feast_config = yaml.safe_load(f) # \u003c---- yaml.safe_load() fails\r\n# ...\r\n```\r\n\r\nThe exact message is as below:\r\n\r\n```python\r\nDefaulted container \"process\" out of: process, init-hostfile (init)\r\nFeast is an open source project that collects anonymized error reporting and usage statistics. To opt out or learn more see https://docs.feast.dev/reference/usage\r\nTraceback (most recent call last):\r\n  File \"/bytewax/dataflow.py\", line 15, in \u003cmodule\u003e\r\n    feast_config = yaml.safe_load(f)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/__init__.py\", line 125, in safe_load\r\n    return load(stream, SafeLoader)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/__init__.py\", line 81, in load\r\n    return loader.get_single_data()\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 51, in get_single_data\r\n    return self.construct_document(node)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 60, in construct_document\r\n    for dummy in generator:\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 413, in construct_yaml_map\r\n    value = self.construct_mapping(node)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 218, in construct_mapping\r\n    return super().construct_mapping(node, deep=deep)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 143, in construct_mapping\r\n    value = self.construct_object(value_node, deep=deep)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 100, in construct_object\r\n    data = constructor(self, node)\r\n  File \"/usr/local/lib/python3.9/site-packages/yaml/constructor.py\", line 427, in construct_undefined\r\n    raise ConstructorError(None, None,\r\nyaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:pathlib.PosixPath'\r\n  in \"/var/feast/feature_store.yaml\", line 119, column 12\r\n```\r\n\r\nInterestingly, method [_create_configuration_map()](https://github.com/feast-dev/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/feast/infra/materialization/contrib/bytewax/bytewax_materialization_engine.py#L312) of class `BytewaxMaterializationEngine` uses `yaml.dump()` instead of `yaml.safe_dump()` to write the config in the first place:\r\n\r\n```python\r\n    # ...\r\n    def _create_configuration_map(self, job_id, paths, feature_view, namespace):\r\n        \"\"\"Create a Kubernetes configmap for this job\"\"\"\r\n\r\n        feature_store_configuration = yaml.dump(self.repo_config.dict())\r\n    # ...\r\n```\r\n\r\nWhen I tried to replace `yaml.dump` by `yaml.safe_dump()` I got the following error:\r\n\r\n```python\r\nyaml.representer.RepresenterError: ('cannot represent an object', \u003cRedisType.redis: 'redis'\u003e)\r\n```\r\n\r\nIt appears that `yaml.SafeDumper` and `yaml.SafeLoader` cannot find  the appropriate representers and/or constructors for `RedisType.redis` and `path.PosixPath`. Perhaps those objects do not have corresponding `to_yaml()` and `from_yaml()` methods.\r\n\r\n## Steps to reproduce\r\n\r\nRun the materialization:\r\n\r\n```bash \r\nfeast materialize  --views \"EXAMPLE_FEATURE_VIEW\" '2023-10-30T00:00:00' '2023-10-30T23:59:59'\r\n```\r\n\r\nGive it some time and check the pods:\r\n\r\n```bash\r\nkubectl get pods -n bytewax\r\nNAME                                                    READY   STATUS   RESTARTS   AGE\r\ndataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-0-9kxgt   0/1     Error    0          25s\r\ndataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-1-d8n4r   0/1     Error    0          25s\r\ndataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-2-wmmsd   0/1     Error    0          25s\r\ndataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-3-c8gn7   0/1     Error    0          25s\r\ndataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-4-hgfbn   0/1     Error    0          25s\r\n```\r\n\r\nThen upon inspecting the logs, I see the error from above:\r\n\r\n```bash\r\nkubectl logs -n bytewax dataflow-4f3a7567-7cc9-4188-9fb1-cfc614451c35-4-hgfbn\r\n```\r\n\r\n### Specifications\r\n\r\n- Version: feast==0.35.0, [pyyaml==6.0.1](https://github.com/feast-dev/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/requirements/py3.10-ci-requirements.txt#L714), [bytewax==0.15.1](https://github.com/feast-dev/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/requirements/py3.10-ci-requirements.txt#L75)\r\n- Platform:\r\n    - local: MacOS\r\n    - bytewax image: a custom build based on the Dockerfile from the feast-dev repo [sdk/python/feast/infra/materialization/contrib/bytewax/Dockerfile](https://github.com/feast-dev/feast/blob/9237361e8d87b8e5dc02dffa74905eab2380fbd7/sdk/python/feast/infra/materialization/contrib/bytewax/Dockerfile#L28) where feast is installed with support for AWS, Bytewax, Redis (as the online store), Postgres (as the SQL registry)\r\n\r\n## Possible Solution\r\n\r\nI was able to make it work by modifying [sdk/python/feast/infra/materialization/contrib/bytewax/dataflow.py](https://github.com/feast-dev/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/feast/infra/materialization/contrib/bytewax/dataflow.py#L14) to use [yaml.load()](https://github.com/yaml/pyyaml/issues/266#issuecomment-472899408) instead of `yaml.safe_load()` and rebuilding the [Bytewax docker image](https://github.com/feast-dev/feast/blob/9237361e8d87b8e5dc02dffa74905eab2380fbd7/sdk/python/feast/infra/materialization/contrib/bytewax/Dockerfile#L28):\r\n\r\n```python\r\n    with open(\"/var/feast/feature_store.yaml\") as f:\r\n        #feast_config = yaml.safe_load(f)\r\n        feast_config = yaml.load(f, Loader=yaml.Loader)\r\n\r\n        with open(\"/var/feast/bytewax_materialization_config.yaml\") as b:\r\n            # I did not test if yaml.safe_load() works for the bytewax config, but just went ahead and replaced it too \r\n            #bytewax_config = yaml.safe_load(b)\r\n            bytewax_config = yaml.load(b, Loader=yaml.Loader)\r\n```","author":{"url":"https://github.com/gterziysky","@type":"Person","name":"gterziysky"},"datePublished":"2024-01-18T17:20:00.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/3893/feast/issues/3893"}