René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`ValueError: The truth value of an empty array is ambiguous` during materialization","articleBody":"## Expected Behavior\n\n`feast materialize` should complete successfully even when the source DataFrame\ncontains an **empty numpy array** (`np.array([])`) in a scalar feature column.\nThe empty array should be treated as a null / missing value and produce an empty\n`ProtoValue()`, consistent with how `None` and `np.nan` are already handled.\n\n## Current Behavior\n\n`feast materialize` crashes with:\n\n```\nValueError: The truth value of an empty array is ambiguous.\nUse `array.size \u003e 0` to check that an array is not empty.\n```\n\nFull stack trace:\n\n```\nTraceback (most recent call last):\n  File \"/opt/app-root/bin/feast\", line 10, in \u003cmodule\u003e\n    sys.exit(cli())\n             ^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/core.py\", line 1485, in __call__\n    return self.main(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/core.py\", line 1406, in main\n    rv = self.invoke(ctx)\n         ^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/core.py\", line 1873, in invoke\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/core.py\", line 1269, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/core.py\", line 824, in invoke\n    return callback(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/click/decorators.py\", line 34, in new_func\n    return f(get_current_context(), *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/cli/cli.py\", line 393, in materialize_command\n    store.materialize(\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/feature_store.py\", line 1816, in materialize\n    provider.materialize_single_feature_view(\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/infra/passthrough_provider.py\", line 456, in materialize_single_feature_view\n    raise e\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/infra/compute_engines/local/compute.py\", line 84, in _materialize_one\n    plan.execute(context)\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/infra/compute_engines/dag/plan.py\", line 51, in execute\n    output = node.execute(context)\n             ^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/infra/compute_engines/local/nodes.py\", line 274, in execute\n    rows_to_write = _convert_arrow_to_proto(\n                    ^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/utils.py\", line 281, in _convert_arrow_to_proto\n    return _convert_arrow_fv_to_proto(table, feature_view, join_keys)  # type: ignore[arg-type]\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/utils.py\", line 298, in _convert_arrow_fv_to_proto\n    proto_values_by_column = {\n                             ^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/utils.py\", line 299, in \u003cdictcomp\u003e\n    column: python_values_to_proto_values(\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/type_map.py\", line 840, in python_values_to_proto_values\n    proto_values = _python_value_to_proto_value(value_type, values)\n                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/app-root/lib64/python3.11/site-packages/feast/type_map.py\", line 772, in _python_value_to_proto_value\n    elif not pd.isnull(value):\nValueError: The truth value of an empty array is ambiguous. Use `array.size \u003e 0` to check that an array is not empty.\n```\n\nThe root cause is in [`sdk/python/feast/type_map.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/type_map.py),\nfunction `_convert_scalar_values_to_proto` (around line 968):\n\n```python\n# Generic scalar conversion\nout = []\nfor value in values:\n    if isinstance(value, ProtoValue):\n        out.append(value)\n    elif not pd.isnull(value):          # ← crashes here\n        out.append(ProtoValue(**{field_name: func(value)}))\n    else:\n        out.append(ProtoValue())\n```\n\n`pd.isnull()` is **vectorised**: when `value` is a numpy array (including an empty\none), it returns a numpy array of booleans instead of a scalar boolean. Applying\nPython's `not` to that array raises `ValueError`. The same pattern exists a few lines\nabove in the `ValueType.BOOL` path (`if not pd.isnull(value)`).\n\n## Steps to reproduce\n\n```python\nimport numpy as np\nfrom feast.type_map import python_values_to_proto_values\nfrom feast.value_type import ValueType\n\n# A scalar column where one row contains an empty array\npython_values_to_proto_values([np.array([]), 1.0, 2.0], ValueType.DOUBLE)\n# → ValueError: The truth value of an empty array is ambiguous\n```\n\n### Specifications\n\n- Version: `v0.60.0` (also reproducible on `main` as of 2026-04-10)\n- Platform: Linux (Python 3.11), macOS (Python 3.11)\n- Subsystem: `feast/type_map.py` – `_convert_scalar_values_to_proto`\n\n## Possible Solution\n\nThe fix belongs in [`sdk/python/feast/type_map.py`](https://github.com/feast-dev/feast/blob/85f5cef78/sdk/python/feast/type_map.py), specifically the **generic scalar conversion loop** at [line 963–975](https://github.com/feast-dev/feast/blob/85f5cef78/sdk/python/feast/type_map.py#L963-L975) (`elif not pd.isnull(value)`).\n\nBefore calling `not pd.isnull(value)`, check whether the value is array-like.\n`pd.isnull()` is vectorised and returns an `np.ndarray` for array inputs, so\ncalling `not` on it raises `ValueError`. The fix must handle three sub-cases:\n\n| Value | Expected outcome |\n|---|---|\n| Empty array (`size == 0`) | null → `ProtoValue()` |\n| Non-empty array containing any null | null → `ProtoValue()` |\n| Non-empty array with all valid data | convert → `ProtoValue(**{field_name: func(value)})` |\n| Plain scalar null | null → `ProtoValue()` |\n| Plain scalar non-null | convert → `ProtoValue(**{field_name: func(value)})` |\n\n```python\n# Generic scalar conversion\nout = []\nfor value in values:\n    if isinstance(value, ProtoValue):\n        out.append(value)\n    elif isinstance(value, np.ndarray) or (\n        hasattr(value, \"__len__\") and not isinstance(value, (str, bytes))\n    ):\n        # Array-like value in a scalar column\n        if hasattr(value, \"size\") and value.size == 0:\n            # Empty numpy array – treat as null\n            out.append(ProtoValue())\n        else:\n            is_null = pd.isnull(value)\n            if hasattr(is_null, \"any\"):\n                # pd.isnull returned an array; null if any element is null\n                out.append(ProtoValue() if is_null.any() else ProtoValue(**{field_name: func(value)}))\n            elif not is_null:\n                out.append(ProtoValue(**{field_name: func(value)}))\n            else:\n                out.append(ProtoValue())\n    elif not pd.isnull(value):\n        out.append(ProtoValue(**{field_name: func(value)}))\n    else:\n        out.append(ProtoValue())\nreturn out\n```\n","author":{"url":"https://github.com/alan-gauthier-jt","@type":"Person","name":"alan-gauthier-jt"},"datePublished":"2026-04-10T08:54:07.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/6255/feast/issues/6255"}