René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"User encoding is skipped for elements iterated from non-generic .NET containers/enumerators","articleBody":"### Environment\n\n-   Pythonnet version: 3.0.5\n-   Python version: 3.12.9\n-   Operating System: Linux (6.12.63-1-MANJARO # 1 SMP PREEMPT_DYNAMIC Thu, 18 Dec 2025 13:46:28 +0000 x86_64 GNU/Linux)\n-   .NET Runtime: coreclr 10.0.0\n\n### Details\n\nAfter creating and registering an encoder (subclass of `IPyObjectEncoder`) to convert CLR objects to Python equivalents, iterating over non-generic containers yields CLR proxy objects instead of the converted Python object. However, doing the same for generic containers yields the converted objects as expected. In either case, accessing elements using the indexer produces a converted object.\n\nHere is a minimal test case demonstrating the issue:\n\n```python\nimport pytest\n\nimport System\nfrom Python.Test import CodecResetter\nfrom Python.Runtime import IPyObjectEncoder, PyObjectConversions\n\n@pytest.mark.parametrize('container_type', ['generic', 'non-generic'])\ndef test_iterator_element_conversion(container_type):\n    \"\"\"Test iterator element conversion from Python.\"\"\"\n    from Python.Test import Spam\n\n    try:\n        from Python.Test import IteratorElementEncoder\n    except ImportError:\n        class IteratorElementEncoder(IPyObjectEncoder):\n            __namespace__ = \"Python.Test\"\n            def CanEncode(self, clr_type):\n                return clr_type.Name == \"Spam\" and clr_type.Namespace == \"Python.Test\"\n            def TryEncode(self, clr_object):\n                return clr_object.GetValue()\n\n    spam_encoder = IteratorElementEncoder()\n    PyObjectConversions.RegisterEncoder(spam_encoder)\n\n    values = [\"first\", \"second\", \"third\"]\n    if container_type == 'generic':\n        container = System.Collections.Generic.List[Spam]()\n        for value in values:\n            container.Add(Spam(value))\n    else:\n        container = System.Array[Spam](Spam(v) for v in values)\n\n    assert type(container[0]) is str\n    assert next(iter(container.GetEnumerator())) == container[0]\n    assert list(container.GetEnumerator()) == values\n\n    CodecResetter.Reset()\n```\n\nAnd here are the results of running the test:\n\n```\n================================================================= FAILURES =================================================================\n______________________________________________ test_iterator_element_conversion[non-generic] _______________________________________________\n\n...\n        assert type(container[0]) is str\n\u003e       assert next(iter(container.GetEnumerator())) == container[0]\nE       AssertionError: assert \u003cPython.Test.Spam object at 0x7ee403c447c0\u003e == 'first'\nE        +  where \u003cPython.Test.Spam object at 0x7ee403c447c0\u003e = next(\u003cCLR.Iterator object at 0x7ee403c449c0\u003e)\nE        +    where \u003cCLR.Iterator object at 0x7ee403c449c0\u003e = iter(\u003cSystem.Collections.IEnumerator object at 0x7ee403c46600\u003e)\nE        +      where \u003cSystem.Collections.IEnumerator object at 0x7ee403c46600\u003e = \u003cbound method 'GetEnumerator'\u003e()\nE        +        where \u003cbound method 'GetEnumerator'\u003e = \u003cPython.Test.Spam[] object at 0x7f25379bb540\u003e.GetEnumerator\n\ntests/test_conversion.py:786: AssertionError\n========================================================= short test summary info ==========================================================\nFAILED tests/test_conversion.py::test_iterator_element_conversion[non-generic] - AssertionError: assert \u003cPython.Test.Spam object at 0x7ee403c447c0\u003e == 'first'\n======================================================= 1 failed, 1 passed =======================================================\n\n```\n\nThe first assert passes for each test because the indexers of both generic and non-generic containers properly convert the object. The final two asserts both fail for the non-generic version because conversion is not performed, and the `Spam` proxies are returned instead of being converted to strings.\n\n### Implementation Details\n\nThe issue stems from `src/runtime/Types/ClassBase.cs:ClassBase.tp_iter_impl()`, which determines the generic type of a container, defaulting to `System.Object` for non-generic containers. The element type is then passed to `src/runtime/Types/Iterator.cs:Iterator` where it is used in calls to `Converter.ToPython()`. The problem is that `System.Object` is passed as the element type even though the type is actually something different that could potentially be converted by a user encoder. But `System.Object` types are not passed on to user converters and are returned as CLR proxies instead.\n\nThe solution is to check if the element type is `System.Object`, and query and use the actual value type instead.","author":{"url":"https://github.com/brandon-avantus","@type":"Person","name":"brandon-avantus"},"datePublished":"2026-01-14T18:24:37.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/2679/pythonnet/issues/2679"}