René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"PEP 649 and `ForwardRef` caching","articleBody":"# Bug report\r\n\r\n### Bug description:\r\n\r\nAny typing construct using `typing._GenericAlias` (and not `types.GenericAlias`) will convert the arguments to `ForwardRef` instances at runtime:\r\n\r\n```python\r\nfrom typing import List\r\n\r\nclass A[T]:\r\n    pass\r\n\r\nList['F']\r\n#\u003e List[ForwardRef('F')]\r\nA['F']\r\n#\u003e A[ForwardRef('F')]\r\n```\r\n\r\nBecause `_GenericAlias.__getitem__` calls are cached with the `_tp_cache()` decorator, we end up with the same `ForwardRef` instance used:\r\n\r\n```python\r\nalias1 = List['F']\r\n#\u003e List[ForwardRef('F')]\r\nalias2 = List['F']\r\n#\u003e List[ForwardRef('F')]\r\n\r\nalias1.__args__[0] is alias2.__args__[0]\r\n#\u003e True\r\n```\r\n\r\nAnd this becomes an issue as `ForwardRef._evaluate()` calls are also cached per instance. Consider the following setup:\r\n\r\n```python\r\n# mod1.py\r\n\r\ndef func(a: List['Forward']): pass\r\n\r\nForward = int\r\n\r\ntyping.get_type_hints(func)\r\n# {'a': List[int]}\r\n\r\n# mod2.py\r\n\r\ndef func(a: List['Forward']): pass\r\n\r\nForward = str\r\n\r\ntyping.get_type_hints(func)\r\n# {'a': List[int]}\r\n```\r\n\r\nNote that this is already an issue on Python \u003c 3.14. However, the impact is somewhat limited as afaik this only happens with functions. The reason is that there is a [really old cache invalidation mechanism seemingly introduced to avoid a leak issue](https://github.com/python/typing/pull/328) in `ForwardRef._evaluate`. This logic would force the evaluation of the forward reference (even if `_evaluate` was already called) if the provided `localns` is different from the `globalns` (on `L920` the evaluation logic goes on):\r\n\r\nhttps://github.com/python/cpython/blob/dae5b16bb3b51702d3506ef21550b220359ff971/Lib/typing.py#L916-L920\r\n\r\nUsing `typing.get_type_hints`, this condition is false when getting annotations of functions, because functions don't have locals and locals are set to globals if this is the case:\r\n\r\nhttps://github.com/python/cpython/blob/8f93dd8a8f237b277abad20d566df90c5cbd7f1e/Lib/typing.py#L2441-L2442\r\n\r\n---\r\n\r\nHowever, the implementation of PEP 649 removed this check and the cached evaluated value is unconditionally used:\r\n\r\nhttps://github.com/python/cpython/blob/8f93dd8a8f237b277abad20d566df90c5cbd7f1e/Lib/annotationlib.py#L95-L101\r\n\r\nWhile the described bug above is pretty uncommon as it only occurs with functions, it also happens in classes with 3.14:\r\n\r\n```python\r\n# mod1.py\r\n\r\nclass A:\r\n    'a': List['Forward']\r\n\r\nForward = int\r\n\r\ntyping.get_type_hints(A)\r\n# {'a': List[int]}\r\n\r\n# mod2.py\r\n\r\nclass A:\r\n    'a': List['Forward']\r\n\r\nForward = str\r\n\r\ntyping.get_type_hints(func)\r\n# {'a': List[int]}\r\n```\r\n\r\nAnd I believe this is going to cause some issues with existing code bases, especially the ones using runtime typing libraries. In Pydantic, we already had issues like this one, as we recently changed our global/local namespace logic (and as such, we had a report of the above issue with classes: https://github.com/cloudflare/cloudflare-python/discussions/116#discussioncomment-11372711).\r\n\r\nWhile we might argue that string annotations are no longer needed in 3.14, it is still relevant for libraries which need to keep support for older Python versions.\r\n\r\nOne possible solution would be to have `_tp_cache()` skip string arguments, so that we don't end up reusing the same `ForwardRef` instances. Not sure how big the impact will be. \r\n\r\n### CPython versions tested on:\r\n\r\n3.14, CPython main branch\r\n\r\n### Operating systems tested on:\r\n\r\n_No response_","author":{"url":"https://github.com/Viicos","@type":"Person","name":"Viicos"},"datePublished":"2025-01-07T18:58:56.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/128593/cpython/issues/128593"}