Title: Recursive Generic Type Support · Issue #13693 · python/mypy · GitHub
Open Graph Title: Recursive Generic Type Support · Issue #13693 · python/mypy
X Title: Recursive Generic Type Support · Issue #13693 · python/mypy
Description: Bug Report In: #731 #13297 (not released yet in v0.971) We now support recursive type hints, such as: JSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None] But in the example in #13297 (comment), which is generic and...
Open Graph Description: Bug Report In: #731 #13297 (not released yet in v0.971) We now support recursive type hints, such as: JSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None] But in the example i...
X Description: Bug Report In: #731 #13297 (not released yet in v0.971) We now support recursive type hints, such as: JSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None] But ...
Opengraph URL: https://github.com/python/mypy/issues/13693
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Recursive Generic Type Support","articleBody":"\u003c!--\r\n If you're new to mypy and you're not sure whether what you're experiencing is a mypy bug, please see the \"Question and Help\" form\r\n instead.\r\n Please also consider:\r\n\r\n - checking our common issues page: https://mypy.readthedocs.io/en/stable/common_issues.html\r\n - searching our issue tracker: https://github.com/python/mypy/issues to see if\r\n it's already been reported\r\n - asking on gitter chat: https://gitter.im/python/typing\r\n-->\r\n\r\n**Bug Report**\r\n\r\n\u003c!--\r\nNote: If the problem you are reporting is about a specific library function, then the typeshed tracker is better suited\r\nfor this report: https://github.com/python/typeshed/issues\r\n-->\r\n\r\nIn:\r\n\r\n- #731\r\n- #13297 (not released yet in v0.971)\r\n\r\nWe now support recursive type hints, such as:\r\n\r\n```python\r\nJSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None]\r\n```\r\n\r\nBut in the example in https://github.com/python/mypy/pull/13297#issue-1323540559, which is generic and has typevars:\r\n\r\n```python\r\nNested = Sequence[Union[T, Nested[T]]]\r\n\r\ndef flatten(seq: Nested[T]) -> List[T]:\r\n flat: List[T] = []\r\n for item in seq:\r\n if isinstance(item, Sequence):\r\n res.extend(flatten(item))\r\n else:\r\n res.append(item)\r\n return flat\r\n\r\nreveal_type(flatten([1, [2, [3]]])) # N: Revealed type is \"builtins.list[builtins.int]\"\r\n```\r\n\r\nThe code is Python syntax illegal that the name `Nested` in `Nested[T]` is referenced before assignment.\r\n\r\nMy own use case: define a generic pytree type:\r\n\r\n```python\r\n\"\"\"pytree_forwardref.py\"\"\"\r\n\r\nfrom typing import Any, Dict, Hashable, List, Optional, Sequence, Tuple, TypeVar, Union, Protocol\r\n\r\nimport torch\r\n\r\nT = TypeVar('T')\r\n\r\nChildren = Sequence[T]\r\n_AuxData = TypeVar('_AuxData', bound=Hashable)\r\nAuxData = Optional[_AuxData]\r\n\r\n\r\nclass CustomTreeNode(Protocol[T]):\r\n \"\"\"The abstract base class for custom pytree nodes.\"\"\"\r\n\r\n def tree_flatten(self) -> Tuple[Children[T], AuxData]:\r\n \"\"\"Flattens the custom pytree node into children and auxiliary data.\"\"\"\r\n\r\n @classmethod\r\n def tree_unflatten(cls, aux_data: AuxData, children: Children[T]) -> 'CustomTreeNode[T]':\r\n \"\"\"Unflattens the children and auxiliary data into the custom pytree node.\"\"\"\r\n\r\n\r\nPyTree = Union[\r\n T,\r\n Tuple['PyTree[T]', ...], # Tuple, NamedTuple\r\n List['PyTree[T]'],\r\n Dict[Any, 'PyTree[T]'], # Dict, OrderedDict, DefaultDict\r\n CustomTreeNode['PyTree[T]'],\r\n]\r\n\r\n\r\nTensorTree = PyTree[torch.Tensor]\r\nprint(TensorTree)\r\n```\r\n\r\nI got `NameError` is use `PyTree[T]`. Or the typevar `T` is not expended when using `ForwardRef` (`'PyTree[T]'`):\r\n\r\n```bash\r\n$ python3 pytree_forwardref.py \r\ntyping.Union[torch.Tensor, typing.Tuple[ForwardRef('PyTree[T]'), ...], typing.List[ForwardRef('PyTree[T]')], typing.Dict[typing.Any, ForwardRef('PyTree[T]')], __main__.CustomTreeNode[ForwardRef('PyTree[T]')]]\r\n```\r\n\r\n**To Reproduce**\r\n\r\n1. `mypy` does not report `NameError` for recursive type:\r\n\r\n 1. Install `pylint` and the dev version of `mypy`:\r\n\r\n ```bash\r\n pip3 install pylint\r\n pip3 install git+https://github.com/python/mypy.git\r\n ```\r\n\r\n 2. Create a file with content:\r\n\r\n ```python\r\n \"\"\"nested.py\"\"\"\r\n\r\n from typing import TypeVar, Sequence, Union\r\n\r\n T = TypeVar('T')\r\n\r\n Nested = Sequence[Union[T, Nested[T]]]\r\n\r\n NestedInt = Nested[int]\r\n print(NestedInt)\r\n ```\r\n\r\n 3. Run `mypy`:\r\n\r\n ```console\r\n $ mypy --enable-recursive-aliases nested.py\r\n Success: no issues found in 1 source file\r\n ```\r\n\r\n 4. Run `pylint`:\r\n\r\n ```console\r\n $ pylint nested.py \r\n ************* Module nested\r\n nested.py:7:27: E0601: Using variable 'Nested' before assignment (used-before-assignment)\r\n\r\n ------------------------------------------------------------------\r\n Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)\r\n\r\n $ python3 nested.py\r\n Traceback (most recent call last):\r\n File \"nested.py\", line 7, in \u003cmodule>\r\n Nested = Sequence[Union[T, Nested[T]]]\r\n NameError: name 'Nested' is not defined\r\n ```\r\n\r\n2. `ForwardRef` does not expand typevars:\r\n\r\n 5. Change to use `ForwardRef`:\r\n\r\n ```python\r\n \"\"\"nested_forwardref.py\"\"\"\r\n\r\n from typing import TypeVar, Sequence, Union\r\n\r\n T = TypeVar('T')\r\n\r\n Nested = Sequence[Union[T, 'Nested[T]']]\r\n\r\n NestedInt = Nested[int]\r\n print(NestedInt)\r\n ```\r\n\r\n 6. Run `mypy`, `pylint` and execute:\r\n\r\n ```console\r\n $ mypy --enable-recursive-aliases nested_forwardref.py\r\n Success: no issues found in 1 source file\r\n\r\n $ pylint nested_forwardref.py\r\n\r\n ------------------------------------\r\n Your code has been rated at 10.00/10\r\n\r\n $ python3 python3 nested_forwardref.py\r\n typing.Sequence[typing.Union[int, ForwardRef('Nested[T]')]]\r\n ```\r\n\r\n**Expected Behavior**\r\n\r\n\u003c!--\r\n How did you expect your project to behave?\r\n It’s fine if you’re not sure your understanding is correct.\r\n Write down what you thought would happen. If you just expected no errors, you can delete this section.\r\n-->\r\n\r\n1. Raise errors for generic recursive types.\r\n\r\nor\r\n\r\n2. support generic `ForwardRef` (preferred, but maybe need support by Python)\r\n\r\n**Actual Behavior**\r\n\r\n\u003c!--\r\n Did something go wrong?\r\n Is something broken, or not behaving as you expected?\r\n-->\r\n\r\nNo error raise when using the variable name without a `ForwardRef`. \r\n\r\nRef:\r\n\r\n- [Python Discussion: Generic `typing.ForwardRef` to support generic recursive types](https://discuss.python.org/t/generic-typing-forwardref-to-support-generic-recursive-types/20168).\r\n\r\n**Your Environment**\r\n\r\n\u003c!-- Include as many relevant details about the environment you experienced the bug in -->\r\n\r\n- Mypy version used: master @ 1d4395f14a0b2a923dd24e881887ff9360ec50fa\r\n- Mypy command-line flags: `--enable-recursive-aliases`\r\n- Mypy configuration options from `mypy.ini` (and other config files): None\r\n- Python version used: 3.8.12\r\n- Operating system and version: Ubuntu 20.04 LTS\r\n\r\n\u003c!--\r\nYou can freely edit this text, please remove all the lines\r\nyou believe are unnecessary.\r\n-->\r\n","author":{"url":"https://github.com/XuehaiPan","@type":"Person","name":"XuehaiPan"},"datePublished":"2022-09-20T06:07:09Z","url":"https://github.com/python/mypy/issues/13693"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:8b26d498-52f7-76df-b4a8-368ad709ac7f |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | C05C:182C25:4D5BAE:682665:6B002F62 |
| html-safe-nonce | 7c8113a7699c29c04f84fa33aa482f7188b2059098f41126d1bfc024b653e101 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDMDVDOjE4MkMyNTo0RDVCQUU6NjgyNjY1OjZCMDAyRjYyIiwidmlzaXRvcl9pZCI6IjE0ODgwODY0NjkxMjA2MzQ3MjIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | aa15ca0b7fc0274813819427f9f2b909a998cb2044872f60692da48cb8474496 |
| hovercard-subject-tag | issue:1378867868 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/13693/issue_layout |
| twitter:image | https://opengraph.githubassets.com/57a00f7012a5c7d2387d395139ce81069d80156bfdffaf945888daf0d160c172/python/mypy/issues/13693 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/57a00f7012a5c7d2387d395139ce81069d80156bfdffaf945888daf0d160c172/python/mypy/issues/13693 |
| og:image:alt | Bug Report In: #731 #13297 (not released yet in v0.971) We now support recursive type hints, such as: JSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None] But in the example i... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | XuehaiPan |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7d0829bbde9e48b2486cb5dd6d147896eb01d32134627091f818958a935ef3f0 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/mypy git https://github.com/python/mypy.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 7053637 |
| octolytics-dimension-repository_nwo | python/mypy |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 7053637 |
| octolytics-dimension-repository_network_root_nwo | python/mypy |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | f3f3a43ca095cdea3abbb394112e971f9fdb94fa |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width