Title: Unexpected type binding when wrapping functions · Issue #12919 · python/mypy · GitHub
Open Graph Title: Unexpected type binding when wrapping functions · Issue #12919 · python/mypy
X Title: Unexpected type binding when wrapping functions · Issue #12919 · python/mypy
Description: Bug Report I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where mypy and pyre disagree, so it might be a bu...
Open Graph Description: Bug Report I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where myp...
X Description: Bug Report I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where myp...
Opengraph URL: https://github.com/python/mypy/issues/12919
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Unexpected type binding when wrapping functions","articleBody":"\u003c!--\r\n If you're new to mypy and you're not sure whether what you're experiencing is a mypy bug, please see the \"Question and Help\" form\r\n instead.\r\n Please also consider:\r\n\r\n - checking our common issues page: https://mypy.readthedocs.io/en/stable/common_issues.html\r\n - searching our issue tracker: https://github.com/python/mypy/issues to see if\r\n it's already been reported\r\n - asking on gitter chat: https://gitter.im/python/typing\r\n-->\r\n\r\n**Bug Report**\r\n\r\nI admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where mypy and pyre disagree, so it might be a bug.\r\n\r\nI am writing code that lifts functions `T -> R` to `Optional[T] -> Optional[R]` in various ways, and when I write wrapper functions, type variables get bound in ways I didn't expect. It is particularly bad when I add protocols for various types on top of it, but I can create the unexpected behaviour without it. Basically, the issue is that if I write a function that changes a `Callable[[T], R]` into a `Callable[[Optional[T]], Optional[R]]`, and then apply such a function to another generic function `Callable[[T],R]` the type variables are no longer generic, and I cannot apply the wrapped function. This happens both when I work with wrapper functions or when I implement the wrapper as a `Generic` class.\r\n\r\n**To Reproduce**\r\n\r\nIf I write a wrapper with a function, it might look like this:\r\n\r\n```python\r\nfrom typing import (\r\n TypeVar, \r\n Protocol, Generic, \r\n Optional as Opt\r\n)\r\n\r\n\r\n_T = TypeVar('_T')\r\n\r\nclass _F(Protocol[_T]):\r\n def __call__(self, __x: _T) -> _T:\r\n ...\r\n\r\n# Lifting with a function...\r\ndef lift(f: _F[_T]) -> _F[Opt[_T]]:\r\n # This function would do more, but I just return f for the\r\n # type checking example\r\n return f # type: ignore\r\n```\r\n\r\nI get the desired behaviour if I wrap a function with a concrete type\r\n\r\n\r\n```python\r\ndef f(x: int) -> int:\r\n return x\r\n\r\nreveal_type(lift(f)) # _F[Opt[int]] -- as expected\r\n```\r\n\r\nbut if I use a generic type, the wrapped function is not generic:\r\n\r\n```python\r\ndef g(x: _T) -> _T:\r\n return x\r\n\r\nreveal_type(lift(g)) # _F[Union[_T`-1,None]] -- _T is bound (incorrectly?)\r\n # pyre thinks this is _F[None] which is also odd...\r\nlift(g)(1) # so won't accept int\r\n```\r\n\r\n**Expected Behavior**\r\n\r\nI was expecting the wrapped function to have the type `Callable[[Opt[_T]], Opt[_T]]` and in particular I would expect to be able to call it with an `int` argument.\r\n\r\n**To Reproduce**\r\n\r\nIf I use a `Generic` class for the lifted function, it can look like this:\r\n\r\n```python\r\n# Moving generic type to Generic...\r\nclass lifted(Generic[_T]):\r\n \"\"\"Lift type _T to Opt[_T].\"\"\"\r\n\r\n def __init__(self, f: _F[_T]) -> None:\r\n self._f = f\r\n\r\n def __call__(self, x: Opt[_T], /) -> Opt[_T]:\r\n \"\"\"Call lifted.\"\"\"\r\n return self._f(x) # type: ignore\r\n\r\nreveal_type(lifted[_T]) # _F[T?] -> lifted[_T?]\r\n```\r\n\r\nand it will work for a function with a concrete type, like `f: int -> int` above\r\n\r\n```python\r\nreveal_type(lifted(f)) # lifted[int] -- perfect\r\nreveal_type(lifted(f).__call__) # Opt[int] -> Opt[int] -- perfect\r\nlifted(f)(1) # We can call with int\r\nlifted(f)(None) # We can call with None\r\n```\r\n\r\nbut things get crazy when I use a generic function like `g: _T -> _T`:\r\n\r\n```python\r\nreveal_type(lifted(g)) # lifted[_T`1] -- _T is bound!\r\n # pyre thinks (correctly I think) it is lifted[_T]\r\nreveal_type(lifted(g).__call__) # Opt[_T`1] -> Opt[_T`1]\r\n # pyre says Any -> Any which I suppose is fine\r\nlifted(g)(1) # incompatible type, int isn't an Opt[_T]\r\n # pyre seems to accep this one\r\nlifted(g)(None) # but at least None is in Opt[_T]...\r\n\r\nreveal_type(lifted[_T](g)) # lifted[_T?] -- Ok\r\nreveal_type(lifted[_T](g).__call__) # def (None) ???\r\n # pyre: Opt[_T] -> Opt[_T] as expected\r\n\r\nlifted[_T](g)(1) # incompatible type \"int\"; expected \"None\"\r\nlifted[_T](g)(None) # but at least None works...\r\n# Pyre doesn't like the [_T] here but accepts the calls without it...\r\n```\r\n\r\n**Expected Behavior**\r\n\r\nAgain, I would expect the `Generic` `lifted` class to accept types that match the wrapped function's type. This is more important when I need to add protocols to the types, but I would also expect it to work here.\r\n\r\n**Your Environment**\r\n\r\nI tested this in [mypy playground](https://mypy-play.net/?mypy=latest&python=3.10&gist=48dc95420afde2911df5b9a72f42e78c) and [pyre playground](https://pyre-check.org/play?input=from%20typing%20import%20(%0A%20%20%20%20TypeVar%2C%20%0A%20%20%20%20Protocol%2C%20Generic%2C%20%0A%20%20%20%20Optional%20as%20Opt%0A)%0A%0A%0A_T%20%3D%20TypeVar(%27_T%27)%0A%0Aclass%20_F(Protocol%5B_T%5D)%3A%0A%20%20%20%20def%20__call__(self%2C%20__x%3A%20_T)%20-%3E%20_T%3A%0A%20%20%20%20%20%20%20%20...%0A%0A%23%20Lifting%20with%20a%20function...%0Adef%20lift(f%3A%20_F%5B_T%5D)%20-%3E%20_F%5BOpt%5B_T%5D%5D%3A%0A%20%20%20%20%23%20This%20function%20would%20do%20more%2C%20but%20I%20just%20return%20f%20for%20the%0A%20%20%20%20%23%20type%20checking%20example%0A%20%20%20%20return%20f%20%23%20type%3A%20ignore%0A%0Adef%20f(x%3A%20int)%20-%3E%20int%3A%0A%20%20%20%20return%20x%0Adef%20g(x%3A%20_T)%20-%3E%20_T%3A%0A%20%20%20%20return%20x%0A%0Areveal_type(lift(f))%20%23%20_F%5BOpt%5Bint%5D%5D%20--%20as%20expected%0Areveal_type(lift(g))%20%23%20_F%5BUnion%5B_T%60-1%2CNone%5D%5D%20--%20_T%20is%20bound%20(incorrectly)%0Alift(g)(1)%20%20%20%20%20%20%20%20%20%20%20%23%20so%20won%27t%20accept%20int%0A%0A%0A%23%20Moving%20generic%20type%20to%20Generic...%0Aclass%20lifted(Generic%5B_T%5D)%3A%0A%20%20%20%20%22%22%22Lift%20type%20_T%20to%20Opt%5B_T%5D.%22%22%22%0A%0A%20%20%20%20def%20__init__(self%2C%20f%3A%20_F%5B_T%5D)%20-%3E%20None%3A%0A%20%20%20%20%20%20%20%20self._f%20%3D%20f%0A%0A%20%20%20%20def%20__call__(self%2C%20x%3A%20Opt%5B_T%5D%2C%20%2F)%20-%3E%20Opt%5B_T%5D%3A%0A%20%20%20%20%20%20%20%20%22%22%22Call%20lifted.%22%22%22%0A%20%20%20%20%20%20%20%20return%20self._f(x)%20%23%20type%3A%20ignore%0A%0Areveal_type(lifted%5B_T%5D)%20%20%23%20_F%5BT%3F%5D%20-%3E%20lifted%5B_T%3F%5D%0A%0Adef%20h(x%3A%20_T)%20-%3E%20_T%3A%0A%20%20%20%20return%20x%0A%0Areveal_type(lifted(f))%20%20%20%20%20%20%20%20%20%20%23%20lifted%5Bint%5D%20--%20perfect%0Areveal_type(lifted(f).__call__)%20%23%20Opt%5Bint%5D%20-%3E%20Opt%5Bint%5D%20--%20perfect%0Alifted(f)(1)%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%23%20We%20can%20call%20with%20int%0Alifted(f)(None)%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%23%20We%20can%20call%20with%20None%0A%0Areveal_type(lifted(g))%20%20%20%20%20%20%20%20%20%20%20%20%23%20lifted%5B_T%601%5D%20--%20_T%20is%20bound!%0Areveal_type(lifted(g).__call__)%20%20%20%23%20Opt%5B_T%601%5D%20-%3E%20Opt%5B_T%601%5D%0Alifted(g)(1)%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%23%20incompatible%20type%2C%20int%20isn%27t%20an%20Opt%5B_T%5D%0Alifted(g)(None)%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%23%20but%20at%20least%20None%20is%20in%20Opt%5B_T%5D...%0A%0Areveal_type(lifted%5B_T%5D(g))%20%20%20%20%20%20%20%20%20%20%23%20lifted%5B_T%3F%5D%20--%20Ok%0Areveal_type(lifted%5B_T%5D(g).__call__)%20%23%20def%20(None)%20%3F%3F%3F%0A%0Alifted(g)(1)%20%20%20%20%20%20%20%20%20%20%20%20%20%23%20incompatible%20type%20%22int%22%3B%20expected%20%22None%22%0Alifted(g)(None)%20%20%20%20%20%20%20%20%20%20%23%20but%20at%20least%20None%20works...%0A)","author":{"url":"https://github.com/mailund","@type":"Person","name":"mailund"},"datePublished":"2022-06-01T08:15:18Z","url":"https://github.com/python/mypy/issues/12919"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:6770293b-f3ba-f51e-f4d2-56cad4211a1b |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D416:2DCAE8:16BEF1C:1EA781D:6AFFEA23 |
| html-safe-nonce | 2090d2fe31e9af06b18ec7043a1b7ce76515ea7cd8f65b989327a5177c4b3528 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENDE2OjJEQ0FFODoxNkJFRjFDOjFFQTc4MUQ6NkFGRkVBMjMiLCJ2aXNpdG9yX2lkIjoiNzA2MzQwMzgyNjk1ODU1OTc3OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | d9b09e635f7dca56867a8efcf993c07bb350272a23ccef725b02d064d2463a5a |
| hovercard-subject-tag | issue:1255321144 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/12919/issue_layout |
| twitter:image | https://opengraph.githubassets.com/5596bcd1b52054e8697fdc741f72666febadc3b25c756134e2b33317034f990a/python/mypy/issues/12919 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/5596bcd1b52054e8697fdc741f72666febadc3b25c756134e2b33317034f990a/python/mypy/issues/12919 |
| og:image:alt | Bug Report I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where myp... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | mailund |
| hostname | github.com |
| expected-hostname | github.com |
| None | 0a0d8686a27478df474dd7415776a7a080cde8838b3cd92ea7601187572e5729 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/mypy git https://github.com/python/mypy.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 7053637 |
| octolytics-dimension-repository_nwo | python/mypy |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 7053637 |
| octolytics-dimension-repository_network_root_nwo | python/mypy |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 16a11879678aaf97e2a42c33ee70830aa084fa37 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width