René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Reconsider making Mapping covariant in the key type","articleBody":"Confusion with invariant container types is one of the more common questions mypy users have. Often these can be easily worked around by using a covariant type such as `Sequence` instead of `List`. However, there is no fully covariant dictionary-like type -- `Mapping` is invariant in the key type, since the key type is used as an argument type of `__getitem__` and `get`. This is particularly problematic in Python 2, as mappings with `unicode` keys are common, and a mapping with `str` keys is not compatible with them, even though they are fine at runtime (at least if the keys are ascii only).\r\n\r\nI can see a few things we could do.\r\n\r\n**1) Don't care about unsafety**\r\n\r\nWe'd make `Mapping` covariant even though it's known to be unsafe (https://github.com/python/typing/pull/273). The argument is that the potential unsafety is less of a problem than the confusing user experience. This wouldn't help with certain related things like using a `unicode` key to access `Mapping[str, X]`, which is arguably okay.\r\n\r\nNote that this was [proposed and rejected](https://github.com/python/typing/pull/273) earlier.\r\n\r\n**2) Use `object` in `Mapping` argument types**\r\n\r\nWe'd make `Mapping` covariant and use `object` in the argument type of `__getitem__` and `get`. This would result in less effective type checking, but I'd consider this to still be correct at least for `get`, since it can always fall back to the default. Also, `__getitem__` would typically only generate a `KeyError` if given a key of an unrelated type, and this is arguably not a type error. Indexing operations on mappings can already fail with `KeyError`, and this isn't considered a type safety issue.\r\n\r\nWe could also recommend that type checkers special case type checking of `Mapping.__getitem__` and `Mapping.get` (and also the corresponding methods in subclasses such as `Dict`). A reasonable rule would be to require that the actual key type in code like `d[k]` overlaps with the declared key type. This would allow things like using `unicode` keys for `Mapping[str, int]`, which should perhaps be fine in Python 2. We could still reject things like `d[1]` if the declared key type is `str`, since `int` and `str` are not overlapping.\r\n\r\nSubclasses of `Mapping` that only support specific runtime key types would now be rejected. For example, this wouldn't work any more:\r\n\r\n```py\r\nclass IntMap(Mapping[int, int]):\r\n    def __getitem__(self, k: int) -\u003e int: ...   # needs an object argument\r\n    ...\r\n```\r\n\r\nHowever, it would be easy enough to refactor this to type check:\r\n\r\n```py\r\nclass IntMap(Mapping[int, int]):\r\n    def __getitem__(self, k: object) -\u003e int: ...\r\n        assert isinstance(k, int)  # sorry, can't deal with non-int keys\r\n        ...\r\n    ...\r\n```\r\n\r\nI think that this is reasonable -- the required assert (or cast) makes it clear that this is potentially unsafe.\r\n\r\n**3) Just teach users to work around using unions or type variables**\r\n\r\nMaybe keeping `Mapping` is invariant in the key type is reasonable. Often the lack of covariance can be worked around by using other type system features. Here are some examples:\r\n\r\n* Use `Union[Mapping[str, int], Mapping[unicode, int]]` instead of `Mapping[unicode, int]`.\r\n* Use `Mapping[AnyStr, int]` in Python 2 and `Mapping[str, int]` in Python 3 using a conditional type alias.\r\n* Define a type variable like `AnyText = TypeVar('AnyText', str, Text)` and use `Mapping[AnyText, int]`. This is a little awkward, since in Python 3 the values of the type variable would be `str` and `str`.\r\n\r\nI don't like these for a few reasons:\r\n* These are not intuitive.\r\n* These are inconsistent with other containers such as `Sequence`. \r\n* These are more verbose than just `Mapping[unicode, int]`.\r\n","author":{"url":"https://github.com/JukkaL","@type":"Person","name":"JukkaL"},"datePublished":"2017-06-28T15:35:21.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/445/typing/issues/445"}