René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Optimiser breaks attribute lookup from metaclass property","articleBody":"**Bug report**\r\n\r\nThis comes from a SymPy issue: https://github.com/sympy/sympy/issues/23774\r\n\r\nIt looks like there is a nondeterministic error in the optimisation introduced in 96346cb6d0593ef9ec122614347ccb053cd63433 from #27722 first included in CPython version 3.11.0a1. The optimisation speeds up calling methods but sometimes retrieves the wrong attribute from a class whose metaclass defines a property method.\r\n\r\nThe way that this arises is quite sensitive to small changes so I haven't been able to distil a standalone reproducer. I'll show how to reproduce this using SymPy below but first this is a simplified schematic of the situation:\r\n```python\r\nclass MetaA(type):\r\n    def __init__(cls, *args, **kws):\r\n        pass\r\n\r\nclass A(metaclass=MetaA):\r\n    def method(self, rule):\r\n        return 'A method'\r\n\r\nclass MetaB(MetaA):\r\n    @property\r\n    def method(self):\r\n        def method_inner(rule):\r\n            return 'MetaB function'\r\n        return method_inner\r\n\r\nclass B(A, metaclass=MetaB):\r\n    pass\r\n\r\nprint(B.method(1))   # MetaB function\r\nprint(B().method(1)) # A method\r\n```\r\nHere `B` is a subclass of `A` but an instance of `MetaB`. Both define `method` but the `MetaB` method should be used when accessed from the class `B` rather than an instance `B()`. The failure seen in SymPy is that sometimes `B.method(1)` will execute as `A.method(1)` which fails because of the missing `self` argument.\r\n\r\nThe following code reproduces the problem and fails something like 50% of the time with SymPy 1.10.1 and CPython 3.11.0a1-3.11.0b4:\r\n```python\r\nfrom sympy import *\r\n\r\nx, y = symbols('x, y')\r\nf = Function('f')\r\n\r\n# These two lines look irrelevant but are needed:\r\nexpr = sin(x*exp(y))\r\nDerivative(expr, y).subs(y, x).doit()\r\n\r\nexpr = Subs(Derivative(f(f(x)), x), f, cos)\r\n\r\n# This is where it blows up:\r\nexpr.doit()\r\n```\r\nThe failure is not deterministic:\r\n```console\r\n$ python bug.py\r\n$ python bug.py\r\n$ python bug.py\r\nTraceback (most recent call last):\r\n  File \"/home/oscar/current/sympy/sympy.git/bug.py\", line 13, in \u003cmodule\u003e\r\n    expr.doit()\r\n    ^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/function.py\", line 2246, in doit\r\n    e = e.subs(vi, p[i])\r\n        ^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/basic.py\", line 997, in subs\r\n    rv = rv._subs(old, new, **kwargs)\r\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/cache.py\", line 70, in wrapper\r\n    retval = cfunc(*args, **kwargs)\r\n             ^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/basic.py\", line 1109, in _subs\r\n    rv = self._eval_subs(old, new)\r\n         ^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/function.py\", line 1737, in _eval_subs\r\n    nfree = new.xreplace(syms).free_symbols\r\n            ^^^^^^^^^^^^^^^^^^\r\nTypeError: Basic.xreplace() missing 1 required positional argument: 'rule'\r\n```\r\nThe failure can be reproduced deterministically by setting the hash seed:\r\n```console\r\n$ PYTHONHASHSEED=1 python bug.py\r\n...\r\nTypeError: Basic.xreplace() missing 1 required positional argument: 'rule'\r\n```\r\nIn the debugger the same code that already failed succeeds:\r\n```console\r\n$ PYTHONHASHSEED=1 python -m pdb bug.py \r\n\u003e /home/oscar/current/sympy/sympy.git/bug.py(1)\u003cmodule\u003e()\r\n-\u003e from sympy import *\r\n(Pdb) c\r\nTraceback (most recent call last):\r\n  File \"/media/oscar/EXT4_STUFF/src/cpython/Lib/pdb.py\", line 1768, in main\r\n    pdb._run(target)\r\n  File \"/media/oscar/EXT4_STUFF/src/cpython/Lib/pdb.py\", line 1646, in _run\r\n    self.run(target.code)\r\n  File \"/media/oscar/EXT4_STUFF/src/cpython/Lib/bdb.py\", line 597, in run\r\n    exec(cmd, globals, locals)\r\n  File \"\u003cstring\u003e\", line 1, in \u003cmodule\u003e\r\n  File \"/home/oscar/current/sympy/sympy.git/bug.py\", line 13, in \u003cmodule\u003e\r\n    expr.doit()\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/function.py\", line 2255, in doit\r\n    e = e.subs(vi, p[i])\r\n        ^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/basic.py\", line 993, in subs\r\n    rv = rv._subs(old, new, **kwargs)\r\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/cache.py\", line 70, in wrapper\r\n    retval = cfunc(*args, **kwargs)\r\n             ^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/basic.py\", line 1105, in _subs\r\n    rv = self._eval_subs(old, new)\r\n         ^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/home/oscar/current/sympy/sympy.git/sympy/core/function.py\", line 1746, in _eval_subs\r\n    nfree = new.xreplace(syms).free_symbols\r\n            ^^^^^^^^^^^^^^^^^^\r\nTypeError: Basic.xreplace() missing 1 required positional argument: 'rule'\r\nUncaught exception. Entering post mortem debugging\r\nRunning 'cont' or 'step' will restart the program\r\n\u003e /home/oscar/current/sympy/sympy.git/sympy/core/function.py(1746)_eval_subs()\r\n-\u003e nfree = new.xreplace(syms).free_symbols\r\n(Pdb) p new.xreplace\r\n\u003cfunction FunctionClass.xreplace.\u003clocals\u003e.\u003clambda\u003e at 0x7f0dd0f763e0\u003e\r\n(Pdb) p new.xreplace(syms)\r\ncos\r\n(Pdb) p new.xreplace(syms).free_symbols\r\nset()\r\n```\r\nThe actual arrangement of SymPy classes is something like this:\r\n```python\r\nclass ManagedProperties(type):\r\n    def __init__(cls, *args, **kws):\r\n        pass\r\n\r\nclass Basic(metaclass=ManagedProperties):\r\n    def xreplace(self, rule):\r\n        print('Basic')\r\n\r\nclass Expr(Basic):\r\n    pass\r\n\r\nclass FunctionClass(ManagedProperties):\r\n    @property\r\n    def xreplace(self):\r\n        return lambda rule: print('functionclass')\r\n\r\nclass Application(Basic, metaclass=FunctionClass):\r\n    pass\r\n\r\nclass Function(Application, Expr):\r\n    pass\r\n\r\nclass cos(Function):\r\n    pass\r\n\r\ncos.xreplace(1)\r\n```\r\n\r\n**Your environment**\r\n\r\n- CPython versions tested on: 3.11.0a1-3.11.0b4 (3.10.5 or lower does not have the bug)\r\n- Operating system and architecture: Ubuntu x86-64.","author":{"url":"https://github.com/oscarbenjamin","@type":"Person","name":"oscarbenjamin"},"datePublished":"2022-07-13T20:09:44.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":18},"url":"https://github.com/94822/cpython/issues/94822"}