René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[3.12] getattr_static can result in reference leaks","articleBody":"# Bug report\r\n\r\n### Bug description:\r\n\r\nI discovered this bug while working on https://github.com/pytorch/pytorch/issues/124302.\r\n\r\nIt looks like calling `getattr_static` is causing an object to be reference leaked:\r\n```python\r\nimport gc\r\nfrom inspect import getattr_static\r\nimport weakref\r\n\r\nclass C1:\r\n    pass\r\n\r\ndef main():\r\n    obj = C1()\r\n    weakref.finalize(obj, lambda: print(\"obj deleted!\"))\r\n    class C2:\r\n        def __init__(self):\r\n            self.obj = obj\r\n    c2 = C2()\r\n    getattr_static(c2, \"bad\", None)\r\n    print(\"done main!\")\r\n\r\nmain()\r\ngc.collect()\r\nprint(\"done!\")\r\n```\r\n\r\nOutput:\r\n```\r\ndone main!\r\ndone!\r\nobj deleted!\r\n```\r\n\r\nIf I comment out the `getattr_static` line, the output is as expected:\r\n```\r\ndone main!\r\nobj deleted!\r\ndone!\r\n```\r\n\r\nIt looks like this PR https://github.com/python/cpython/pull/104267 indirectly cached calls to `getattr_static`, which is resulting in reference leaks. Perhaps this cache needs to use weak references?\r\n\r\nOriginal PyTorch code for reference (`torch.compile` calls `getattr_static` on `mod` at some point):\r\n```python\r\nimport torch\r\n\r\nimport gc\r\nimport sys\r\nimport weakref\r\n\r\nfrom inspect import getattr_static\r\n\r\ndef dbg(o):\r\n    refs = gc.get_referrers(o)\r\n    print(len(refs), sys.getrefcount(o))\r\n    return refs\r\n\r\ngm_list = []\r\ndef backend(gm, _):\r\n    gm_list.append(weakref.ref(gm, lambda _: print(\"gm deleted\")))\r\n    # breakpoint()\r\n    return gm\r\n\r\ndef main():\r\n    param = torch.nn.Parameter(torch.randn(5, 5))\r\n\r\n    class Mod(torch.nn.Module):\r\n        def __init__(self):\r\n            super().__init__()\r\n            self.param = param\r\n\r\n        def forward(self, x):\r\n            return self.param * x\r\n\r\n    mod = Mod()\r\n    ref = weakref.ref(param, lambda _: print(\"obj deleted\"))\r\n    opt_mod = torch.compile(mod, backend=backend)\r\n    print(opt_mod(torch.randn(5, 5)))\r\n    return ref\r\n\r\nref = main()\r\n\r\ngc.collect()\r\nprint(\"done!\")\r\nprint(ref)\r\n```\r\n\r\n\r\n### CPython versions tested on:\r\n\r\n3.12\r\n\r\n### Operating systems tested on:\r\n\r\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-118202\n* gh-118232\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/williamwen42","@type":"Person","name":"williamwen42"},"datePublished":"2024-04-17T21:58:22.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/118013/cpython/issues/118013"}