Title: gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative) by dolfinus · Pull Request #141171 · python/cpython · GitHub
Open Graph Title: gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative) by dolfinus · Pull Request #141171 · python/cpython
X Title: gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative) by dolfinus · Pull Request #141171 · python/cpython
Description: Issue: gh-92810 Alternative implementation of #131914 which doesn't modify class attributes on every __subclasscheck__ call. Instead it checks in __new__ does current class have def __subclasses__(cls) overriden, and if so, enables case for scls in cls.__subclasses__(), which is disabled by default. To handle cases like: class Number(ABC): ... class Real(Number): ... class Integral(Real): ... Integral.register(int) assert issubclass(int, Number) is True which previously were implemented via __subclasses__, method cls.register(subclass) is calling super(cls).register(subclass) recursively ("bubble-up" registration to all the parents). For benchmark from #131914: sudo ./python -m pyperf system tune taskset -c 0 ./python benchmark.py --metaclass abc.ABCMeta --rounds 3 --classes 5000 taskset -c 0 ./python benchmark.py --metaclass _py_abc.ABCMeta --rounds 3 --classes 5000 Impl Max memory before, MB Max memory after, MB _abc 6331 49 _py_abc 4422 59 Impl Total time before Total time after _abc 6m 16s 1m 28s _py_abc 8m 42s 3m 50s Check Impl before after Impl before after isinstance(child, Parent) _abc 0.115us 4MiB...15MiB 0.120us 4MiB...15MiB _py_abc 0.212us 11MiB...24MiB 0.221us 11MiB...24MiB issubclass(Child, Parent) _abc 0.105us 0MiB...1MiB 0.112us 0MiB...1MiB _py_abc 0.203us 6MiB...8MiB 0.212us 6MiB...8MiB isinstance(child, Grandparent) _abc 0.112us 0MiB...2MiB 0.117us 0MiB...2MiB _py_abc 0.210us 4MiB...7MiB 0.218us 4MiB...7MiB issubclass(Child, Grandparent) _abc 0.103us 0MiB 0.110us 0MiB _py_abc 0.201us 0MiB...1MiB 0.210us 0MiB...1MiB not isinstance(child, Sibling) _abc 0.113us 4MiB...14MiB 0.115us 4MiB...15MiB _py_abc 0.348us 13MiB...23MiB 0.362us 13MiB...23MiB not issubclass(Child, Sibling) _abc 0.105us 1MiB 0.110us 1MiB _py_abc 0.328us 8MiB...10MiB 0.337us 9MiB...11MiB not isinstance(child, Cousin) _abc 0.115us 1MiB...2MiB 0.116us 1MiB...2MiB _py_abc 0.350us 7MiB...9MiB 0.362us 7MiB...9MiB not issubclass(Child, Cousin) _abc 0.104us 0MiB 0.109us 0MiB _py_abc 0.329us 4MiB 0.337us 4MiB...5MiB not isinstance(child, Uncle) _abc 7.268us 6174MiB...6333MiB 0.121us 0MiB...1MiB _py_abc 9.957us 4382MiB...4422MiB 0.365us 6MiB not issubclass(Child, Uncle) _abc 7.099us 6171MiB 0.117us 0MiB _py_abc 9.936us 4380MiB 0.337us 4MiB Memory increment is measured during isinstance() / issubclass() calls, not during preparation, like class creation or registration where actual registry allocation is performed. So memory usage in tables below is almost always 0. Timing drop in _py_abc implementation for 2 first rows is due to if subclass in cls._abc_registry: check added to match _abc.c implementation. Check Impl before after Impl before after isinstance(child, Parent.register) _abc 0.273us 0MiB 0.273us 0MiB _py_abc 0.440us 0MiB 0.270us 0MiB issubclass(Child, Parent.register) _abc 0.154us 0MiB 0.159us 0MiB _py_abc 0.427us 0MiB 0.258us 0MiB isinstance(child, Grandparent.register) _abc 0.114us 0MiB 0.117us 0MiB _py_abc 0.253us 0MiB 0.256us 0MiB issubclass(Child, Grandparent.register) _abc 0.103us 0MiB 0.111us 0MiB _py_abc 0.240us 0MiB 0.245us 0MiB not isinstance(child, Sibling.register) _abc 0.027us 0MiB 0.028us 1MiB _py_abc 0.028us 0MiB 0.028us 2MiB not issubclass(Child, Sibling.register) _abc 0.018us 0MiB 0.018us 1MiB _py_abc 0.018us 0MiB 0.018us 2MiB not isinstance(child, Cousin.register) _abc 0.028us 0MiB 0.028us 2MiB _py_abc 0.028us 0MiB 0.029us 3MiB not issubclass(Child, Cousin.register) _abc 0.018us 0MiB 0.018us 2MiB _py_abc 0.019us 0MiB 0.019us 3MiB not isinstance(child, Uncle.register) _abc 0.249us 0MiB 0.234us 2MiB _py_abc 0.843us 0MiB 0.907us 4MiB not issubclass(Child, Uncle.register) _abc 0.238us 0MiB 0.230us 2MiB _py_abc 0.815us 0MiB 0.854us 4MiB Check Impl before after Impl before after isinstance(child, Parent.__subclasses__) _abc 0.128us 0MiB 0.132us 0MiB _py_abc 0.305us 0MiB 0.308us 0MiB issubclass(Child, Parent.__subclasses__) _abc 0.118us 0MiB 0.125us 0MiB _py_abc 0.293us 0MiB 0.292us 0MiB isinstance(child, Grandparent.__subclasses__) _abc 0.126us 0MiB 0.130us 0MiB _py_abc 0.304us 0MiB 0.306us 0MiB issubclass(Child, Grandparent.__subclasses__) _abc 0.116us 0MiB 0.122us 0MiB _py_abc 0.293us 0MiB 0.292us 0MiB not isinstance(child, Sibling.__subclasses__) _abc 0.028us 0MiB 0.028us 0MiB _py_abc 0.028us 0MiB 0.028us 1MiB not issubclass(Child, Sibling.__subclasses__) _abc 0.018us 0MiB 0.018us 0MiB _py_abc 0.018us 0MiB 0.018us 1MiB not isinstance(child, Cousin.__subclasses__) _abc 0.028us 0MiB 0.028us 0MiB _py_abc 0.028us 0MiB 0.028us 1MiB not issubclass(Child, Cousin.__subclasses__) _abc 0.018us 0MiB 0.018us 0MiB _py_abc 0.018us 0MiB 0.020us 1MiB not isinstance(child, Uncle.__subclasses__) _abc 0.145us 0MiB 0.150us 1MiB _py_abc 0.553us 0MiB 0.574us 2MiB not issubclass(Child, Uncle.__subclasses__) _abc 0.134us 0MiB 0.144us 1MiB _py_abc 0.526us 0MiB 0.542us 2MiB Just to check that nothing is broken: Check Impl before after Impl before after not isinstance(child, Unrelated) _abc 0.028us 0MiB 0.028us 0MiB _py_abc 0.028us 0MiB 0.028us 0MiB not issubclass(Child, Unrelated) _abc 0.018us 0MiB 0.018us 0MiB _py_abc 0.018us 0MiB 0.018us 0MiB not isinstance(child, UnrelatedABC) _abc 0.118us 0MiB 0.118us 0MiB _py_abc 0.477us 0MiB 0.479us 0MiB not issubclass(Child, UnrelatedABC) _abc 0.110us 0MiB 0.112us 0MiB _py_abc 0.450us 0MiB 0.451us 0MiB Flamegraphs for _py_abc impl and test issubclass_uncle (the most time and memory consuming case on main): main_vs_pr141171.tar.gz Alternative: #150540 drops recursive subclass check completely, and does not mess around with class attributes.
Open Graph Description: Issue: gh-92810 Alternative implementation of #131914 which doesn't modify class attributes on every __subclasscheck__ call. Instead it checks in __new__ does current class have def __subclas...
X Description: Issue: gh-92810 Alternative implementation of #131914 which doesn't modify class attributes on every __subclasscheck__ call. Instead it checks in __new__ does current class have def __sub...
Opengraph URL: https://github.com/python/cpython/pull/141171
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:8596e457-9b18-1814-9f01-2179914da183 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 8394:3E418A:15EA352:1CD3221:6B01FBD6 |
| html-safe-nonce | 352baa85aac25b6c489b63a3b0cd6013c3cd76aaa41f1545ab91f34d12b433ae |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4Mzk0OjNFNDE4QToxNUVBMzUyOjFDRDMyMjE6NkIwMUZCRDYiLCJ2aXNpdG9yX2lkIjoiMjc3NDc3NjA4NzM3MTE4NTExMCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 295573f26ae21391916c1c6b03ee9eb721c116f7b494f9f3dae8cde9d580caef |
| hovercard-subject-tag | pull_request:2986427877 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/141171/files |
| twitter:image | https://avatars.githubusercontent.com/u/4661021?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/4661021?s=400&v=4 |
| og:image:alt | Issue: gh-92810 Alternative implementation of #131914 which doesn't modify class attributes on every __subclasscheck__ call. Instead it checks in __new__ does current class have def __subclas... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width