Title: gh-132042: Speed up slot lookup during class creation by eendebakpt · Pull Request #159479 · python/cpython · GitHub
Open Graph Title: gh-132042: Speed up slot lookup during class creation by eendebakpt · Pull Request #159479 · python/cpython
X Title: gh-132042: Speed up slot lookup during class creation by eendebakpt · Pull Request #159479 · python/cpython
Description: fixup_slot_dispatchers() resolves the special-method slots of a new class with one dict lookup per slot name and per class in the MRO: about 92 names times the MRO length, ~180 lookups for class A: pass and ~650 for a MutableMapping subclass. This PR scans each dict along the MRO once instead, matches its dunder keys against a small static table of slot names, and records the first hit per slot. The rest of update_one_slot() is unchanged and the result is identical to find_name_in_mro() for every slot. The per-name lookup remains as the fallback for dicts that cannot be scanned (non-string keys, more than 1000 entries, or on the free-threaded build a dict whose mutex is held by another thread, since the scan never blocks while holding the type lock) and for update_slot(). Compared to earlier work on this problem, there is no caching and no per-type memory (#4902 (gh-76527) cached the slot descriptors ; #132618 builds a merged MRO dict per class creation (allocation plus a PyDict_Merge per base) and had to change test_descr). A regression test covers class creation racing with setattr on a base on the free-threaded build. @sergey-miryanov Benchmarks -O3 non-PGO build on Linux, pyperf pinned to one core: Benchmark main PR class A: pass 4.01 µs 2.43 µs: 1.65x faster class E(D), depth-5 single-inheritance chain 7.01 µs 2.84 µs: 2.47x faster class T(MagicMock) 7.87 µs 3.37 µs: 2.34x faster class T(MutableMapping) 9.79 µs 5.58 µs: 1.76x faster class T(A, B) (multiple inheritance) 5.49 µs 2.79 µs: 1.97x faster Mock() (creates a class per instance) 16.2 µs 10.7 µs: 1.52x faster MagicMock() 77.4 µs 63.4 µs: 1.22x faster types.new_class('A') 6.77 µs 4.42 µs: 1.53x faster typing.Protocol subclass 21.2 µs 15.1 µs: 1.40x faster typing.TypedDict subclass 27.6 µs 22.2 µs: 1.24x faster namedtuple('P', 'x y z') 29.4 µs 27.7 µs: 1.06x faster Enum with two members 29.9 µs 28.1 µs: 1.07x faster @dataclass with two fields 127 µs 125 µs: not significant A.__len__ = f 219 ns 172 ns: 1.27x faster A.__tablename__ = 'x' (dunder, not a slot) 115 ns 69.7 ns: 1.65x faster The benchmark script from #132156 / #132618 (1000-iteration cases): Benchmark main PR 1000-empty_cls 6.42 ms 3.96 ms: 1.62x faster 1000-cls_with_dunders 7.24 ms 4.91 ms: 1.47x faster 1000-empty_cls_with_bases-bases=['A', 'B'] 9.06 ms 4.61 ms: 1.96x faster 1000-cls_with_bases-bases=['A', 'B'] 9.51 ms 5.67 ms: 1.68x faster 1000-empty_cls_with_bases-bases=['A', 'B', 'D'] 9.93 ms 5.07 ms: 1.96x faster 1000-cls_with_bases-bases=['A', 'B', 'D'] 11.2 ms 5.98 ms: 1.87x faster 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun'] 8.75 ms 4.70 ms: 1.86x faster 1000-cls_with_bases-bases=['A_dun', 'B_dun'] 9.77 ms 5.74 ms: 1.70x faster 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] 10.2 ms 5.51 ms: 1.85x faster 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] 11.1 ms 6.49 ms: 1.71x faster 1000-empty_cls_with_bases-bases=['Logger'] 8.40 ms 4.76 ms: 1.76x faster 1000-cls_with_bases-bases=['Logger'] 9.14 ms 5.95 ms: 1.54x faster 1000-empty_cls_with_bases-bases=['DatagramHandler'] 10.3 ms 5.39 ms: 1.91x faster 1000-cls_with_bases-bases=['DatagramHandler'] 11.2 ms 6.47 ms: 1.74x faster 1000-empty_cls_with_bases-bases=['MagicMock'] 12.3 ms 5.73 ms: 2.15x faster 1000-cls_with_bases-bases=['MagicMock'] 13.0 ms 6.97 ms: 1.87x faster 1000-empty_cls_with_bases-bases=['Shelf'] 14.7 ms 8.41 ms: 1.75x faster 1000-cls_with_bases-bases=['Shelf'] 15.6 ms 9.96 ms: 1.57x faster 1000-empty_cls_with_bases-bases=['tuple'] 7.35 ms 4.75 ms: 1.55x faster 1000-cls_with_bases-bases=['tuple'] 8.23 ms 6.10 ms: 1.35x faster 1000-empty_cls_with_bases-bases=['dict'] 7.57 ms 5.09 ms: 1.49x faster 1000-cls_with_bases-bases=['dict'] 8.37 ms 6.15 ms: 1.36x faster 1000-empty_cls_with_bases-bases=['list'] 7.56 ms 5.11 ms: 1.48x faster 1000-cls_with_bases-bases=['list'] 8.48 ms 6.52 ms: 1.30x faster Geometric mean (ref) 1.67x faster Notes: #132618 no longer applies to main, so for a direct comparison its approach was re-implemented on top of current main. On 14 of the cases above, run back to back on the same machine, this PR is 1.78x faster than main (geometric mean) and the #132618 approach 1.39x. Whole-process effect: python -I -c pass executes 8% fewer instructions (47.6M to 43.8M, callgrind), and importing class-heavy stdlib modules (asyncio, typing, unittest.mock, email.message, argparse, inspect, decimal, tkinter) is 4-8% faster in wall time. Three-way comparison: main vs this PR vs #132618 on current main Benchmark main this PR #132618 1000-empty_cls 3.90 ms 2.52 ms: 1.55x faster 3.13 ms: 1.24x faster 1000-cls_with_dunders 4.44 ms 3.10 ms: 1.43x faster 3.71 ms: 1.20x faster 1000-empty_cls_with_bases-bases=['A', 'B', 'D'] 6.30 ms 3.18 ms: 1.98x faster 3.91 ms: 1.61x faster 1000-cls_with_bases-bases=['A', 'B', 'D'] 6.95 ms 3.83 ms: 1.81x faster 4.55 ms: 1.53x faster 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] 6.24 ms 3.24 ms: 1.93x faster 3.98 ms: 1.57x faster 1000-empty_cls_with_bases-bases=['Logger'] 5.37 ms 2.88 ms: 1.87x faster 3.97 ms: 1.35x faster 1000-cls_with_bases-bases=['Logger'] 5.96 ms 3.53 ms: 1.69x faster 4.64 ms: 1.29x faster 1000-empty_cls_with_bases-bases=['DatagramHandler'] 6.67 ms 3.15 ms: 2.12x faster 4.27 ms: 1.56x faster 1000-empty_cls_with_bases-bases=['MagicMock'] 7.91 ms 3.47 ms: 2.28x faster 4.81 ms: 1.64x faster 1000-cls_with_bases-bases=['MagicMock'] 8.47 ms 4.12 ms: 2.06x faster 5.48 ms: 1.54x faster 1000-empty_cls_with_bases-bases=['Shelf'] 9.48 ms 5.00 ms: 1.90x faster 6.35 ms: 1.49x faster 1000-empty_cls_with_bases-bases=['tuple'] 4.74 ms 2.91 ms: 1.63x faster 3.66 ms: 1.30x faster 1000-cls_with_bases-bases=['dict'] 5.33 ms 3.72 ms: 1.43x faster 4.69 ms: 1.13x faster 1000-empty_cls_with_bases-bases=['list'] 4.76 ms 3.07 ms: 1.55x faster 4.06 ms: 1.17x faster Geometric mean (ref) 1.78x faster 1.39x faster Benchmark script for the first table import pyperf runner = pyperf.Runner() runner.timeit("class A: pass", stmt="class A: pass") runner.timeit("class E(D) depth-5 chain", setup="class A: pass\nclass B(A): pass\nclass C(B): pass\nclass D(C): pass", stmt="class E(D): pass") runner.timeit("class T(MagicMock)", setup="from unittest.mock import MagicMock", stmt="class T(MagicMock): pass") runner.timeit("class T(MutableMapping)", setup="from cohis PR runner.timeit("types.new_class", setup="import types", stmt="types.new_class('A')") runner.timeit("typing.Protocol", setup="from typing import Protocol", stmt="class P(Protocol):\n def f(self) -> int: ...") runner.timeit("typing.TypedDict", setup="from typing import TypedDict", stmt="class P(TypedDict):\n x: int\n y: int") runner.timeit("namedtuple", setup="from collections import namedtuple", stmt="namedtuple('P', 'x y z')") runner.timeit("Enum", setup="from enum import Enum", stmt="class Color(Enum):\n RED = 1\n GREEN = 2") runner.timeit("dataclass", setup="from dataclasses import dataclass", stmt="@dataclass\nclass P:\n x: int\n y: int") runner.timeit("setattr slot dunder", setup="class A: pass\nf = lambda s: 0", stmt="A.__len__ = f") runner.timeit("setattr non-slot dunder", setup="class A: pass", stmt="A.__tablename__ = 'x'") Generated with help from Claude Code Issue: gh-132042
Open Graph Description: fixup_slot_dispatchers() resolves the special-method slots of a new class with one dict lookup per slot name and per class in the MRO: about 92 names times the MRO length, ~180 lookups for class A:...
X Description: fixup_slot_dispatchers() resolves the special-method slots of a new class with one dict lookup per slot name and per class in the MRO: about 92 names times the MRO length, ~180 lookups for class A:...
Opengraph URL: https://github.com/python/cpython/pull/159479
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:681ba1c2-fd2c-4923-de1c-618a50a3be97 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 9DE4:33444D:37ABD7E:49FBD81:6B0212FD |
| html-safe-nonce | 0f3b24dacdc7794dd54624658c833340c332ce37c8d6d6aac1a87ee60d31810b |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5REU0OjMzNDQ0RDozN0FCRDdFOjQ5RkJEODE6NkIwMjEyRkQiLCJ2aXNpdG9yX2lkIjoiNzQ0ODE0NjQxMzIxODgyOTA1MyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | a2b6d53f872c47d4f20b05307bd4775cd1d43a4dd91ddb847565f1231335276b |
| hovercard-subject-tag | pull_request:4876762589 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/159479/files |
| twitter:image | https://avatars.githubusercontent.com/u/883786?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/883786?s=400&v=4 |
| og:image:alt | fixup_slot_dispatchers() resolves the special-method slots of a new class with one dict lookup per slot name and per class in the MRO: about 92 names times the MRO length, ~180 lookups for class A:... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width