René's URL Explorer Experiment

Title: gh-143658: importlib.metadata: Use `str.translate` to improve performance of `importlib.metadata.Prepared.normalized` by hugovk · Pull Request #143660 · python/cpython · GitHub

Open Graph Title: gh-143658: importlib.metadata: Use `str.translate` to improve performance of `importlib.metadata.Prepared.normalized` by hugovk · Pull Request #143660 · python/cpython

X Title: gh-143658: importlib.metadata: Use `str.translate` to improve performance of `importlib.metadata.Prepared.normalized` by hugovk · Pull Request #143660 · python/cpython

Description: We can apply @henryiii's improvement to packaging in pypa/packaging#1030 (see also https://iscinumpy.dev/post/packaging-faster/) to improve the performance of canonicalize_name and make it ~3.7 times faster. Benchmark Run Prepared.normalize(n) on every name in PyPI: # benchmark_names_stdlib.py import sqlite3 import timeit from importlib.metadata import Prepared # Get data with: # curl -L https://github.com/pypi-data/pypi-json-data/releases/download/latest/pypi-data.sqlite.gz | gzip -d > pypi-data.sqlite # Or ues pre-cached files from: # https://gist.github.com/hugovk/efdbee0620cc64df7b405b52cf0b6e42 CACHE_FILE = "/tmp/bench/names.txt" DB_FILE = "/tmp/bench/pypi-data.sqlite" try: with open(CACHE_FILE) as f: TEST_ALL_NAMES = [line.rstrip("\n") for line in f] except FileNotFoundError: TEST_ALL_NAMES = [] with sqlite3.connect(DB_FILE) as conn: with open(CACHE_FILE, "w") as cache: for (name,) in conn.execute("SELECT name FROM projects"): if name: TEST_ALL_NAMES.append(name) cache.write(name + "\n") def bench(): for n in TEST_ALL_NAMES: Prepared.normalize(n) if __name__ == "__main__": print(f"Loaded {len(TEST_ALL_NAMES):,} names") t = timeit.timeit("bench()", globals=globals(), number=1) print(f"Time: {t:.4f} seconds") Benchmark data can be found at https://gist.github.com/hugovk/efdbee0620cc64df7b405b52cf0b6e42 Before With optimisations: ❯ ./python.exe benchmark_names_stdlib.py Loaded 8,344,947 names Time: 5.1483 seconds After ❯ ./python.exe benchmark_names_stdlib.py Loaded 8,344,947 names Time: 1.3754 seconds 3.7 times faster. Issue: gh-143658

Open Graph Description: We can apply @henryiii's improvement to packaging in pypa/packaging#1030 (see also https://iscinumpy.dev/post/packaging-faster/) to improve the performance of canonicalize_name and make it ~3.7...

X Description: We can apply @henryiii's improvement to packaging in pypa/packaging#1030 (see also https://iscinumpy.dev/post/packaging-faster/) to improve the performance of canonicalize_name and make it ...

Opengraph URL: https://github.com/python/cpython/pull/143660

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width