René's URL Explorer Experiment

Title: gh-101196: Make isdir/isfile/exists faster on Windows by mdboom · Pull Request #101324 · python/cpython · GitHub

Open Graph Title: gh-101196: Make isdir/isfile/exists faster on Windows by mdboom · Pull Request #101324 · python/cpython

X Title: gh-101196: Make isdir/isfile/exists faster on Windows by mdboom · Pull Request #101324 · python/cpython

Description: This makes isdir, isfile and exists faster on Windows, by making fewer API calls than calling os.stat and then looking at that result. On the following microbenchmarks: Microbenchmarks import os.path import timeit for i in range(100): os.makedirs(f"exists{i}", exist_ok=True) os.symlink(f"exists{i}", f"symlink{i}", target_is_directory=True) for i in range(100): with open(f"file{i}.txt", "w") as fd: fd.write('x' * 100) os.symlink(f"file{i}.txt", f"file_symlink{i}.txt") def test_isdir_exists(): isdir = os.path.isdir for i in range(100): isdir(f"exists{i}") def test_isdir_symlink(): isdir = os.path.isdir for i in range(100): isdir(f"symlink{i}") def test_isdir_missing(): isdir = os.path.isdir for i in range(100): isdir(f"missing{i}") def test_isfile_exists(): isfile = os.path.isfile for i in range(100): isfile(f"file{i}.txt") def test_isfile_symlink(): isfile = os.path.isfile for i in range(100): isfile(f"file_symlink{i}.txt") def test_isfile_missing(): isfile = os.path.isfile for i in range(100): isfile(f"missing{i}") def test_exists_exists(): exists = os.path.exists for i in range(50): exists(f"file{i}.txt") exists(f"exists{i}") def test_exists_symlink(): exists = os.path.exists for i in range(50): exists(f"file_symlink{i}.txt") exists(f"symlink{i}") def test_exists_missing(): exists = os.path.exists for i in range(100): exists(f"missing{i}") print("isdir exists:", timeit.timeit(test_isdir_exists, number=200)) print("isdir symlink:", timeit.timeit(test_isdir_symlink, number=200)) print("isdir missing:", timeit.timeit(test_isdir_missing, number=200)) print("isfile exists:", timeit.timeit(test_isfile_exists, number=200)) print("isfile symlink:", timeit.timeit(test_isfile_symlink, number=200)) print("isfile missing:", timeit.timeit(test_isfile_missing, number=200)) print("exists exists:", timeit.timeit(test_exists_exists, number=200)) print("exists symlink:", timeit.timeit(test_exists_symlink, number=200)) print("exists missing:", timeit.timeit(test_exists_missing, number=200)) for i in range(100): os.unlink(f"symlink{i}") os.unlink(f"file_symlink{i}.txt") for i in range(100): os.rmdir(f"exists{i}") for i in range(100): os.unlink(f"file{i}.txt") I get the following results: benchmark main this PR ratio isdir exists 27.3 us 23.0 us 0.84 isdir symlink 38.0 us 33.1 us 0.87 isdir missing 9.1 us 6.5 us 0.72 isfile exists 20.7 us 16.7 us 0.81 isfile symlink 32.4 us 28.1 us 0.87 isfile missing 9.0 us 6.7 us 0.74 exists exists 24.1 us 19.2 us 0.8 exists symlink 35.6 us 29.4 us 0.83 exists missing 9.0 us 6.5 us 0.72 So therefore, these operations are around 13%-28% faster, depending on whether the file exists, is a symlink etc. NOTE: This does not improve the performance of pathlib.Path.is_dir and friends. The behavior there is slightly different in terms of error handling, but I think a similar approach could also be applied there. If this PR is acceptable, I plan to do that as follow-up work. Issue: gh-101196

Open Graph Description: This makes isdir, isfile and exists faster on Windows, by making fewer API calls than calling os.stat and then looking at that result. On the following microbenchmarks: Microbenchmarks import os.p...

X Description: This makes isdir, isfile and exists faster on Windows, by making fewer API calls than calling os.stat and then looking at that result. On the following microbenchmarks: Microbenchmarks import os.p...

Opengraph URL: https://github.com/python/cpython/pull/101324

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width