René's URL Explorer Experiment

Title: gh-83714: Implement os.statx by jbosboom · Pull Request #139178 · python/cpython · GitHub

Open Graph Title: gh-83714: Implement os.statx by jbosboom · Pull Request #139178 · python/cpython

X Title: gh-83714: Implement os.statx by jbosboom · Pull Request #139178 · python/cpython

Description: This PR implements os.statx, an interface to the Linux statx(2) system call introduced in kernel version 4.11 (April 2017) and first available in glibc version 2.28 (August 2018). This is derived from my earlier PR #136334 with the changes to os.stat removed, plus other changes (see below). This PR implements the return value of os.statx with a custom C type statx_result, which contains a struct statx and uses member and getset descriptors to lazily create Python objects on attribute access. In a comment on the previous PR, @vstinner suggested using types.SimpleNamespace as the return value. I implemented that on the statx-simplenamespace branch in my fork. Using the benchmarks from this script (note they are not all fair comparisons): Benchmark statx-nocache namespace statx-sizemtime-mask-size-mtimens 1.86 us 3.07 us: 1.65x slower statx-basic-all 2.29 us 4.57 us: 1.99x slower statx-everything-all 2.61 us 6.34 us: 2.43x slower It's slower than this PR when only size and mtime are requested, because it creates objects for the unconditionally valid members and both the float-seconds and int-nanoseconds timestamps. We could get some of this back by defining our own mask bits (bit 32 and above). As more bits are set in the mask and attributes are accessed, the gap widens. I'm not sure if that's due to dict resizing or slower attribute access or both, and I don't see how to improve either. The advantage of the namespace implementation is its simplicity, and any speed hacks would dilute that. (Also, because it doesn't create unrequested members, it's not a perfect wrapper around the system call. For "real" use it doesn't matter, but for testing the syscall, you'll miss some quirks. For example, btrfs seems to always return atime and btime, but returns mtime and ctime (always together) only if mtime and/or ctime were requested.) In this PR, most attributes are implemented with member descriptors pointing into the struct statx or a member in statx_result, so each attribute access on statx_result creates a new int or float object. My previous PR cached the created objects in the statx_result for the commonly-used attributes to avoid creating them more than once, which requires getset descriptors. Obviously, creating objects has a cost, but getset descriptors are slower than member descriptors. The crossover point turns out to be about two accesses (per attribute). If an attribute is only used once, this PR is faster; twice, slightly faster or even; three or more times, the previous PR is faster. I decided the complexity of the cache wasn't worth it. There's a cleaned-up version of the caching implementation on my fork if you want to take your own measurements. (I also didn't test the cache in the free-threaded build; I think the atomics/locking is correct, but I don't really understand how critical sections that can be "suspended" can possibly be safe.) Issue: gh-83714 📚 Documentation preview 📚: https://cpython-previews--139178.org.readthedocs.build/

Open Graph Description: This PR implements os.statx, an interface to the Linux statx(2) system call introduced in kernel version 4.11 (April 2017) and first available in glibc version 2.28 (August 2018). This is derived ...

X Description: This PR implements os.statx, an interface to the Linux statx(2) system call introduced in kernel version 4.11 (April 2017) and first available in glibc version 2.28 (August 2018). This is derived ...

Opengraph URL: https://github.com/python/cpython/pull/139178

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width