René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Benchmarks for common python I/O patterns","articleBody":"As work happens on I/O pieces I've been building specialized micro-benchmarks (ex. [gh-120754 Speed up open().read() pattern by reducing the number of system calls](https://github.com/python/cpython/issues/120754) and others have [gh-117151: IO performance improvement, increase io.DEFAULT_BUFFER_SIZE to 128k](https://github.com/python/cpython/issues/117151)), it would be nice to have more general benchmarks to validate I/O performance for common cases.\n\nTalking a little with people at PyConUS there was some interest in the tests, and a general desire for I/O tests not to be enabled by default, but to be a group which can be manually run.\n\nGeneral I/O shapes I'm hoping to add benchmarks for:\n - read/write all of the byes of a file in a single call (including pathlib.Path.read_text, pathlib.Path.write_text)\n - read/write many small files (ex. `.pyc` files, maybe just `compile_all`?)\n - streaming bytes read/write (ex. to a pipe / console such as stdin/stdout/stderr, non-seekable devices)\n - read/write a zipfile, tarfile (read + seek, write + seek, in particular buffering behavior)\n - use `zipimport`\n - Create a `zipapp`\n - Multi-threaded write to stdout, stderr (ex. logging in a large application/codebase)\n\nNote: With these aiming to stay at the Binary / Bytes IO layer as much as possible (not touch Text I/O for now)","author":{"url":"https://github.com/cmaloney","@type":"Person","name":"cmaloney"},"datePublished":"2025-06-10T05:57:11.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/399/pyperformance/issues/399"}