René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Improve import time","articleBody":"`importlib_resources` currently takes a while to import. From rough local testing with whatever Python versions I have installed, the figures are something like this:\n\n| OS | Python Implementation | Python Version(s) | Import Time (ms) |\n| ------------- | ------------- | ------------- | ------------- |\n| Windows 10 | CPython | 3.9-3.13 | ~27-37 |\n| Windows 10 | PyPy | 3.10 | ~89 |\n| WSL 2 | CPython | 3.9-3.13 | ~8.5-12.5 |\n\n(The numbers were obtained by cloning the main branch and running `\u003cpython\u003e -m timeit -n 1 -r 1 -- \"import importlib_resources\"` a bunch without a venv, just raw import without site-packages baggage. If need be, I can get more precise results.)\n\nFor context on my perspective, most of the standard library modules that I've interacted with take a fraction of that time.\n\nTo improve this, I've been messing about and managed to get the figures down by at least *7x*:\n\n| OS | Python Implementation | Python Version(s) | Import Time (ms) |\n| ------------- | ------------- | ------------- | ------------- |\n| Windows 10 | CPython | 3.9-3.13 | ~3.5-6.5 |\n| Windows 10 | PyPy | 3.10 | ~5.5 |\n| WSL 2 | CPython | 3.9-3.13 | ~0.6-1.5 |\n\nThis improvement mostly comes from the following:\n1. Replacing the `inspect` usage with a cheaper pattern that other stdlib modules use.\n2. Isolating typing-related symbols and annotations-related symbols so that they are only imported when deferred annotations using them are evaluated (with e.g. `inspect.get_annotations` or `typing.get_type_hints`).\n3. Isolating less-used but very expensive modules/submodules such that they are only imported on demand, i.e. only when deferred annotations containing them are evaluated or functions using them are called.\n4. Consolidating small submodules to avoid reading and compiling several extra files (also, it makes the structure more like `importlib_metadata`'s, imo).\n\nThat's all without a few tricks employed by other standard library modules like `importlib._bootstrap` and `importlib.util`, e.g. avoiding importing `functools` and `contextlib` in exchange for slightly more verbose code. Doing those would make the improvement at least *10x*.\n\nConsidering this is a \"foundational\" library (meant to partially replace the widely used `pkg_resources`) and is also in the standard library, and the potential for improvement is so large with a pretty small amount of effort, I was wondering if there would be interest in improving this, and if so, whether it would be acceptable for me to attempt that by upstreaming some of my work?\n\nP.S. The branch I've been experimenting on is [here](https://github.com/Sachaa-Thanasius/importlib_resources/tree/experiment) if you're curious, though it does too many things (e.g. fixes some bugs, completes the typing, adds unnecessary personalization) for me to consider making a PR with it directly. Makes more sense to do things piecemeal.\n","author":{"url":"https://github.com/Sachaa-Thanasius","@type":"Person","name":"Sachaa-Thanasius"},"datePublished":"2025-03-30T02:53:22.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/326/importlib_resources/issues/326"}