René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Replace module level mutable containers with immutable containers","articleBody":"There are many module level containers with constants. By replacing the mutable containers with immutable variants (e.g. replace a set with a frozenset or a list with a tuple) we improve performance (especially in the free-threaded build, see for example https://github.com/python/cpython/pull/138429) and avoid accidental modification of these containers.\n\nThe number of module level lists, dicts and sets on current main is:\n```\nnumber of mutable module level containers by type:\n\u003cclass 'dict'\u003e: 266\n\u003cclass 'list'\u003e: 150\n\u003cclass 'set'\u003e: 63\n\u003cclass 'collections.defaultdict'\u003e: 2\n\u003cclass '_strptime.TimeRE'\u003e: 1\n\u003cclass 'email._encoded_words._QByteMap'\u003e: 1\n```\n\n\u003cdetails\u003e\u003csummary\u003eScript to list all the module level mutable containers\u003c/summary\u003e\n\n```\nimport sys\nimport importlib\nimport pkgutil\nfrom collections import Counter\n\nblacklist = ('idlelib.idle')\nexcluded_submodule_names = ('__main__')\nsearch_submodules = 2\nmutable_containers = (list, dict,  set)\n\ndef list_container_types(module, mcc):\n    print_module = False\n    for a in dir(module):\n        if a in ('__all__', '__path__', '__builtins__', '__annotations__', '__conditional_annotations__'):\n            # why is __all__ a list and not a tuple?\n            continue\n        # if not a.startswith('_'):\n        #    continue\n        attr = getattr(module, a)\n        tp = type(attr)\n        if issubclass(tp, mutable_containers):\n            if not print_module:\n                print(f'{module}:')\n                print_module = True\n            print(f'  {a}: {tp}')\n            mcc.update([tp])\n\n\ndef search_modules(module_names, search_submodules: int, mcc):\n    for name in module_names:\n        if name in blacklist:\n            continue\n        try:\n            module = importlib.import_module(name)\n        except:\n            print(f'{name}: error on import')\n            module = None\n        list_container_types(module, mcc)\n\n        if search_submodules:\n            try:\n                sub_names = list(z.name for z in pkgutil.iter_modules(module.__path__))\n            except Exception as ex:\n                sub_names = []\n            mm = [name + '.' + sub_name for sub_name in sub_names if sub_name not in excluded_submodule_names]\n            search_modules(mm, search_submodules - 1, mcc)\n\nmcc = Counter()\nmodule_names = sorted(list(sys.builtin_module_names)) + sorted(list(sys.stdlib_module_names))\nsearch_modules(module_names, search_submodules=2, mcc = mcc)\n\nprint()\nprint('number of module level containers by type:')\nfor key, value in mcc.items():\n    print(f'{key}: {value}')\n```\n\n\u003c/details\u003e\n\nNot all the mutable containers can be replaced by immutable containers. Some of them need to be mutable (e.g. `copyreg.dispatch_table`). And some of them are part of the public API and we might not want to change the type only for performance reasons.\n\nExample candidates: `_pydatetime._DAYNAMES` (would improve performance of `date.cdate`), `token.EXACT_TOKEN_TYPES`, `xml.etree.ElementTree.HTML_EMPTY`\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-139004\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/eendebakpt","@type":"Person","name":"eendebakpt"},"datePublished":"2025-09-16T19:58:55.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/139003/cpython/issues/139003"}