René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Socket leak when opening, and closing sockets","articleBody":"Even though https://github.com/jstasiak/python-zeroconf/issues/171 and the subsequent fix in PR #188/270 addressed one class of socket leaks, we’re still seeing file-descriptor leaks when creating and tearing down Zeroconf instances in rapid succession (e.g. one per “cycle”). Over time, the process exhausts its FD limit with errors like OSError: [Errno 24] Too many open files.\nWhat’s happening\n\nPersistent sockets remain open\nEvery time Zeroconf(interfaces=[…]) is called, it creates a set of listening and responding sockets. Although those sockets are closed (via .close()) when the instance is torn down, some underlying file descriptors remain open in the OS.\n\nLingering asyncio selectors and threads\nThe library spins up an asyncio event loop per instance. Even after .close(), we observe threads (in selector_events.py and _core.py) that never fully tear down, holding onto their self-pipes or sockets.\n\nifaddr adapter enumeration leak\nIn our environment, each new Zeroconf triggers an ifaddr.get_adapters() call—which under certain Linux distributions seems to leave open netlink sockets or file handles that are never garbage-collected.\n\n\nReproduction steps\n\nWrite a simple loop that does:\n\n```python\nfrom zeroconf import Zeroconf\nimport time\nimport os\n\n# Run a tight loop creating \u0026 closing Zeroconf instances\nfor i in range(1, 1001):\n    zc = Zeroconf(interfaces=['127.0.0.1'])\n    zc.close()\n    if i % 100 == 0:\n        print(f\"Iteration {i}\")\n        time.sleep(0.1)  # give the OS a moment\n\ndef fd_count():\n    return len(os.listdir(f\"/proc/{os.getpid()}/fd\"))\n\nfor i in range(1,1001):\n    zc = Zeroconf(interfaces=['127.0.0.1'])\n    zc.close()\n    if i % 100 == 0:\n        print(f\"Iteration {i}, open-fds: {fd_count()}\")\n```\n\n\nExpected behavior\n\nAfter zc.close(), all sockets, event loops, and threads associated with that Zeroconf instance should be fully torn down, and the OS file descriptor count should return to its prior level.\n\n\nEnvironment\n\npython-zeroconf version: ≥ 0.28.0 (includes the fixes from PR #188/270)\nOS: Ubuntu 24.04, Linux kernel 6.x\nPython: 3.12\n\n\nSuggested investigation areas\n\n- Verify that every internal selector and asyncio loop is explicitly closed and that its underlying sockets/pipes are closed too.\n- Audit ifaddr.get_adapters() and its use of netlink sockets—ensure that any sockets opened for adapter enumeration are closed.\n- Add a regression test that repeatedly constructs and closes Zeroconf objects and asserts that FD count remains constant.\n\n\n\nI'm trying to write a unidirectional mDNS reflector solution, it works flawlessly if I start, and then stop the main reflector script rather than simply closing the sockets, but this requires a wrapper script to then handle this with added complexity along with added noise \u0026 advertisements appearing, then disappearing when the reflector script exits.\n\n\nUnfortunately, I don't know enough to patch this, and implement a fix effectively, I've been fault finding with the use of AI to be able to get this write up where it is so there could be some incorrect information, but running the code above I can re-produce the issue on different systems.\n\nIf you need any help testing, I'd be happy to assist where I can","author":{"url":"https://github.com/brayden-b-LA","@type":"Person","name":"brayden-b-LA"},"datePublished":"2025-05-25T09:42:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/1589/python-zeroconf/issues/1589"}