René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Unexpected traceback output in test_concurrent_futures and crash","articleBody":"# Crash report\n\nWhen I run tests in single-process mode I get unexpected traceback output but tests was marked as passed:\n\n\u003cdetails\u003e\n\n```\n$ ./python -m test -r\n== CPython 3.13.0a0 (heads/regretest-no-remove-test-module:eb7766baa4, Sep 13 2023, 10:58:19) [GCC 11.4.0]\n== Linux-5.15.0-79-generic-x86_64-with-glibc2.35 little-endian\n== Python build: debug\n== cwd: /home/serhiy/py/cpython/build/test_python_worker_3200061æ\n== CPU count: 8\n== encodings: locale=UTF-8, FS=utf-8\nUsing random seed 68644298\n0:00:00 load avg: 0.75 Run 463 tests sequentially\n...\n0:12:03 load avg: 1.72 [191/463] test.test_concurrent_futures.test_process_pool\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\n0:12:21 load avg: 1.52 [192/463] test_codecencodings_tw\n...\n0:25:47 load avg: 1.21 [361/463] test.test_concurrent_futures.test_deadlock\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 201, in send_bytes\n    self._send_bytes(m[offset:offset + size])\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 422, in _send_bytes\n    self._send(buf)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 385, in _send\n    n = write(self._handle, buf)\n        ^^^^^^^^^^^^^^^^^^^^^^^^\nTypeError: 'NoneType' object cannot be interpreted as an integer\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 201, in send_bytes\n    self._send_bytes(m[offset:offset + size])\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 422, in _send_bytes\n    self._send(buf)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 385, in _send\n    n = write(self._handle, buf)\n        ^^^^^^^^^^^^^^^^^^^^^^^^\nTypeError: 'NoneType' object cannot be interpreted as an integer\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 201, in send_bytes\n    self._send_bytes(m[offset:offset + size])\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 422, in _send_bytes\n    self._send(buf)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 385, in _send\n    n = write(self._handle, buf)\n        ^^^^^^^^^^^^^^^^^^^^^^^^\nTypeError: 'NoneType' object cannot be interpreted as an integer\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\n0:26:01 load avg: 1.10 [362/463] test_bigaddrspace\n...\n0:26:20 load avg: 1.08 [368/463] test.test_concurrent_futures.test_init\nTraceback (most recent call last):\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/queues.py\", line 250, in _feed\n    send_bytes(obj)\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 185, in send_bytes\n    self._check_closed()\n  File \"/home/serhiy/py/cpython/Lib/multiprocessing/connection.py\", line 138, in _check_closed\n    raise OSError(\"handle is closed\")\nOSError: handle is closed\n0:26:23 load avg: 1.15 [369/463] test_eof\n...\n```\n\u003c/details\u003e\n\nThen Python crashed after reporting total success:\n```\nTotal duration: 38 min 29 sec\nTotal tests: run=40,827 skipped=1,248\nTotal test files: run=453/463 skipped=10 resource_denied=10\nResult: SUCCESS\nFatal Python error: Segmentation fault\n\nCurrent thread 0x00007fb61c438740 (most recent call first):\n  Garbage-collecting\n  \u003cno Python frame\u003e\nПомилка адресування (збережено знімок оперативної пам’яті)\n```\nThe last line is \"Segfault (core dumped)\" in Ukrainian, but I have no core dump because `ulimit -c` was 0.\n\nI ran tests with a modification of #108952 which does not remove test module or any imported during testing modules before or after running a test.\n\ncc @vstinner\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-109397\n* gh-109780\n* gh-109882\n* gh-111934\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/serhiy-storchaka","@type":"Person","name":"serhiy-storchaka"},"datePublished":"2023-09-13T10:00:46.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/109370/cpython/issues/109370"}