René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"test_tools: test_freeze_simple_script() times out after 15 min: --with-lto --enable-optimizations (s390x LTO buildbots)","articleBody":"Reproduction:\r\n1. `make clean \u0026\u0026 ./configure --with-lto --enable-optimizations \u0026\u0026 make`\r\n2. `./python.exe -E  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j6 -v test_tools -m test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script`\r\n\r\nEnv:\r\n- Fedora in the test suite: https://buildbot.python.org/all/#/builders/545/builds/3524\r\n- Macos, locally\r\n\r\nTraceback:\r\n\r\n```\r\n» ./python.exe -E  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j6 -v test_tools\r\n/Users/sobolev/Desktop/cpython/python.exe -u -W default -bb -E -E -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j6 -v test_tools\r\n== CPython 3.12.0a6+ (heads/main:1fd603fad2, Mar 27 2023, 09:38:01) [Clang 11.0.0 (clang-1100.0.33.16)]\r\n== macOS-10.14.6-x86_64-i386-64bit little-endian\r\n== Python build: release LTO+PGO\r\n== cwd: /Users/sobolev/Desktop/cpython/build/test_python_89351æ\r\n== CPU count: 4\r\n== encodings: locale=UTF-8, FS=utf-8\r\nUsing random seed 5597523\r\n0:00:00 load avg: 1.77 Run tests in parallel using 6 child processes (timeout: 15 min, worker timeout: 20 min)\r\n0:00:30 load avg: 2.74 running: test_tools (30.0 sec)\r\n0:01:00 load avg: 3.09 running: test_tools (1 min)\r\n0:01:30 load avg: 2.95 running: test_tools (1 min 30 sec)\r\n0:02:00 load avg: 3.15 running: test_tools (2 min)\r\n0:02:30 load avg: 3.15 running: test_tools (2 min 30 sec)\r\n0:03:00 load avg: 3.19 running: test_tools (3 min)\r\n0:03:30 load avg: 4.23 running: test_tools (3 min 30 sec)\r\n0:04:00 load avg: 4.75 running: test_tools (4 min)\r\n0:04:30 load avg: 4.91 running: test_tools (4 min 30 sec)\r\n0:05:00 load avg: 3.85 running: test_tools (5 min)\r\n0:05:30 load avg: 4.70 running: test_tools (5 min 30 sec)\r\n0:06:00 load avg: 4.65 running: test_tools (6 min)\r\n0:06:30 load avg: 3.73 running: test_tools (6 min 30 sec)\r\n0:07:00 load avg: 3.65 running: test_tools (7 min)\r\n0:07:30 load avg: 3.33 running: test_tools (7 min 30 sec)\r\n0:08:00 load avg: 3.28 running: test_tools (8 min)\r\n0:08:30 load avg: 3.01 running: test_tools (8 min 30 sec)\r\n0:09:00 load avg: 3.82 running: test_tools (9 min)\r\n0:09:30 load avg: 4.18 running: test_tools (9 min 30 sec)\r\n0:10:00 load avg: 3.76 running: test_tools (10 min)\r\n0:10:30 load avg: 3.58 running: test_tools (10 min 30 sec)\r\n0:11:00 load avg: 4.73 running: test_tools (11 min)\r\n0:11:30 load avg: 4.45 running: test_tools (11 min 30 sec)\r\n0:12:00 load avg: 4.22 running: test_tools (12 min)\r\n0:12:30 load avg: 3.22 running: test_tools (12 min 30 sec)\r\n0:13:00 load avg: 2.67 running: test_tools (13 min)\r\n0:13:30 load avg: 4.42 running: test_tools (13 min 30 sec)\r\n0:14:00 load avg: 4.41 running: test_tools (14 min)\r\n0:14:30 load avg: 3.99 running: test_tools (14 min 30 sec)\r\n0:15:00 load avg: 5.46 running: test_tools (15 min)\r\n0:15:02 load avg: 5.46 [1/1/1] test_tools crashed (Exit code 1)\r\nTimeout (0:15:00)!\r\nThread 0x0000000113c4d5c0 (most recent call first):\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/selectors.py\", line 415 in select\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 2075 in _communicate\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 1207 in communicate\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 550 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Tools/freeze/test/freeze.py\", line 25 in _run_quiet\r\n  File \"/Users/sobolev/Desktop/cpython/Tools/freeze/test/freeze.py\", line 180 in prepare\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/test_tools/test_freeze.py\", line 27 in test_freeze_simple_script\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 579 in _callTestMethod\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 623 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 678 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/runner.py\", line 208 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py\", line 1106 in _run_suite\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py\", line 1232 in run_unittest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 281 in _test_module\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 317 in _runtest_inner2\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 360 in _runtest_inner\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 219 in _runtest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 265 in runtest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest_mp.py\", line 98 in run_tests_worker\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 732 in _main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 711 in main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 775 in main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/regrtest.py\", line 43 in _main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/regrtest.py\", line 47 in \u003cmodule\u003e\r\n  File \"\u003cfrozen runpy\u003e\", line 88 in _run_code\r\n  File \"\u003cfrozen runpy\u003e\", line 198 in _run_module_as_main\r\n\r\n== Tests result: FAILURE ==\r\n\r\n10 slowest tests:\r\n\r\n1 test failed:\r\n    test_tools\r\n0:15:02 load avg: 5.46\r\n0:15:02 load avg: 5.46 Re-running failed tests in verbose mode\r\n0:15:02 load avg: 5.46 Re-running test_tools in verbose mode\r\ntest_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... Timeout (0:15:00)!\r\nThread 0x000000010b8a55c0 (most recent call first):\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/selectors.py\", line 415 in select\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 2075 in _communicate\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 1207 in communicate\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/subprocess.py\", line 550 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Tools/freeze/test/freeze.py\", line 25 in _run_quiet\r\n  File \"/Users/sobolev/Desktop/cpython/Tools/freeze/test/freeze.py\", line 180 in prepare\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/test_tools/test_freeze.py\", line 27 in test_freeze_simple_script\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 579 in _callTestMethod\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 623 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/case.py\", line 678 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 122 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/suite.py\", line 84 in __call__\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/unittest/runner.py\", line 208 in run\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py\", line 1106 in _run_suite\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py\", line 1232 in run_unittest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 281 in _test_module\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 317 in _runtest_inner2\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 360 in _runtest_inner\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 235 in _runtest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py\", line 265 in runtest\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 353 in rerun_failed_tests\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 756 in _main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 711 in main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/main.py\", line 775 in main\r\n  File \"/Users/sobolev/Desktop/cpython/Lib/test/__main__.py\", line 2 in \u003cmodule\u003e\r\n  File \"\u003cfrozen runpy\u003e\", line 88 in _run_code\r\n  File \"\u003cfrozen runpy\u003e\", line 198 in _run_module_as_main\r\n```\r\n\r\nLooks like timeout is killing this test. I tried to increase it, it helped:\r\n\r\n```\r\n» ./python.exe -E  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=1800 -j2 --junit-xml test-results.xml -j6 -v test_tools -m test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script\r\n/Users/sobolev/Desktop/cpython/python.exe -u -W default -bb -E -E -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=1800 -j2 --junit-xml test-results.xml -j6 -v test_tools -m test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script\r\n== CPython 3.12.0a6+ (heads/main:1fd603fad2, Mar 27 2023, 09:38:01) [Clang 11.0.0 (clang-1100.0.33.16)]\r\n== macOS-10.14.6-x86_64-i386-64bit little-endian\r\n== Python build: release LTO+PGO\r\n== cwd: /Users/sobolev/Desktop/cpython/build/test_python_39615æ\r\n== CPU count: 4\r\n== encodings: locale=UTF-8, FS=utf-8\r\nUsing random seed 1597906\r\n0:00:00 load avg: 2.79 Run tests in parallel using 6 child processes (timeout: 30 min, worker timeout: 35 min)\r\n0:01:00 load avg: 3.25 running: test_tools (1 min)\r\n0:01:30 load avg: 3.33 running: test_tools (1 min 30 sec)\r\n0:02:00 load avg: 3.41 running: test_tools (2 min)\r\n0:02:30 load avg: 3.59 running: test_tools (2 min 30 sec)\r\n0:03:00 load avg: 3.33 running: test_tools (3 min)\r\n0:03:30 load avg: 3.43 running: test_tools (3 min 30 sec)\r\n0:04:00 load avg: 3.08 running: test_tools (4 min)\r\n0:04:30 load avg: 2.80 running: test_tools (4 min 30 sec)\r\n0:05:00 load avg: 5.62 running: test_tools (5 min)\r\n0:05:30 load avg: 5.53 running: test_tools (5 min 30 sec)\r\n0:06:00 load avg: 4.24 running: test_tools (6 min)\r\n0:06:30 load avg: 3.55 running: test_tools (6 min 30 sec)\r\n0:07:00 load avg: 4.57 running: test_tools (7 min)\r\n0:07:30 load avg: 4.09 running: test_tools (7 min 30 sec)\r\n0:08:00 load avg: 3.80 running: test_tools (8 min)\r\n0:08:30 load avg: 4.14 running: test_tools (8 min 30 sec)\r\n0:09:00 load avg: 7.55 running: test_tools (9 min)\r\n0:09:30 load avg: 7.24 running: test_tools (9 min 30 sec)\r\n0:10:00 load avg: 5.84 running: test_tools (10 min)\r\n0:10:30 load avg: 4.38 running: test_tools (10 min 30 sec)\r\n0:11:00 load avg: 4.52 running: test_tools (11 min)\r\n0:11:30 load avg: 4.17 running: test_tools (11 min 30 sec)\r\n0:12:00 load avg: 4.23 running: test_tools (12 min)\r\n0:12:30 load avg: 4.35 running: test_tools (12 min 30 sec)\r\n0:13:00 load avg: 4.11 running: test_tools (13 min)\r\n0:13:30 load avg: 3.37 running: test_tools (13 min 30 sec)\r\n0:14:00 load avg: 3.12 running: test_tools (14 min)\r\n0:14:30 load avg: 3.73 running: test_tools (14 min 30 sec)\r\n0:15:00 load avg: 3.18 running: test_tools (15 min)\r\n0:15:30 load avg: 2.94 running: test_tools (15 min 30 sec)\r\n0:16:00 load avg: 4.23 running: test_tools (16 min)\r\n0:16:30 load avg: 4.90 running: test_tools (16 min 30 sec)\r\n0:17:00 load avg: 4.58 running: test_tools (17 min)\r\n0:17:30 load avg: 4.44 running: test_tools (17 min 30 sec)\r\n0:18:00 load avg: 5.53 running: test_tools (18 min)\r\n0:18:30 load avg: 4.19 running: test_tools (18 min 30 sec)\r\n0:19:00 load avg: 3.39 running: test_tools (19 min)\r\n0:19:30 load avg: 2.78 running: test_tools (19 min 30 sec)\r\n0:19:55 load avg: 2.72 [1/1] test_tools passed (19 min 55 sec)\r\n\r\n== Tests result: SUCCESS ==\r\n\r\n1 test OK.\r\n\r\n10 slowest tests:\r\n- test_tools: 19 min 55 sec\r\n\r\nTotal duration: 19 min 55 sec\r\nTests result: SUCCESS\r\n```\r\n\r\nSo, should be increase this timeout in CI? Or should we try optimizing this test somehow?\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-103170\n* gh-109591\n* gh-109614\n* gh-109616\n* gh-110075\n* gh-110447\n* gh-110448\n* gh-110449\n* gh-110451\n* gh-110453\n* gh-110454\n* gh-110456\n* gh-110457\n* gh-110458\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/sobolevn","@type":"Person","name":"sobolevn"},"datePublished":"2023-03-27T08:08:59.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/103053/cpython/issues/103053"}