René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Some CI tests are flaky or real failures","articleBody":"There are some CI tests that seem to be flaky and fail. \n\n1. Failure in ctest.\n```console\nTest project /home/runner/work/lpython/lpython\n    Start 1: test_stacktrace\n1/2 Test #1: test_stacktrace ..................   Passed    0.00 sec\n    Start 2: test_lpython\n2/2 Test #2: test_lpython .....................Subprocess aborted***Exception:   0.86 sec\n[doctest] doctest version is \"2.4.8\"\n[doctest] run with \"--help\" for options\n0 0 0 0 0 0 0 0 0 0 \b\ntest_lpython: /home/runner/micromamba/envs/lp/include/llvm/IR/DataLayout.h:656: uint64_t llvm::StructLayout::getElementOffset(unsigned int) const: Assertion `Idx \u003c NumElements \u0026\u0026 \"Invalid element idx!\"' failed.\n===============================================================================\n/home/runner/work/lpython/lpython/src/lpython/tests/test_llvm.cpp:[15](https://github.com/lcompilers/lpython/actions/runs/13378363981/job/37362241953?pr=2823#step:6:16)56:\nTEST CASE:  PythonCompiler classes\n\n/home/runner/work/lpython/lpython/src/lpython/tests/test_llvm.cpp:1556: FATAL ERROR: test case CRASHED: SIGABRT - Abort (abnormal termination) signal\n\n===============================================================================\n[doctest] test cases:  55 |  54 passed | 1 failed | [17](https://github.com/lcompilers/lpython/actions/runs/13378363981/job/37362241953?pr=2823#step:6:18) skipped\n[doctest] assertions: 486 | 486 passed | 0 failed |\n[doctest] Status: FAILURE!\n\n\n50% tests passed, 1 tests failed out of 2\n\nTotal Test time (real) =   0.87 sec\n\nThe following tests FAILED:\n\t  2 - test_lpython (Subprocess aborted)\nErrors while running CTest\nError: Process completed with exit code 8.\n```\n\n2. Failure in reference tests\n```console\ncompiler_tester.tester.RunException: Testing with reference output failed.\nruntime_errors/test_assert_01.py * run_dbg\nThe JSON metadata differs against reference results\nReference JSON: tests/reference/run_dbg-test_assert_01-2f34744.json\nOutput JSON:    tests/output/run_dbg-test_assert_01-2f34744.json\nOmitting 9 identical items\nDiffering items:\n{'stderr_hash': '32b0a24f111e577fe4fc5b3f4a5994b951e34dde7986b3fb750c5f5e'} != {'stderr_hash': '4811af471c73572b285e9ea01c8689abdd3cb32c717b3cd4876d2669'}\n{'returncode': 134} != {'returncode': 1}\nDiff against: tests/reference/run_dbg-test_assert_01-2f34744.stderr\n1,7c1,2\n\u003c   File \"tests/runtime_errors/test_assert_01.py\", line 1\n\u003c     def test():\n\u003c   File \"tests/runtime_errors/test_assert_01.py\", line 4\n\u003c     test()\n\u003c   File \"tests/runtime_errors/test_assert_01.py\", line 2\n\u003c     assert False\n\u003c AssertionError\n---\n\u003e *** buffer overflow detected ***: terminated\n\u003e Aborted (core dumped)\n\nError: Process completed with exit code 1.\n```","author":{"url":"https://github.com/ubaidsk","@type":"Person","name":"ubaidsk"},"datePublished":"2025-02-17T20:59:03.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/2824/lpython/issues/2824"}