René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Refactor test.libregrtest","articleBody":"I propose to refactor ``test.libregrtest`` to make it easier to maintain and to prepare adding type annotations.\r\n\r\nThe regrtest project has a long history. It was added in 1996 by commit 152494aea24669a3d74460fa460a4ed45696bc75. When it was created, it was 170 lines long and had 4 command line options: ``-v`` (verbose), ``-q`` (quiet), ``-g`` (generate) and ``-x`` (exclude). Slowly, it got more and more features:\r\n\r\n* Better command line interface with ``argparse`` (it used ``getopt`` at the begining)\r\n* Run tests in parallel with multiple processes (this code caused me a lot of headaches!)\r\n* Detect when the \"environment\" is altered: warnings filters, loggers, etc.\r\n* Re-run failed tests in verbose mode (now they are run in fresh processes)\r\n* Detect memory, reference and file descriptor leaks\r\n* Detect leaked files by creating a temporary directory for each test worker process\r\n* Best effort to restore the machine to its previous state: wait until threads and processes complete, remove temporary files, etc.\r\n* etc.\r\n\r\nSome of these features were implemented in ``test.support`` + ``test.libregrtest``.\r\n\r\nA few years ago, I decided to split the giant mono ``regrtest.py`` file (for example, it was 2 200 lines of Python code in Python 2.7) into sub-files (!). To make it possible, I passed ``ns`` argument which is a bag of \"global variables\" (technically, it's a ``Namespace`` class, see ``cmdline.py``).\r\n\r\nThe problem is that for type annotation, it's very unclear what a Namespace contains. It may or may not have arguments (see my commit message of this PR: ``Add missing attributes to Namespace: coverage, threshold, wait.``), argument types are weakly defined, etc. Moreover, ``ns`` is not only used to \"get\" variables, but also to **set** variables! For example, find_tests() overrides ``ns.args``. How is it possible to know which ``ns`` attributes are used? Are they \"read-only\"? We don't know just by reading a function prototype.\r\n\r\nThis large refactoring cleans up everything in a serie of small changes to pass simple types like ``bool``, ``str`` or ``tuple[str]``. It's easier to guess the purpose of a function and its behavior just from its prototype.\r\n\r\nI tried to create only short files, the longest is still sadly ``main.py`` with 891 lines.\r\n\r\n```\r\n$ wc -l *.py|sort -n\r\n     2 __init__.py\r\n    56 pgo.py\r\n   124 win_utils.py\r\n   159 setup.py\r\n   202 refleak.py\r\n   307 utils.py\r\n   329 save_env.py\r\n   451 cmdline.py\r\n   575 runtest.py\r\n   631 runtest_mp.py\r\n   891 main.py\r\n  3727 total\r\n```\r\n\r\nTo understand where the ``ns`` magic bag of global variables, look at ``regrtest.py`` monster in Python 2.7. Its main() functions defines not less than 34 functions inside the main() function! Variables are defined in the main() prototype!\r\n\r\n```\r\ndef main(tests=None, testdir=None, verbose=0, quiet=False,\r\n         exclude=False, single=False, randomize=False, fromfile=None,\r\n         findleaks=False, use_resources=None, trace=False, coverdir='coverage',\r\n         runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,\r\n         random_seed=None, use_mp=None, verbose3=False, forever=False,\r\n         header=False, pgo=False, failfast=False, match_tests=None):\r\n```\r\n\r\nI supposed that it was designed to be able to use regrtest as an API: pass parameters to the main() function, without having to use the command line interface. In the main branch, this feature is still supported, the magic ``**kwargs`` bag:\r\n\r\n```py\r\ndef main(tests=None, **kwargs):\r\n    Regrtest().main(tests=tests, **kwargs)\r\n```\r\n\r\n\u003c!-- gh-linked-prs --\u003e\r\n### Linked PRs\r\n* gh-109163\r\n* gh-109168\r\n* gh-109170\r\n* gh-109171\r\n* gh-109172\r\n* gh-109177\r\n* gh-109202\r\n* gh-109204\r\n* gh-109205\r\n* gh-109206\r\n* gh-109208\r\n* gh-109212\r\n* gh-109229\n* gh-109243\n* gh-109246\n* gh-109248\n* gh-109250\n* gh-109253\n\u003c!-- /gh-linked-prs --\u003e\r\n","author":{"url":"https://github.com/vstinner","@type":"Person","name":"vstinner"},"datePublished":"2023-09-08T21:58:25.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/109162/cpython/issues/109162"}