René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"libregrtest: add option to run every test case in separate process","articleBody":"# Feature or enhancement\n\n### Proposal:\n\nlibregrtest can run every test module in a separate process, and can even run several test modules in parallel. It has options to simply output all test modules or all test cases.\n\nI propose to add an option to run every test case in a separate process. Test cases from the same module should still run sequentially, because they can use the same resource (for example create a server at fixed port).\n\nThis will slow down the tests several times, but it is not for regular running in the CI, it is for manual running once every months or for manual running selected test module while writing new tests or debugging test flakiness. It can help to solve the following issues:\n\n1. Order dependency. Some tests are only passed because they run in specific order, and the order of tests in a module is the same, only the order of test modules is randomized. If run the test in a separate process, it is always the first, and cannot depend on others.\n2. Environment changes and leaks. Some tests change the environment (e.g. leak temporary files or do not wait for terminating a thread or a subprocess), and it is not noticed because subsequent tests have cleaned after it. If run the test in a separate process, it is always the last, and nobody cleans after it.\n\ncc @vstinner @gpshead\n\n### Has this already been discussed elsewhere?\n\nNo response given\n\n### Links to previous discussion of this feature:\n\n_No response_","author":{"url":"https://github.com/serhiy-storchaka","@type":"Person","name":"serhiy-storchaka"},"datePublished":"2023-09-25T06:55:49.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/109817/cpython/issues/109817"}