René's URL Explorer Experiment


Title: Regression tests do not support exclusion and pgo in the same invocation · Issue #135494 · python/cpython · GitHub

Open Graph Title: Regression tests do not support exclusion and pgo in the same invocation · Issue #135494 · python/cpython

X Title: Regression tests do not support exclusion and pgo in the same invocation · Issue #135494 · python/cpython

Description: Bug report Bug description: Background I've been working to build from source on Alpine. Starting in 3.13 pgo optimizations now error and that led to the test_re tests failing and blocking builds. I didn't want to fully disable pgo optim...

Open Graph Description: Bug report Bug description: Background I've been working to build from source on Alpine. Starting in 3.13 pgo optimizations now error and that led to the test_re tests failing and blocking builds. ...

X Description: Bug report Bug description: Background I've been working to build from source on Alpine. Starting in 3.13 pgo optimizations now error and that led to the test_re tests failing and blocking buil...

Opengraph URL: https://github.com/python/cpython/issues/135494

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Regression tests do not support exclusion and pgo in the same invocation","articleBody":"# Bug report\n\n### Bug description:\n\n## Background\n\nI've been working to build from source on Alpine.  Starting in 3.13 pgo optimizations now error and that led to the test_re tests failing and blocking builds.  I didn't want to fully disable pgo optimizations and miss out on all of the related performance wins.\n\n## Attempted Fix and Problem Identification\n\nI initially attempted to preclude this by leveraging `PROFILE_TASK=\"-m test --pgo -x test_re\"` when calling make.  This had the impact of making all tests *except* the pgo tests run.  After some spelunking I found that this logic from  `Lib/test/libregrtest/main.py` is the culprit.\n\n```python\ndef find_tests(self, tests: TestList | None = None) -> tuple[TestTuple, TestList | None]:\n        if tests is None:\n            tests = []\n        if self.single_test_run:\n            self.next_single_filename = os.path.join(self.tmp_dir, 'pynexttest')\n            try:\n                with open(self.next_single_filename, 'r') as fp:\n                    next_test = fp.read().strip()\n                    tests = [next_test]\n            except OSError:\n                pass\n\n        if self.fromfile:\n            tests = []\n            # regex to match 'test_builtin' in line:\n            # '0:00:00 [  4/400] test_builtin -- test_dict took 1 sec'\n            regex = re.compile(r'\\btest_[a-zA-Z0-9_]+\\b')\n            with open(os.path.join(os_helper.SAVEDCWD, self.fromfile)) as fp:\n                for line in fp:\n                    line = line.split('#', 1)[0]\n                    line = line.strip()\n                    match = regex.search(line)\n                    if match is not None:\n                        tests.append(match.group())\n\n        strip_py_suffix(tests)\n\n        if self.pgo:\n            # add default PGO tests if no tests are specified\n            setup_pgo_tests(self.cmdline_args, self.pgo_extended)\n\n        if self.tsan:\n            setup_tsan_tests(self.cmdline_args)\n\n        if self.tsan_parallel:\n            setup_tsan_parallel_tests(self.cmdline_args)\n\n        exclude_tests = set()\n        if self.exclude:\n            for arg in self.cmdline_args:\n                exclude_tests.add(arg)\n            self.cmdline_args = []\n\n        alltests = findtests(testdir=self.test_dir,\n                             exclude=exclude_tests)\n\n        if not self.fromfile:\n            selected = tests or self.cmdline_args\n            if selected:\n                selected = split_test_packages(selected)\n            else:\n                selected = alltests\n        else:\n            selected = tests\n\n        if self.single_test_run:\n            selected = selected[:1]\n            try:\n                pos = alltests.index(selected[0])\n                self.next_single_test = alltests[pos + 1]\n            except IndexError:\n                pass\n\n        # Remove all the selected tests that precede start if it's set.\n        if self.starting_test:\n            try:\n                del selected[:selected.index(self.starting_test)]\n            except ValueError:\n                print(f\"Cannot find starting test: {self.starting_test}\")\n                sys.exit(1)\n\n        random.seed(self.random_seed)\n        if self.randomize:\n            random.shuffle(selected)\n\n        for priority_test in reversed(self.prioritize_tests):\n            try:\n                selected.remove(priority_test)\n            except ValueError:\n                print(f\"warning: --prioritize={priority_test} used\"\n                        f\" but test not actually selected\")\n                continue\n            else:\n                selected.insert(0, priority_test)\n\n        return (tuple(selected), tests)\n```\n\nDue to the order of operations what ends up happening is the pgo test suite gets added to `cmdline_args` and then they all get excluded leaving everything else to run.\n\n## Hacky Workaround\n\nMy eventual workaround is... less than great.  I ultimately settled on two choices: explicitly maintain my own copy of the PGO test list in my dockerfile or update the list defined in the testing framework.  I opted for the later and have added this to my build script.\n\n```bash\nsed -Ei \"s/'test_re',?$//g\"  Lib/test/libregrtest/pgo.py\n```\n\nThis felt like the less likely option to break over time as we not only build 3.13 but other versions as well.\n\n## Suggested Improvement Options\n\n**1. Automatic Exclusion for Musl:**  \nAutomatically exclude incompatible tests on musl-based systems, which appears consistent with other test behaviors.  This minimizes friction for maintainers who otherwise may forgo PGO optimizations.\n\nFrom my view, this is the preferred option.  \n\n**2. Improved Test Exclusion Logic:**\nAdjust `--pgo` and `-x` flag logic to allow intuitive exclusion of PGO tests, or otherwise enable this behavior, and combine with clearer documentation for builders.\n\n**3. Explicit Error Messaging:**\nExplicitly error when these flags are used together to reduce confusion and accidental behaviors.  This aligns with other checks for mutually exclusive flags. \n\n\n### CPython versions tested on:\n\n3.13\n\n### Operating systems tested on:\n\nOther\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-135713\n* gh-135880\n* gh-135881\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/Kaelten","@type":"Person","name":"Kaelten"},"datePublished":"2025-06-13T22:52:58Z","url":"https://github.com/python/cpython/issues/135494"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:a14ee9fa-9693-afe9-b586-8ad33846b933
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idE324:19AA62:44E7059:5748C3F:6B023325
html-safe-nonce0e1f466e2d260ed7bee90102378ce6faa0bfe8de33def1d2a4eec7633ad75b03
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFMzI0OjE5QUE2Mjo0NEU3MDU5OjU3NDhDM0Y6NkIwMjMzMjUiLCJ2aXNpdG9yX2lkIjoiODA5OTYzMzU2OTgwNDIwMjc4OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac16d886d259481e414c4858f8ddb96f52189c5306be7b0f29c45bed62fc980eae
hovercard-subject-tagissue:3145003114
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/135494/issue_layout
twitter:imagehttps://opengraph.githubassets.com/f1e7f11d4a89b45cca175e6f0f981fede234d09cd1dfeae5a7ccf85c61e435e1/python/cpython/issues/135494
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/f1e7f11d4a89b45cca175e6f0f981fede234d09cd1dfeae5a7ccf85c61e435e1/python/cpython/issues/135494
og:image:altBug report Bug description: Background I've been working to build from source on Alpine. Starting in 3.13 pgo optimizations now error and that led to the test_re tests failing and blocking builds. ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameKaelten
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/cpython/issues/135494#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F135494
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F135494
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/issues/135494
Reloadhttps://github.com/python/cpython/issues/135494
Reloadhttps://github.com/python/cpython/issues/135494
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
https://github.com/python/cpython
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/135494
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/135494
Regression tests do not support exclusion and pgo in the same invocationhttps://github.com/python/cpython/issues/135494#top
testsTests in the Lib/test dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22tests%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
https://github.com/Kaelten
Kaeltenhttps://github.com/Kaelten
on Jun 13, 2025https://github.com/python/cpython/issues/135494#issue-3145003114
gh-135494: Fix python -m test --pgo -x test_re #135713https://github.com/python/cpython/pull/135713
[3.14] gh-135494: Fix python -m test --pgo -x test_re (GH-135713) #135880https://github.com/python/cpython/pull/135880
[3.13] gh-135494: Fix python -m test --pgo -x test_re (#135713) #135881https://github.com/python/cpython/pull/135881
Kaeltenhttps://github.com/Kaelten
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
on Jun 13, 2025https://github.com/python/cpython/issues/135494#event-18142659082
picnixzhttps://github.com/picnixz
testsTests in the Lib/test dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22tests%22
on Jun 13, 2025https://github.com/python/cpython/issues/135494#event-18142690414
https://github.com/vstinner
vstinnerhttps://github.com/vstinner
on Jun 19, 2025https://github.com/python/cpython/issues/135494#issuecomment-2988060788
test_re fails on Alpine Linux (with musl) #131342https://github.com/python/cpython/issues/131342
6146295https://github.com/python/cpython/commit/6146295a5b8e9286ccb8f90818b764c9a0192090
vstinnerhttps://github.com/vstinner
on Jun 19, 2025https://github.com/python/cpython/issues/135494#event-18230770023
pythongh-135494: Fix python -m test --pgo -x test_rehttps://github.com/vstinner/cpython/commit/41136a1c7ad4b6ec81671eb6c9c3adc36a36cf0a
pythongh-135494https://github.com/python/cpython/issues/135494
41136a1https://github.com/vstinner/cpython/commit/41136a1c7ad4b6ec81671eb6c9c3adc36a36cf0a
https://github.com/vstinner
vstinnerhttps://github.com/vstinner
on Jun 19, 2025https://github.com/python/cpython/issues/135494#issuecomment-2988113753
#135713https://github.com/python/cpython/pull/135713
vstinnerhttps://github.com/vstinner
on Jun 24, 2025https://github.com/python/cpython/issues/135494#event-18291470295
gh-135494: Fix python -m test --pgo -x test_re (#135713)https://github.com/python/cpython/commit/15c6d63fe6fc62c6d78d2fad81965a8e6f7b7b98
gh-135494https://github.com/python/cpython/issues/135494
#135713https://github.com/python/cpython/pull/135713
15c6d63https://github.com/python/cpython/commit/15c6d63fe6fc62c6d78d2fad81965a8e6f7b7b98
miss-islingtonhttps://github.com/miss-islington
on Jun 24, 2025https://github.com/python/cpython/issues/135494#event-18291472299
pythongh-135494: Fix python -m test --pgo -x test_re (pythonGH-135713)https://github.com/miss-islington/cpython/commit/417dabb11c984ff7e043c3176eb0c55fbf899d80
pythongh-135494https://github.com/python/cpython/issues/135494
pythonGH-135713https://github.com/python/cpython/pull/135713
417dabbhttps://github.com/miss-islington/cpython/commit/417dabb11c984ff7e043c3176eb0c55fbf899d80
vstinnerhttps://github.com/vstinner
on Jun 24, 2025https://github.com/python/cpython/issues/135494#event-18291541196
pythongh-135494: Fix python -m test --pgo -x test_re (python#135713)https://github.com/vstinner/cpython/commit/02f452c40492a37c7ea4b951cea438923c981b36
pythongh-135494https://github.com/python/cpython/issues/135494
python#135713https://github.com/python/cpython/pull/135713
02f452chttps://github.com/vstinner/cpython/commit/02f452c40492a37c7ea4b951cea438923c981b36
[3.14] gh-135494: Fix python -m test --pgo -x test_re (GH-135713) (#1…https://github.com/python/cpython/commit/961dd80e0ee7e068132a196d8afe7a4dd8e16dbf
gh-135494https://github.com/python/cpython/issues/135494
GH-135713https://github.com/python/cpython/pull/135713
#1…https://github.com/python/cpython/pull/135880
961dd80https://github.com/python/cpython/commit/961dd80e0ee7e068132a196d8afe7a4dd8e16dbf
[3.13] gh-135494: Fix python -m test --pgo -x test_re (#135713) (#135881https://github.com/python/cpython/commit/a27398d5b3ca5c59ac52cd72cf6878bbaca49c80
gh-135494https://github.com/python/cpython/issues/135494
#135713https://github.com/python/cpython/pull/135713
#135881https://github.com/python/cpython/pull/135881
a27398dhttps://github.com/python/cpython/commit/a27398d5b3ca5c59ac52cd72cf6878bbaca49c80
AndPuQinghttps://github.com/AndPuQing
on Jul 11, 2025https://github.com/python/cpython/issues/135494#event-18584719740
pythongh-135494: Fix python -m test --pgo -x test_re (python#135713)https://github.com/AndPuQing/cpython/commit/576fe7ae837c92e1fc01459cee8f62e66355b9b5
pythongh-135494https://github.com/python/cpython/issues/135494
python#135713https://github.com/python/cpython/pull/135713
576fe7ahttps://github.com/AndPuQing/cpython/commit/576fe7ae837c92e1fc01459cee8f62e66355b9b5
Pranjal095https://github.com/Pranjal095
on Jul 12, 2025https://github.com/python/cpython/issues/135494#event-18600181782
pythongh-135494: Fix python -m test --pgo -x test_re (python#135713)https://github.com/Pranjal095/cpython/commit/7fc55b7d49d0cf3d48935db62bc9321cb89a0125
pythongh-135494https://github.com/python/cpython/issues/135494
python#135713https://github.com/python/cpython/pull/135713
7fc55b7https://github.com/Pranjal095/cpython/commit/7fc55b7d49d0cf3d48935db62bc9321cb89a0125
taegyunkimhttps://github.com/taegyunkim
on Aug 4, 2025https://github.com/python/cpython/issues/135494#event-18976841764
pythongh-135494: Fix python -m test --pgo -x test_re (python#135713)https://github.com/taegyunkim/cpython/commit/b2951fd99cae4b19582b7e537c0666d418311e8e
pythongh-135494https://github.com/python/cpython/issues/135494
python#135713https://github.com/python/cpython/pull/135713
b2951fdhttps://github.com/taegyunkim/cpython/commit/b2951fd99cae4b19582b7e537c0666d418311e8e
https://github.com/hugovk
hugovkhttps://github.com/hugovk
on Aug 8, 2025https://github.com/python/cpython/issues/135494#issuecomment-3168500147
hugovkhttps://github.com/hugovk
completedhttps://github.com/python/cpython/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Aug 8, 2025https://github.com/python/cpython/issues/135494#event-19058354367
Agent-Hellboyhttps://github.com/Agent-Hellboy
on Aug 19, 2025https://github.com/python/cpython/issues/135494#event-19221801932
pythongh-135494: Fix python -m test --pgo -x test_re (python#135713)https://github.com/Agent-Hellboy/cpython/commit/f3cd82a4ea4f1d5e3c30daac0f2296b02c3a21fe
pythongh-135494https://github.com/python/cpython/issues/135494
python#135713https://github.com/python/cpython/pull/135713
f3cd82ahttps://github.com/Agent-Hellboy/cpython/commit/f3cd82a4ea4f1d5e3c30daac0f2296b02c3a21fe
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/cpython/issues/135494
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/cpython/issues/135494
testsTests in the Lib/test dirhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22tests%22
type-bugAn unexpected behavior, bug, or errorhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-bug%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.