René's URL Explorer Experiment


Title: Switch to pytest from our homegrown test runner · Issue #1673 · python/mypy · GitHub

Open Graph Title: Switch to pytest from our homegrown test runner · Issue #1673 · python/mypy

X Title: Switch to pytest from our homegrown test runner · Issue #1673 · python/mypy

Description: Ok, so when I saw the PR a while back that "parallelized" the test runner, I was super hyped. But the hype wore off when I realized that the unit tests were still run sequentially. In particular, it continuously bugs me how, when running...

Open Graph Description: Ok, so when I saw the PR a while back that "parallelized" the test runner, I was super hyped. But the hype wore off when I realized that the unit tests were still run sequentially. In particular, i...

X Description: Ok, so when I saw the PR a while back that "parallelized" the test runner, I was super hyped. But the hype wore off when I realized that the unit tests were still run sequentially. In par...

Mail addresses
notifications@github.com

Opengraph URL: https://github.com/python/mypy/issues/1673

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Switch to pytest from our homegrown test runner","articleBody":"Ok, so when I saw the PR a while back that \"parallelized\" the test runner, I was super hyped. But the hype wore off when I realized that the unit tests were still run sequentially.\n\nIn particular, it continuously bugs me how, when running the full unit test suite, there's absolutely no sort of progress indicator of any kind. To make things worse, `runtests.py` is largely undocumented, leaving me to wonder why stuff like `./runtests.py unit-test -a '*something*'` works but not `./runtests.py -a '*something*'`.\n\nIn addition, to an extent, `-a` is slightly...useless. I mean, because the naming convention for tests is so inconsistent, it's rarely ever useful.\n\nAlso, test fixtures are a cruel mess. It took me 30 minutes of debugging to figure out why the tests were giving out obscure `KeyError`s that I couldn't reproduce in my own code. Turns out, the fixture I was using didn't define `list`. But defining `list` caused other tests to brutally fail, so then I just created a new fixture. IMO this whole thing should either be reworked or (better yet) completely gutted and thrown into a fire pit. A very _hot_ fire pit. With sulfur.\n\nSo I see two options:\n- Move to a different test runner. Some have suggested pytest, which I personally _really_ like! Pytest has the [xdist plugin](http://pytest.readthedocs.io/en/latest/xdist.html), which supports running multiple tests in parallel.\n- Revamp the current test runner. Ideally, this would include fixing fixtures (oh, the irony...), improving `runtests.py`, and making the actual unit tests run in parallel, preferably also with a progress bar of some sort.\n\nThoughts? I could try and help out here a bit if you guys come to a consensus of some sort.\n","author":{"url":"https://github.com/refi64","@type":"Person","name":"refi64"},"datePublished":"2016-06-07T20:40:16Z","url":"https://github.com/python/mypy/issues/1673"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:2ccb1343-48f1-8d85-6081-c4e4d1391282
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id83E6:1EF2:35F963:4AE58B:6B000C4E
html-safe-nonceb0fbfe24daae78063fcdd582a53b5fa18398da70917c49cdd2038a670a69c309
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4M0U2OjFFRjI6MzVGOTYzOjRBRTU4Qjo2QjAwMEM0RSIsInZpc2l0b3JfaWQiOiIyMjM4NTU5MTg3MjIxMDg5MzU4IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac0b460eadc33955354130beffad5f8ee0c648177fdf3fc1968656fab8bf00bd5a
hovercard-subject-tagissue:159019215
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/1673/issue_layout
twitter:imagehttps://opengraph.githubassets.com/87933ddf6b90860614014295840cab6fbeaa7e9c58e8f66622f096f4318a9769/python/mypy/issues/1673
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/87933ddf6b90860614014295840cab6fbeaa7e9c58e8f66622f096f4318a9769/python/mypy/issues/1673
og:image:altOk, so when I saw the PR a while back that "parallelized" the test runner, I was super hyped. But the hype wore off when I realized that the unit tests were still run sequentially. In particular, i...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamerefi64
hostnamegithub.com
expected-hostnamegithub.com
None0a0d8686a27478df474dd7415776a7a080cde8838b3cd92ea7601187572e5729
turbo-cache-controlno-preview
go-importgithub.com/python/mypy git https://github.com/python/mypy.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id7053637
octolytics-dimension-repository_nwopython/mypy
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id7053637
octolytics-dimension-repository_network_root_nwopython/mypy
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release16a11879678aaf97e2a42c33ee70830aa084fa37
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/mypy/issues/1673#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F1673
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F1673
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fmypy
Reloadhttps://github.com/python/mypy/issues/1673
Reloadhttps://github.com/python/mypy/issues/1673
Reloadhttps://github.com/python/mypy/issues/1673
pythonhttps://github.com/python
mypyhttps://github.com/python/mypy
Codehttps://github.com/python/mypy
Issues2.7k (2.7k)https://github.com/python/mypy/issues
Pull requests547 (547)https://github.com/python/mypy/pulls
Actionshttps://github.com/python/mypy/actions
Projectshttps://github.com/python/mypy/projects
Wikihttps://github.com/python/mypy/wiki
Security and qualityhttps://github.com/python/mypy/security
Insightshttps://github.com/python/mypy/insights
https://github.com/python/mypy
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/1673
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/1673
Switch to pytest from our homegrown test runnerhttps://github.com/python/mypy/issues/1673#top
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
priority-0-highhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-0-high%22
topic-testshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-tests%22
https://github.com/refi64
refi64https://github.com/refi64
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issue-159019215
xdist pluginhttp://pytest.readthedocs.io/en/latest/xdist.html
Nexthttps://github.com/python/mypy/issues/1673?timeline_page=1
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issuecomment-224432906
#1668https://github.com/python/mypy/issues/1668
gnpricehttps://github.com/gnprice
on Jun 7, 2016https://github.com/python/mypy/issues/1673#event-685055552
https://github.com/ddfisher
ddfisherhttps://github.com/ddfisher
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issuecomment-224433333
https://github.com/gvanrossum
gvanrossumhttps://github.com/gvanrossum
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issuecomment-224434089
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issuecomment-224436946
https://pytest.org/latest/parametrize.htmlhttps://pytest.org/latest/parametrize.html
#1673 (comment)https://github.com/python/mypy/issues/1673#issuecomment-224434089
https://github.com/notifications/unsubscribe/AABuDRvZfrjPF-sEQfTUhsdfDcXEFYvgks5qJfDDgaJpZM4IwVl5https://github.com/notifications/unsubscribe/AABuDRvZfrjPF-sEQfTUhsdfDcXEFYvgks5qJfDDgaJpZM4IwVl5
https://github.com/refi64
refi64https://github.com/refi64
on Jun 7, 2016https://github.com/python/mypy/issues/1673#issuecomment-224439190
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Jun 8, 2016https://github.com/python/mypy/issues/1673#issuecomment-224557124
#380https://github.com/python/mypy/issues/380
JukkaLhttps://github.com/JukkaL
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
on Jun 8, 2016https://github.com/python/mypy/issues/1673#event-685620417
https://github.com/gvanrossum
gvanrossumhttps://github.com/gvanrossum
on Jun 8, 2016https://github.com/python/mypy/issues/1673#issuecomment-224692386
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jun 8, 2016https://github.com/python/mypy/issues/1673#issuecomment-224730933
gnpricehttps://github.com/gnprice
on Jul 27, 2016https://github.com/python/mypy/issues/1673#event-737167930
Set up to use pytest for our data-driven tests, and switch testcheck …https://github.com/python/mypy/commit/255fb6170aba9cb2939f7a65147d1f3b9ef6f1ac
255fb61https://github.com/python/mypy/commit/255fb6170aba9cb2939f7a65147d1f3b9ef6f1ac
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jul 27, 2016https://github.com/python/mypy/issues/1673#issuecomment-235753281
#1944https://github.com/python/mypy/pull/1944
Set up to use pytest for our data-driven tests, and switch testcheck over to it #1944https://github.com/python/mypy/pull/1944
Set up to use pytest for our data-driven tests, and switch testcheck over to it #1944https://github.com/python/mypy/pull/1944
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jul 27, 2016https://github.com/python/mypy/issues/1673#issuecomment-235753539
https://github.com/refi64
refi64https://github.com/refi64
on Jul 27, 2016https://github.com/python/mypy/issues/1673#issuecomment-235755571
@gnpricehttps://github.com/gnprice
https://github.com/gnprice
gnpricehttps://github.com/gnprice
on Jul 28, 2016https://github.com/python/mypy/issues/1673#issuecomment-235772320
@kirbyfan64https://github.com/kirbyfan64
#1723https://github.com/python/mypy/pull/1723
#1944https://github.com/python/mypy/pull/1944
#1723https://github.com/python/mypy/pull/1723
https://github.com/elazarg
elazarghttps://github.com/elazarg
on Aug 26, 2017https://github.com/python/mypy/issues/1673#issuecomment-325162189
fixture filehttps://github.com/elazarg/mypy/blob/pytest/mypy/test/conftest.py
testhttps://github.com/elazarg/mypy/blob/pytest/mypy/test/testtypes.py
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Aug 30, 2017https://github.com/python/mypy/issues/1673#issuecomment-326017599
https://github.com/elazarg
elazarghttps://github.com/elazarg
on Sep 20, 2017https://github.com/python/mypy/issues/1673#issuecomment-330769615
#3880https://github.com/python/mypy/pull/3880
https://github.com/gvanrossum
gvanrossumhttps://github.com/gvanrossum
on Sep 20, 2017https://github.com/python/mypy/issues/1673#issuecomment-330867938
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Sep 20, 2017https://github.com/python/mypy/issues/1673#issuecomment-330885653
https://github.com/emmatyping
emmatypinghttps://github.com/emmatyping
on Dec 15, 2017https://github.com/python/mypy/issues/1673#issuecomment-352115302
#4369https://github.com/python/mypy/pull/4369
JukkaLhttps://github.com/JukkaL
on Feb 6, 2018https://github.com/python/mypy/issues/1673#event-1460864563
Remove myunit (#4369)https://github.com/python/mypy/commit/7ba2df4636c913acf632c098f10aa154ebba6770
#4369https://github.com/python/mypy/pull/4369
7ba2df4https://github.com/python/mypy/commit/7ba2df4636c913acf632c098f10aa154ebba6770
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Feb 6, 2018https://github.com/python/mypy/issues/1673#issuecomment-363535301
@elazarghttps://github.com/elazarg
https://github.com/ilevkivskyi
ilevkivskyihttps://github.com/ilevkivskyi
on May 18, 2018https://github.com/python/mypy/issues/1673#issuecomment-390298941
@elazarghttps://github.com/elazarg
ilevkivskyihttps://github.com/ilevkivskyi
priority-0-highhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-0-high%22
topic-testshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-tests%22
priority-1-normalhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-1-normal%22
on May 18, 2018https://github.com/python/mypy/issues/1673#event-1634927253
gvanrossumhttps://github.com/gvanrossum
on Jun 23, 2018https://github.com/python/mypy/issues/1673#event-1697398960
move test_stubs and test_samples to pytest (#5142)https://github.com/python/mypy/commit/88a6a4ebc26b7ef3fcf04b67a1da25166e4df57c
#5142https://github.com/python/mypy/pull/5142
88a6a4ehttps://github.com/python/mypy/commit/88a6a4ebc26b7ef3fcf04b67a1da25166e4df57c
https://github.com/ilevkivskyi
ilevkivskyihttps://github.com/ilevkivskyi
on Jul 18, 2018https://github.com/python/mypy/issues/1673#issuecomment-406107249
#5274https://github.com/python/mypy/pull/5274
ilevkivskyihttps://github.com/ilevkivskyi
completedhttps://github.com/python/mypy/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Jul 18, 2018https://github.com/python/mypy/issues/1673#event-1741371043
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/mypy/issues/1673
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/mypy/issues/1673
featurehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22feature%22
priority-0-highhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22priority-0-high%22
topic-testshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-tests%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.