René's URL Explorer Experiment


Title: Default "--soft-error-limit" behavior causes tons of headache for mypy newbies with large code bases · Issue #14915 · python/mypy · GitHub

Open Graph Title: Default "--soft-error-limit" behavior causes tons of headache for mypy newbies with large code bases · Issue #14915 · python/mypy

X Title: Default "--soft-error-limit" behavior causes tons of headache for mypy newbies with large code bases · Issue #14915 · python/mypy

Description: Bug Report While not strictly a bug, I would argue that there is a "bug" in the logic behind the decision to add a default --soft-error-limit of 200 (added in #10579). The idea was to make it easier to identify missing stubs as the root ...

Open Graph Description: Bug Report While not strictly a bug, I would argue that there is a "bug" in the logic behind the decision to add a default --soft-error-limit of 200 (added in #10579). The idea was to make it easie...

X Description: Bug Report While not strictly a bug, I would argue that there is a "bug" in the logic behind the decision to add a default --soft-error-limit of 200 (added in #10579). The idea was to mak...

Opengraph URL: https://github.com/python/mypy/issues/14915

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Default \"--soft-error-limit\" behavior causes tons of headache for mypy newbies with large code bases","articleBody":"**Bug Report**\r\n\r\nWhile not strictly a bug, I would argue that there is a \"bug\" in the logic behind the decision to add a default `--soft-error-limit` of `200` (added in https://github.com/python/mypy/pull/10579). The idea was to make it easier to identify missing stubs as the root cause for a flood of errors, especially useful when settings are strict. However, I would assume that strict settings are typically used by `mypy` experts while newbies would typically start with the default settings and gradually work their way up to strict settings (if at all). So while the default values of the strictness settings favor newbies, I think the default behavior of `--soft-error-limits` inconsistently (and unhelpfully) favors experts at the expense of newbies.\r\n\r\nFurthermore, this error limit applies to all errors, regardless of whether they are caused by missing stubs or not. This leads to very confusing error outputs for large code bases that are in the process of being typed by `mypy` newbies. Since Python typing already is a complicated heap of **(insert word here)**, even for devs with  tons of experience in non-typed Python, I would think it is of importance for the `mypy` project to try to lower the threshold for adoption by newbies as much as possible. However, the default setting of `--soft-error-limit` does the opposite: It is counterintuitive that `mypy` by default has an error limit that silences errors.\r\n\r\nThe default behavior is especially confusing when `mypy` is used in the context of an IDE such as PyCharm (also mentioned by @ilevkivskyi in this comment https://github.com/python/mypy/pull/10579#discussion_r645623803). In this case, the error messages are filtered by module name so that the note `(Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)` will appear in a random module and most probably not be read by the user. In my personal experience, trying to debug the discrepancy between the inline `mypy` errors and the lack of the same errors in the global view made me in the end want to pull out my last few remaining strands of hair rather than continue the hunt. This particular discrepancy was one of the worst issues I have ever tried to debug, as there were no patterns to which errors were silenced. Debugging, on the other hand, is typically done by trying to find the pattern that cause the bug to appear:\r\n\r\n> **Example debugging session**: In my particular debugging approach, I tried figuring out why an inline error in a particular module was silenced in a global mypy run by carrying out a binary search of including/excluding modules to locate the exact reason for the silencing, while `grep`-ing the output on the specific module I selected. `grep`-ing of course silenced the real error message (however it did appear in my module of interest once in a while, but with no apparent pattern). In the end I had a set of unrelated modules where, for some unexplained reason, removing any one of them from the `mypy` run would \"un-silence\" the error for the module of interest. In the end, I concluded that there was no pattern, there had to be a limit somewhere. Googling the note text mentioned above also did not directly lead me to the answer, that also took a while. I am sure other approaches to debugging this issue can be equally or more confusing, as exemplified in issues https://github.com/python/mypy/issues/10956 and https://github.com/python/mypy/issues/14389.\r\n\r\n\r\n**To Reproduce**\r\n\r\nAs a `mypy` newbie, run `mypy` on a large code base that have been partially or fully typed, but where detailed fixing of `mypy` errors have been postponed due to the priority of developing actual features. To increase the pain level to the maximum, do this in PyCharm or other IDE where errors are filtered by module. Then:\r\n\r\n1. Be motivated that there are not more than around 200 errors in the whole code base!\r\n2. Start fixing errors in a module of choice\r\n3. Discover that the inline errors from the module do not appear in the global view\r\n4. Try to debug the discrepancy: choose your favorite debugging method and whatever pattern seems most likely to you as the cause\r\n5. Discover that the more you debug, the less sense you are able to make of the issue\r\n6. Start questioning your own sanity!\r\n7. Alternatively, start questioning the sanity of the `mypy` library\r\n8. (If you endure wasting hours of your precious time): Find the actual cause of the bug. Set `--soft-error-limit=-1` and witness the total number of errors multiply\r\n9. Despair at the real number of errors you need to fix!\r\n\r\n**Expected Behavior**\r\n\r\n1. All errors are reported by default\r\n2. (Alternatively) All errors not caused by missing stubs are reported by default\r\n3. If `--soft-error-limit` is set, a note should be added at the __very end__ of the error output that __also__ mentions the actual config/command-line setting to tweak\r\n4. A config setting `soft_error_limit` should be added that corresponds to the command-line flag\r\n5. Log output at `verbosity=1` should, for each module, log whether any errors were silenced due to the setting\r\n\r\n**Actual Behavior**\r\n\r\nNo more than 200 errors are reported (in my case only 139 errors and 41 notes were reported, for some reason). Errors are randomly silenced in global runs for no apparent reason and with no easily noticed error messages. Reading the log at `verbosity=1` gives no clue to the reason why errors are silenced.\r\n\r\n**Your Environment**\r\n\r\n\u003c!-- Include as many relevant details about the environment you experienced the bug in -->\r\n\r\n- Mypy version used: 1.1.1\r\n- Mypy command-line flags: `--show-column-numbers --follow-imports silent` (default for PyCharm `mypy` plugin)\r\n- Mypy configuration options from `mypy.ini` (and other config files): \r\nmypy_path = \"src:tests\"\r\n- Python version used: 3.10\r\n\r\n\u003c!-- You can freely edit this text, please remove all the lines you believe are unnecessary. -->\r\n","author":{"url":"https://github.com/sveinugu","@type":"Person","name":"sveinugu"},"datePublished":"2023-03-17T12:31:48Z","url":"https://github.com/python/mypy/issues/14915"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:0201f7ae-b2a3-5469-265c-c0f7a42b3bf9
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idE95E:DD1C:10EC555:16AAC6B:6AFFC75A
html-safe-nonce3bb8700ad831fcf780f9b8fba90acde8d5e2cda780be90ada2c32f6cf54f6e54
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFOTVFOkREMUM6MTBFQzU1NToxNkFBQzZCOjZBRkZDNzVBIiwidmlzaXRvcl9pZCI6IjUxODEyNzU3ODY5MjUwMzMzMDYiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmace85ddffdd3d2272ff9abfe8e37a04a880f4a51f3d3edaa8dd6ff736c2dc5d1aa
hovercard-subject-tagissue:1629237697
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/14915/issue_layout
twitter:imagehttps://opengraph.githubassets.com/cc75545dccaefa314efccf74e3b5ea76d70704e9a52d7f489afa7e04eb421197/python/mypy/issues/14915
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/cc75545dccaefa314efccf74e3b5ea76d70704e9a52d7f489afa7e04eb421197/python/mypy/issues/14915
og:image:altBug Report While not strictly a bug, I would argue that there is a "bug" in the logic behind the decision to add a default --soft-error-limit of 200 (added in #10579). The idea was to make it easie...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamesveinugu
hostnamegithub.com
expected-hostnamegithub.com
None1e805b832df235bb21305db712c3b6432920019dfc12cb25b9cb8d3b0c9403cf
turbo-cache-controlno-preview
go-importgithub.com/python/mypy git https://github.com/python/mypy.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id7053637
octolytics-dimension-repository_nwopython/mypy
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id7053637
octolytics-dimension-repository_network_root_nwopython/mypy
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8fa4a54e6871c7908eda1c246ad5972982c0b59d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/mypy/issues/14915#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F14915
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fmypy%2Fissues%2F14915
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fmypy
Reloadhttps://github.com/python/mypy/issues/14915
Reloadhttps://github.com/python/mypy/issues/14915
Reloadhttps://github.com/python/mypy/issues/14915
pythonhttps://github.com/python
mypyhttps://github.com/python/mypy
Codehttps://github.com/python/mypy
Issues2.7k (2.7k)https://github.com/python/mypy/issues
Pull requests547 (547)https://github.com/python/mypy/pulls
Actionshttps://github.com/python/mypy/actions
Projectshttps://github.com/python/mypy/projects
Wikihttps://github.com/python/mypy/wiki
Security and qualityhttps://github.com/python/mypy/security
Insightshttps://github.com/python/mypy/insights
https://github.com/python/mypy
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/14915
New issuehttps://github.com/login?return_to=https://github.com/python/mypy/issues/14915
#15138https://github.com/python/mypy/pull/15138
Default "--soft-error-limit" behavior causes tons of headache for mypy newbies with large code baseshttps://github.com/python/mypy/issues/14915#top
#15138https://github.com/python/mypy/pull/15138
bugmypy got something wronghttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22bug%22
good-first-issuehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22good-first-issue%22
topic-configurationConfiguration files and flagshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-configuration%22
topic-usabilityhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-usability%22
https://github.com/sveinugu
sveinuguhttps://github.com/sveinugu
on Mar 17, 2023https://github.com/python/mypy/issues/14915#issue-1629237697
#10579https://github.com/python/mypy/pull/10579
@ilevkivskyihttps://github.com/ilevkivskyi
#10579 (comment)https://github.com/python/mypy/pull/10579#discussion_r645623803
#10956https://github.com/python/mypy/issues/10956
#14389https://github.com/python/mypy/issues/14389
sveinuguhttps://github.com/sveinugu
bugmypy got something wronghttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22bug%22
on Mar 17, 2023https://github.com/python/mypy/issues/14915#event-8777807024
AlexWaygoodhttps://github.com/AlexWaygood
topic-usabilityhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-usability%22
topic-configurationConfiguration files and flagshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-configuration%22
on Mar 17, 2023https://github.com/python/mypy/issues/14915#event-8777895725
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Apr 23, 2023https://github.com/python/mypy/issues/14915#issuecomment-1519074160
JukkaLhttps://github.com/JukkaL
good-first-issuehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22good-first-issue%22
on Apr 23, 2023https://github.com/python/mypy/issues/14915#event-9074958480
hauntsaninjahttps://github.com/hauntsaninja
completedhttps://github.com/python/mypy/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#15138https://github.com/python/mypy/pull/15138
on Apr 26, 2023https://github.com/python/mypy/issues/14915#event-9097643787
hauntsaninjahttps://github.com/hauntsaninja
on Apr 26, 2023https://github.com/python/mypy/issues/14915#event-9097643844
Set soft error limit default to -1 (unlimited) (#15138)https://github.com/python/mypy/commit/bd2a641635d4dd5fe24939287f5be71152d9cd06
#15138https://github.com/python/mypy/pull/15138
https://docs.github.com/articles/closing-issues-via-commit-messages
bd2a641https://github.com/python/mypy/commit/bd2a641635d4dd5fe24939287f5be71152d9cd06
https://github.com/toryth
torythhttps://github.com/toryth
on Apr 28, 2023https://github.com/python/mypy/issues/14915#issuecomment-1527666296
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/mypy/issues/14915
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/mypy/issues/14915
bugmypy got something wronghttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22bug%22
good-first-issuehttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22good-first-issue%22
topic-configurationConfiguration files and flagshttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-configuration%22
topic-usabilityhttps://github.com/python/mypy/issues?q=state%3Aopen%20label%3A%22topic-usability%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.