René's URL Explorer Experiment


Title: Better testing of stubs · Issue #754 · python/typeshed · GitHub

Open Graph Title: Better testing of stubs · Issue #754 · python/typeshed

X Title: Better testing of stubs · Issue #754 · python/typeshed

Description: Reviewing stubs for correctness is hard, and the existing stubs have many bugs (that are gradually being fixed). We could perhaps improve both of these by having better tests for stubs. Here's a concrete idea for that (inspired by a prop...

Open Graph Description: Reviewing stubs for correctness is hard, and the existing stubs have many bugs (that are gradually being fixed). We could perhaps improve both of these by having better tests for stubs. Here's a co...

X Description: Reviewing stubs for correctness is hard, and the existing stubs have many bugs (that are gradually being fixed). We could perhaps improve both of these by having better tests for stubs. Here's ...

Opengraph URL: https://github.com/python/typeshed/issues/754

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Better testing of stubs","articleBody":"Reviewing stubs for correctness is hard, and the existing stubs have many bugs (that are gradually being fixed). We could perhaps improve both of these by having better tests for stubs. Here's a concrete idea for that (inspired by a proposal I remember having seen at Gitter):\r\n\r\n* Write a tester that checks definitions a stub conform to what happens at runtime by importing the target module, parsing the stub and running some checks.\r\n  * We'd find if a stub defines a module-level thing that's not actually there at runtime.\r\n  * We can verify that module attributes have the right types (at least for `int`, `str`, `bool` and other simple types).\r\n  * Check that a class in a stub is a class at runtime (there probably are some exceptions which we can whitelist).\r\n  * Check that a function in a stub is a callable at runtime.\r\n  * Check for the existence of methods.\r\n  * Check that static and class methods are declared as such (assuming that we can reliably introspect this).\r\n  * Check that argument names of functions agree (when we can introspect them).\r\n  * Check that arguments with a `None` default value have type `Optional[x]`.\r\n* For 3rd party modules, `pip install` the package before running the tests. We can have a config file that specifies the versions of 3rd party modules to use to make the tests repeatable.\r\n\r\nWe could blacklist the modules that currently don't pass the test and gradually burn down the blacklist.\r\n\r\nThis wouldn't perfect and several things couldn't be checked automatically:\r\n\r\n* Instance attributes are hard to check for using introspection.\r\n* Things missing from stubs are hard to check for, since it's not clear how to know whether something is an internal thing or an imported things that shouldn't be part of the public interface.\r\n* Types can't generally be tested for correctness, except for some instances of missing `Optional[...]` and maybe a few other things.\r\n* Internal things accidentally included in stubs also probably can't be automatically detected.\r\n\r\nHowever, this could still be pretty valuable, and this shouldn't be too hard to implement. We could start with very rudimentary checks and gradually improve the testing tool as we encounter new bugs in the stubs that we could automatically guard against.","author":{"url":"https://github.com/JukkaL","@type":"Person","name":"JukkaL"},"datePublished":"2016-12-07T14:52:56Z","url":"https://github.com/python/typeshed/issues/754"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:57fa71db-c6d3-9640-01e9-f3ea5f5791e4
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idB52A:3A7693:1FA9A32:2AD58D4:6B00165F
html-safe-nonce428e807b9f48f3871142f0267b0e4998a4d65712dc51847d80b68b5d8b3f9ed6
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNTJBOjNBNzY5MzoxRkE5QTMyOjJBRDU4RDQ6NkIwMDE2NUYiLCJ2aXNpdG9yX2lkIjoiNTI0NDU1ODAxOTc0NzkxMTI2MyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacceaa5c14ae9d8ac0f0495ab1f99b6d71ccaff6cb55d92ed3c2e878ff42197d22
hovercard-subject-tagissue:194069987
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/typeshed/754/issue_layout
twitter:imagehttps://opengraph.githubassets.com/082af062f9d308dc44aa29b7d6876ced9560e2aa714d41abb92c8883a68b1bba/python/typeshed/issues/754
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/082af062f9d308dc44aa29b7d6876ced9560e2aa714d41abb92c8883a68b1bba/python/typeshed/issues/754
og:image:altReviewing stubs for correctness is hard, and the existing stubs have many bugs (that are gradually being fixed). We could perhaps improve both of these by having better tests for stubs. Here's a co...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameJukkaL
hostnamegithub.com
expected-hostnamegithub.com
Nonea2dd2291b1b13cc9c06f27ec92fa3dbaf34116fc24c9347b8539db61884c59f4
turbo-cache-controlno-preview
go-importgithub.com/python/typeshed git https://github.com/python/typeshed.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id31696383
octolytics-dimension-repository_nwopython/typeshed
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id31696383
octolytics-dimension-repository_network_root_nwopython/typeshed
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release70121d4e855ee0bb7b75cf15d061d91f8863c0f8
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/typeshed/issues/754#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Ftypeshed%2Fissues%2F754
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Ftypeshed%2Fissues%2F754
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Ftypeshed
Reloadhttps://github.com/python/typeshed/issues/754
Reloadhttps://github.com/python/typeshed/issues/754
Reloadhttps://github.com/python/typeshed/issues/754
pythonhttps://github.com/python
typeshedhttps://github.com/python/typeshed
Codehttps://github.com/python/typeshed
Issues196 (196)https://github.com/python/typeshed/issues
Pull requests251 (251)https://github.com/python/typeshed/pulls
Actionshttps://github.com/python/typeshed/actions
Security and qualityhttps://github.com/python/typeshed/security
Insightshttps://github.com/python/typeshed/insights
https://github.com/python/typeshed
New issuehttps://github.com/login?return_to=https://github.com/python/typeshed/issues/754
New issuehttps://github.com/login?return_to=https://github.com/python/typeshed/issues/754
Better testing of stubshttps://github.com/python/typeshed/issues/754#top
project: policyOrganization of the typeshed projecthttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22project%3A%20policy%22
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issue-194069987
https://github.com/gvanrossum
gvanrossumhttps://github.com/gvanrossum
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issuecomment-265472781
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issuecomment-265481379
https://github.com/matthiaskramm
matthiaskrammhttps://github.com/matthiaskramm
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issuecomment-265544979
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issuecomment-265579627
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Dec 7, 2016https://github.com/python/typeshed/issues/754#issuecomment-265580033
https://github.com/vlasovskikh
vlasovskikhhttps://github.com/vlasovskikh
on Jan 26, 2017https://github.com/python/typeshed/issues/754#issuecomment-275427340
#862https://github.com/python/typeshed/pull/862
DefinitelyTypedhttps://github.com/DefinitelyTyped/DefinitelyTyped
https://github.com/matthiaskramm
matthiaskrammhttps://github.com/matthiaskramm
on Jan 26, 2017https://github.com/python/typeshed/issues/754#issuecomment-275440871
https://github.com/vlasovskikh
vlasovskikhhttps://github.com/vlasovskikh
on Jan 26, 2017https://github.com/python/typeshed/issues/754#issuecomment-275444975
@matthiaskrammhttps://github.com/matthiaskramm
https://github.com/gvanrossum
gvanrossumhttps://github.com/gvanrossum
on Jan 26, 2017https://github.com/python/typeshed/issues/754#issuecomment-275465696
https://github.com/vlasovskikh
vlasovskikhhttps://github.com/vlasovskikh
on Jan 26, 2017https://github.com/python/typeshed/issues/754#issuecomment-275485413
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Jan 27, 2017https://github.com/python/typeshed/issues/754#issuecomment-275663086
https://github.com/python/mypy/blob/master/test-data/unit/pythoneval.testhttps://github.com/python/mypy/blob/master/test-data/unit/pythoneval.test
https://github.com/vlasovskikh
vlasovskikhhttps://github.com/vlasovskikh
on Jan 27, 2017https://github.com/python/typeshed/issues/754#issuecomment-275725536
https://github.com/JukkaL
JukkaLhttps://github.com/JukkaL
on Jan 27, 2017https://github.com/python/typeshed/issues/754#issuecomment-275749290
https://github.com/vlasovskikh
vlasovskikhhttps://github.com/vlasovskikh
on Feb 5, 2017https://github.com/python/typeshed/issues/754#issuecomment-277548496
#917https://github.com/python/typeshed/pull/917
gvanrossumhttps://github.com/gvanrossum
help wantedhttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22help%20wanted%22
help wantedhttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22help%20wanted%22
on Mar 29, 2017https://github.com/python/typeshed/issues/754#event-1021013184
gvanrossumhttps://github.com/gvanrossum
help wantedhttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22help%20wanted%22
on Apr 30, 2017https://github.com/python/typeshed/issues/754#event-1063484990
srittauhttps://github.com/srittau
project: policyOrganization of the typeshed projecthttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22project%3A%20policy%22
on Sep 12, 2018https://github.com/python/typeshed/issues/754#event-1840045821
https://github.com/hauntsaninja
hauntsaninjahttps://github.com/hauntsaninja
on Jan 21, 2020https://github.com/python/typeshed/issues/754#issuecomment-576530895
https://github.com/hauntsaninja/mypy/tree/stubtest2/scriptshttps://github.com/hauntsaninja/mypy/tree/stubtest2/scripts
@JukkaLhttps://github.com/JukkaL
https://gist.github.com/hauntsaninja/1ecb078ccbd293112b67fb7236727f5dhttps://gist.github.com/hauntsaninja/1ecb078ccbd293112b67fb7236727f5d
https://gist.github.com/hauntsaninja/012ad27b253c042f1144bedbd5985554https://gist.github.com/hauntsaninja/012ad27b253c042f1144bedbd5985554
https://gist.github.com/hauntsaninja/c713dc5e3bf5b419489cca5fd472cd71https://gist.github.com/hauntsaninja/c713dc5e3bf5b419489cca5fd472cd71
https://github.com/python/typeshed/pulls?utf8=✓&q=is%3Apr+author%3Ahauntsaninjahttps://github.com/python/typeshed/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Ahauntsaninja
#996https://github.com/python/typeshed/issues/996
https://github.com/JelleZijlstra
JelleZijlstrahttps://github.com/JelleZijlstra
on Jan 23, 2020https://github.com/python/typeshed/issues/754#issuecomment-577761525
@hauntsaninjahttps://github.com/hauntsaninja
https://github.com/srittau
srittauhttps://github.com/srittau
on Jan 23, 2020https://github.com/python/typeshed/issues/754#issuecomment-577766428
https://github.com/hauntsaninja
hauntsaninjahttps://github.com/hauntsaninja
on Jan 24, 2020https://github.com/python/typeshed/issues/754#issuecomment-578054504
python/mypy#8325https://github.com/python/mypy/pull/8325
https://gist.github.com/hauntsaninja/012ad27b253c042f1144bedbd5985554https://gist.github.com/hauntsaninja/012ad27b253c042f1144bedbd5985554
https://github.com/hauntsaninja
hauntsaninjahttps://github.com/hauntsaninja
on Aug 30, 2020https://github.com/python/typeshed/issues/754#issuecomment-683485804
#1339https://github.com/python/typeshed/issues/1339
hauntsaninjahttps://github.com/hauntsaninja
completedhttps://github.com/python/typeshed/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Aug 30, 2020https://github.com/python/typeshed/issues/754#event-3707721200
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/typeshed/issues/754
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/typeshed/issues/754
project: policyOrganization of the typeshed projecthttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22project%3A%20policy%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.