René's URL Explorer Experiment


Title: stdlib stubs are unnecessarily strict with file-like objects · Issue #4212 · python/typeshed · GitHub

Open Graph Title: stdlib stubs are unnecessarily strict with file-like objects · Issue #4212 · python/typeshed

X Title: stdlib stubs are unnecessarily strict with file-like objects · Issue #4212 · python/typeshed

Description: Problem Currently the IO situation is less than ideal. Not only are IO[str]/TextIO and IO[bytes]/BinaryIO a bit confusing (interchangeable in most cases), but the use of IO through stdlib is inconsistent and doing things like passing an ...

Open Graph Description: Problem Currently the IO situation is less than ideal. Not only are IO[str]/TextIO and IO[bytes]/BinaryIO a bit confusing (interchangeable in most cases), but the use of IO through stdlib is incons...

X Description: Problem Currently the IO situation is less than ideal. Not only are IO[str]/TextIO and IO[bytes]/BinaryIO a bit confusing (interchangeable in most cases), but the use of IO through stdlib is incons...

Opengraph URL: https://github.com/python/typeshed/issues/4212

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"stdlib stubs are unnecessarily strict with file-like objects","articleBody":"## Problem\r\n\r\nCurrently the IO situation is less than ideal. Not only are `IO[str]`/`TextIO` and `IO[bytes]`/`BinaryIO` a bit confusing (interchangeable in most cases), but the use of `IO` through stdlib is inconsistent and doing things like passing an object with a `write()` method to `json.dump()` does not work.\r\n\r\nThis is because the `IO` object, while describing the actual objects returned by `open()` perfectly, is not suitable to represent the \"file-like object\" interface. This interface is well known, documented prominently in the standard library's documentation ([glossary: \"file object\" and \"file-like object\"](https://docs.python.org/3/glossary.html#term-file-object)) and a testament to duck-typing; however it's not compatible with how typeshed is currently written (for the most part).\r\n\r\n## Proposal\r\n\r\nI propose to introduce `Protocol`s (not abstract classes) to be used for parameters where a \"file object\" is expected, allowing one to correctly type their file-like objects without having to inherit one of the abstract base classes. Furthermore, I think we should have two protocols representing files that can be read from or written to.\r\n\r\nThis work can be done incrementally, and I am willing to spend time doing this if there is no veto to this ticket.\r\n\r\n## Pros\r\n\r\nThis would allow a file-like object to be passed to `json.dump()`, `zipfile.ZipFile`, and others (like it already can to `csv.write()`).\r\n\r\nUsing `Protocol`s of this small scale would allow objects that already conform to be used in interfaces expecting file-like object, without having to implement too many methods (or explicitly inherit from the base class, as is required now). This should lower the effort of bringing libraries to the typing world. Using two separate protocols is similar to how most languages do this, off the top of my head:\r\n\r\n* Rust ([`io::Read`](https://doc.rust-lang.org/nightly/std/io/trait.Read.html) and [`io::Write`](https://doc.rust-lang.org/nightly/std/io/trait.Write.html) traits).\r\n* Java (`InputStream` with `read()`, `OutputStream` with `write()` and `flush()`)\r\n* C++ ([`istream`](http://www.cplusplus.com/reference/istream/istream/), [`ostream`](http://www.cplusplus.com/reference/ostream/ostream/))\r\n\r\nIt is interesting to note that the protocols I describe already exist in `typeshed`. Not wanting to put `IO` where the documentation called for `file-like object`, protocols have already been introduced:\r\n\r\n* for `shutil.copyfileobj()`: [`_Reader` and `_Writer` protocols](https://github.com/python/typeshed/blob/2bd06a0a3dd0a37127484e3b543c232cc0fc3577/stdlib/2and3/shutil.pyi#L46-L50)\r\n* for `csv.writer()`: [`_csv._Writer` protocol](https://github.com/python/typeshed/blob/afe1e543b3433fa821500b34f79c620110262f01/stdlib/2and3/_csv.pyi#L44-L45)\r\n\r\nIntroducing those protocols would also allow us to remove some of the `IO[str]`/`TextIO` complexity: while `TextIO` and `BinaryIO` are still needed for the native file objects (they have additional methods compared to `IO`), the protocols used for function parameters everywhere can be only `Read[str]` and `Read[bytes]`.\r\n\r\n## Cons\r\n\r\nThis is a sizeable change, and people are likely to use both the base class and those protocols for some time. However code using the base class should not break when passed to functions expecting the protocol.\r\n\r\nAnother caveat is that this might give a false sense of security: libraries in the wild do their own check to determine if an object conform to the interface, and for example `pandas` will not accept to write on a file object that does implement `__iter__`. Therefore objects conforming to the protocol might still not be accepted by (IMHO buggy) libraries, while inheriting the base class would make their objects look more like file objects (maybe too much, since it gives everything the attributes of both a readable and a writable file!).\r\n\r\n## Draft\r\n\r\nUnfortunately this is where the typeshed-bikeshed starts, but this is my proposal:\r\n\r\n```python\r\nAnyStr = typing.TypeVar('AnyStr', str, bytes)  # typing.AnyStr\r\n\r\nclass WriteIO(Protocol[AnyStr]):\r\n    def write(self, s: AnyStr) -> int: ...\r\n    def flush(self) -> None: ...\r\n    def close(self) -> None: ...\r\n    def __enter__(self) -> 'WriteIO[AnyStr]': ...\r\n    def __exit__(self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException],\r\n                 exc_tb: Optional[TracebackType]) -> Optional[bool]: ...\r\n\r\nclass ReadIO(Protocol[AnyStr]):\r\n    def read(self, size: typing.Optional[int] = None) -> AnyStr: ...\r\n    def close(self) -> None: ...\r\n    def __enter__(self) -> 'WriteIO[AnyStr]': ...\r\n    def __exit__(self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException],\r\n                 exc_tb: Optional[TracebackType]) -> Optional[bool]: ...\r\n    def __iter__(self) -> Iterator[AnyStr]: ...\r\n```\r\n\r\nAdditional protocols can be added to provide `seek()`/`tell()` (similar to Rust's [io::Seek trait](https://doc.rust-lang.org/nightly/std/io/trait.Seek.html))","author":{"url":"https://github.com/remram44","@type":"Person","name":"remram44"},"datePublished":"2020-06-10T02:17:09Z","url":"https://github.com/python/typeshed/issues/4212"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:e97e630f-a6be-6431-4c1a-6515cfd3db8d
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idE616:196792:2ECF0AC:3E125DB:6B002BFD
html-safe-nonce0251c527c39d031185222117c8352fce8bf6a3d7693f6251741b6be328ddafc6
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNjE2OjE5Njc5MjoyRUNGMEFDOjNFMTI1REI6NkIwMDJCRkQiLCJ2aXNpdG9yX2lkIjoiMzA5MTU4MTQyOTY3NTU3MTE3IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac7c7f2a986d0bbb90a456098da70c8d51bfc003989a1a9340b4d8540b01647841
hovercard-subject-tagissue:635874737
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/typeshed/4212/issue_layout
twitter:imagehttps://opengraph.githubassets.com/e9f0d4f35c138fa63544ec85f5e024319d84a821298e0fe5759a5510ac0b5f5d/python/typeshed/issues/4212
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/e9f0d4f35c138fa63544ec85f5e024319d84a821298e0fe5759a5510ac0b5f5d/python/typeshed/issues/4212
og:image:altProblem Currently the IO situation is less than ideal. Not only are IO[str]/TextIO and IO[bytes]/BinaryIO a bit confusing (interchangeable in most cases), but the use of IO through stdlib is incons...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameremram44
hostnamegithub.com
expected-hostnamegithub.com
None7d0829bbde9e48b2486cb5dd6d147896eb01d32134627091f818958a935ef3f0
turbo-cache-controlno-preview
go-importgithub.com/python/typeshed git https://github.com/python/typeshed.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id31696383
octolytics-dimension-repository_nwopython/typeshed
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id31696383
octolytics-dimension-repository_network_root_nwopython/typeshed
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasef3f3a43ca095cdea3abbb394112e971f9fdb94fa
ui-targetcanary-2
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/typeshed/issues/4212#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Ftypeshed%2Fissues%2F4212
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Ftypeshed%2Fissues%2F4212
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Ftypeshed
Reloadhttps://github.com/python/typeshed/issues/4212
Reloadhttps://github.com/python/typeshed/issues/4212
Reloadhttps://github.com/python/typeshed/issues/4212
pythonhttps://github.com/python
typeshedhttps://github.com/python/typeshed
Codehttps://github.com/python/typeshed
Issues196 (196)https://github.com/python/typeshed/issues
Pull requests251 (251)https://github.com/python/typeshed/pulls
Actionshttps://github.com/python/typeshed/actions
Security and qualityhttps://github.com/python/typeshed/security
Insightshttps://github.com/python/typeshed/insights
https://github.com/python/typeshed
New issuehttps://github.com/login?return_to=https://github.com/python/typeshed/issues/4212
New issuehttps://github.com/login?return_to=https://github.com/python/typeshed/issues/4212
stdlib stubs are unnecessarily strict with file-like objectshttps://github.com/python/typeshed/issues/4212#top
topic: ioI/O related issueshttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22topic%3A%20io%22
https://github.com/remram44
remram44https://github.com/remram44
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issue-635874737
glossary: "file object" and "file-like object"https://docs.python.org/3/glossary.html#term-file-object
io::Readhttps://doc.rust-lang.org/nightly/std/io/trait.Read.html
io::Writehttps://doc.rust-lang.org/nightly/std/io/trait.Write.html
istreamhttp://www.cplusplus.com/reference/istream/istream/
ostreamhttp://www.cplusplus.com/reference/ostream/ostream/
_Reader and _Writer protocolshttps://github.com/python/typeshed/blob/2bd06a0a3dd0a37127484e3b543c232cc0fc3577/stdlib/2and3/shutil.pyi#L46-L50
_csv._Writer protocolhttps://github.com/python/typeshed/blob/afe1e543b3433fa821500b34f79c620110262f01/stdlib/2and3/_csv.pyi#L44-L45
io::Seek traithttps://doc.rust-lang.org/nightly/std/io/trait.Seek.html
https://github.com/JelleZijlstra
JelleZijlstrahttps://github.com/JelleZijlstra
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-641688343
@srittauhttps://github.com/srittau
https://github.com/remram44
remram44https://github.com/remram44
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-641698464
https://github.com/JelleZijlstra
JelleZijlstrahttps://github.com/JelleZijlstra
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-641706745
#4161https://github.com/python/typeshed/pull/4161
https://github.com/srittau
srittauhttps://github.com/srittau
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-641817581
https://github.com/remram44
remram44https://github.com/remram44
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-642039146
https://github.com/JelleZijlstra
JelleZijlstrahttps://github.com/JelleZijlstra
on Jun 10, 2020https://github.com/python/typeshed/issues/4212#issuecomment-642039820
https://github.com/ramalho
ramalhohttps://github.com/ramalho
on Jun 14, 2020https://github.com/python/typeshed/issues/4212#issuecomment-643707467
@remram44https://github.com/remram44
io.Readerhttps://golang.org/pkg/io/#Reader
io.Closerhttps://golang.org/pkg/io/#Closer
io.ReadCloserhttps://golang.org/pkg/io/#ReadCloser
io.ReadSeekerhttps://golang.org/pkg/io/#ReadSeeker
io.ReadWriteSeekerhttps://golang.org/pkg/io/#ReadWriteSeeker
@srittauhttps://github.com/srittau
#4174https://github.com/python/typeshed/issues/4174
srittauhttps://github.com/srittau
topic: ioI/O related issueshttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22topic%3A%20io%22
on Jun 14, 2020https://github.com/python/typeshed/issues/4212#event-3441573950
https://github.com/srittau
srittauhttps://github.com/srittau
on Jun 14, 2020https://github.com/python/typeshed/issues/4212#issuecomment-643748473
python/typing#564https://github.com/python/typing/issues/564
https://github.com/srittau
srittauhttps://github.com/srittau
on May 16, 2021https://github.com/python/typeshed/issues/4212#issuecomment-841872398
python/typing#213https://github.com/python/typing/issues/213
https://github.com/srittau
srittauhttps://github.com/srittau
on Aug 11, 2021https://github.com/python/typeshed/issues/4212#issuecomment-896664289
srittauhttps://github.com/srittau
completedhttps://github.com/python/typeshed/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Aug 11, 2021https://github.com/python/typeshed/issues/4212#event-5140788213
https://github.com/twoertwein
twoertweinhttps://github.com/twoertwein
on Nov 17, 2021https://github.com/python/typeshed/issues/4212#issuecomment-971146930
pandas-dev/pandas#43951https://github.com/pandas-dev/pandas/pull/43951
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/typeshed/issues/4212
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/typeshed/issues/4212
topic: ioI/O related issueshttps://github.com/python/typeshed/issues?q=state%3Aopen%20label%3A%22topic%3A%20io%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.