René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"GCS Blob does not support the io.IOBase interface","articleBody":"I'm branching this issue from the (currently resolved) issue #2871. The title covers the request, but to add some context see below.\r\n\r\nMany python libraries (pandas, biopython, pillow, etc.) support reading from a 'python file object' which canonically are objects that implement the [io.IOBase](https://docs.python.org/2/library/io.html#io.IOBase) interface. If we could open a blob as a file object, we could do operations as shown in the example below where I'm just reading a few lines from a (possibly very large) file.\r\n\r\n```\r\nfrom Bio import SeqIO\r\nfrom google.cloud.storage import blob\r\n\r\nwith blob.Blob('/data/example.fastq', 'mybucket').open('rb') as fileobj:\r\n    for i, rec in enumerate(SeqIO.parse(fileobj, 'fastq')):\r\n        print(rec.seq)\r\n        print('  name=%s\\n  annotations=%r' % (rec.name, rec.annotations))\r\n        if i \u003e 5:\r\n            break\r\n```\r\n[`google-resumable-media`](https://googlecloudplatform.github.io/google-resumable-media-python/latest/) covers some of the need expressed in this bug, but it does not satisfy users who need blobs to be parsed by libraries expecting standard file objects. The standard advice might be to download files first but that advice ignores the expense and time of downloading large files when indexed operations are available.\r\n\r\nSome work has been done towards this end - the following two repositories have solutions. That said, I think most people would be far more happy to adopt a solution from this project since it has a far larger community and more active maintenance / governance.\r\n - https://github.com/Akrog/gcs-client\r\n - https://github.com/martindurant/gcsfs\r\n","author":{"url":"https://github.com/eap","@type":"Person","name":"eap"},"datePublished":"2017-08-30T21:49:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":7},"url":"https://github.com/3903/google-cloud-python/issues/3903"}