[英]How do I read a tarfile from a generator?
Create a zip file from a generator in Python? 在Python中用生成器创建一个zip文件? describes a solution for writing a .zip to disk from a bunch of files.
描述了从一堆文件中将.zip写入磁盘的解决方案。
I have a similar problem in the opposite direction. 我在相反的方向上有类似的问题。 I am being given a generator:
我被给了一个发电机:
stream = attachment.iter_bytes()
print type(stream)
and I would love to pipe it to a tar gunzip file-like object: 而且我想将它管道传输到tar gunzip文件对象:
b = io.BytesIO(stream)
f = tarfile.open(mode='r:gz', fileobj = b)
f.list()
But I can't: 但我不能:
<type 'generator'>
Error: 'generator' does not have the buffer interface
I can solve this in the shell like so: 我可以在shell中解决这个问题:
$ curl --options http://URL | tar zxf - ./path/to/interesting_file
How can I do the same in Python under the given conditions? 如何在给定条件下在Python中执行相同的操作?
I had to wrap the generator in a file-like object built on top of the io module. 我不得不将生成器包装在io模块顶部构建的类文件对象中。
def generator_to_stream(generator, buffer_size=io.DEFAULT_BUFFER_SIZE):
class GeneratorStream(io.RawIOBase):
def __init__(self):
self.leftover = None
def readable(self):
return True
def readinto(self, b):
try:
l = len(b) # : We're supposed to return at most this much
chunk = self.leftover or next(generator)
output, self.leftover = chunk[:l], chunk[l:]
b[:len(output)] = output
return len(output)
except StopIteration:
return 0 # : Indicate EOF
return io.BufferedReader(GeneratorStream())
With this, you can open the tar file and extract its content. 有了这个,您可以打开tar文件并提取其内容。
stream = generator_to_stream(any_stream)
tar_file = tarfile.open(fileobj=stream, mode='r|*')
#: Do whatever you want with the tar_file now
for member in tar_file:
member_file = tar_file.extractfile(member)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.