简体   繁体   中英

Read multiple gzip files to 1 fileobject in python

i want to read multiple gzip file to 1 file object currently i am doing

import gzip 
a = gzip.open(path2zipfile1) 
for line in a.readline()
  #do some stuff

but i need to read from say 2 files

a = gzip.open(path2zipfile1)  #read zip1
a = gzip.open(path2zipfile2, 'rU') #appending file object with contents of 2nd file 
for line in a.readlines()
  #this should give me contents from zip1 then zip2

unable to find the right mode to do so

use itertools.chain :

import itertools, gzip

files = ['path2zipfile1', 'path2zipfile2']
it = (gzip.open(f, 'rt') for f in files)

for line in itertools.chain.from_iterable(it):
    print(line)

another version without itertools :

def gen(files):
    for f in files:
        fo = gzip.open(f, 'rt')
        while True:
            line = fo.readline()
            if not line:
                break
            yield line

files = ['path2zipfile1', 'path2zipfile2']
for line in gen(files):
    print(line)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM