I want to read a bz2 file in a server, decompress it and read with csv parser, but I still have the error;
myfile = bz2.BZ2File(bio.read(), "rb")
TypeError: file() argument 1 must be encoded string without NULL bytes, not str
import paramiko
from config import config
import bz2
import csv
import StringIO
from io import BytesIO
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(config.get('mrc_ssh', 'host'), username=config.get('mrc_ssh', 'user'))
sftp_client = ssh.open_sftp()
_file = sftp_client.open('/home/myfile.bz2')
bio = BytesIO(_file.read())
print bio
myfile = bz2.BZ2File(bio.read(), "rb")
reader = csv.DictReader(myfile)
for row in reader:
print row
bz2.BZ2File
takes a filename as the first argument. Not actual data.
Either use (if you can store locally the file):
myfile = bz2.BZ2File('/home/myfile.bz2', "rb")
Or use the one-shot decompression function bz2.decompress
uncompressed_data = bz2.decompress(bio.read())
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.