在 python 中作为字节对象传递时，为什么密码不能在 s3 中打开我的 zip 文件？

Question

I have a small but mysterious and unsolvable problem using python to open a password protected file in an AWS S3 bucket.

The password I have been given is definitely correct and I can download the zip to Windows and extract it to reveal the csv data I need. However I need to code up a process to load this data into a database regularly.

The password has a pattern like this (includes mixed case letters, numbers and a single "@"):-

ABCD@Efghi12324567890

The code below works with other zip files I place in the location with the same password:-

import boto3
import pyzipper
from io import BytesIO

s3_resource = boto3.resource('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
zip_obj = s3_resource.Object(bucket_name=my_bucket, key=my_folder + my_zip)
buffer = BytesIO(zip_obj.get()["Body"].read())
z = pyzipper.ZipFile(buffer)
my_newfile=z.namelist()[0]
s3_resource.meta.client.upload_fileobj(
    z.open(my_newfile, pwd=b"ABCD@Efghi12324567890"), #HERE IS THE OPEN COMMAND
    Bucket=my_bucket,
    Key=my_folder + my_newfile)

I am told the password is incorrect:-

RuntimeError: Bad password for file 'ThisIsTheFileName.csv'

I resorted to using pyzipper rather than zipfile, since zipfile didn't support the compression method of the file in question:-

That compression method is not supported

In 7-zip I can see the following for the zip file:-

Method: AES-256 Deflate
Characteristics: WzAES: Encrypt
Host OS: FAT

So to confirm:-
-The password is definitely correct (can open it manually)
-The code seems ok - it opens my zip files with the same password

What is the issue here please and how do I fix it?

You would have my sincere thanks!

Phil

Answer 1

With some help from a colleague and a useful article, I now have this working.

Firstly as per the compression type, I have found it necessary to use the AESZipFile() method of pyzipper (although this method also seemed to work on other compression types).

Secondly the AESZipFile() method apparently accepts a BytesIO object as well as a file path, presumably because this is what it sees when it opens the file. Therefore the zip file can be extracted in situ without having to download it first.

This method creates the pyzipper object which you can then read by specifying the file name and the password.

The final code looks like this:-

import pyzipper
import boto3
from io import BytesIO

my_bucket = ''
my_folder = ''
my_zip = ''
my_password = b''
aws_access_key_id=''
aws_secret_access_key=''

s3 = boto3.client('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
s3_file = s3.get_object(Bucket=my_bucket, Key=my_folder + my_zip)
s3_iodata = BytesIO(s3_file['Body'].read())
f = pyzipper.AESZipFile(s3_iodata)
my_file = f.namelist()[0]
file_content = f.read(my_file, pwd = my_password)
response = s3.put_object(Body=file_content, Bucket=my_bucket, Key=my_folder + my_file)

Here is an article that was useful:-
https://www.linkedin.com/pulse/extract-files-from-zip-archives-in-situ-aws-s3-using-python-tom-reid

I hope this is helpful to someone,

Phil