将 S3 存储桶中的 gzip 文件提取到另一个 S3 存储桶

Question

im trying to copy the gzip file from one S3 bucket and extract its content to another S3 bucket using gzip library. im getting an error

Seek from end not supported

import boto3, json
from io import BytesIO
import gzip

def lambda_handler():
    try:
        s3 = boto3.resource('s3')
        copy_source = {
            'Bucket': 'srcbucket',
            'Key': 'samp.gz'
            }
        bucket = s3.Bucket('destbucket')
        bucketSrc = s3.Bucket('srcbucket')

        s3Client = boto3.client('s3', use_ssl=False)

        s3Client.upload_fileobj(                      # upload a new obj to s3
            Fileobj=gzip.GzipFile(              # read in the output of gzip -d
                None,                           # just return output as BytesIO
                'rb',                           # read binary
                fileobj=BytesIO(s3Client.get_object(Bucket='srcbucket', Key='samp.gz')['Body'].read())),
            Bucket='destbucket',                      # target bucket, writing to
            Key="")               # target key, writing to

    except Exception as e:
        print(e)

Answer 1

You can't unzip the ZIP file and upload its constituent files the way you're trying to.

You could unzip the entire ZIP file to Lambda local disk in /tmp (note this has a limit of 512MB diskspace) then upload file by file. Or, if it will not fit on disk or you prefer not to persist to desk, then you can stream the contents of the ZIP file into memory, file by file, and then upload each stream to S3). In both solutions, you will need to supply an appropriate key for each and every upload.