Run a Python script on S3 Files

Question

I want to run a python script on my entire S3 Bucket. The script takes the files and inserts them into a csv file.

how can I run on the S3 files like a local script does? using "python https://s3url/ " doesn't work for me.

Answer 1

You can use boto3 to get the list of all the files in s3 bucket:

import boto3

bucketName = "Your S3 BucketName"

# Create an S3 client
s3 = boto3.client('s3')

for key in s3.list_objects(Bucket=bucketName)['Contents']: 
    print(key['Key'])

Answer 2

A good idea would be to use boto3 . Here is a simple guide on how to use the module