Writing to s3 using imageio and boto3

Question

I want to write images to aws s3. As a video plays i am trying to process images through some functions and then when its done I wish to store it to a specific path. imageio directly checks the extension in the name and writes the image for the appropriate file format.

s3 = boto3.resource('s3')
bucket = s3.Bucket(bucket)
obj = bucket.Object(filepath+'/'+second+'.jpg')
img.imwrite(obj)

If I were to write this to a local location and then write it to s3 then it works but is there a better way where I could store it to s3 without having to write it locally.

Any help is appreciated.

Answer 1

You can use something like BytesIO from python's io package, to create the file object in memory, and pass that to the boto3 client, like this:

from io import BytesIO

s3 = boto3.resource('s3')
bucket = s3.Bucket(bucket)
in_memory_file = BytesIO()
img.imwrite(in_memory_file)
obj = bucket.Object(filepath+'/'+second+'.jpg')
obj.upload_fileobj(in_memory_file)

This should solve the problem, without having to write the file to disk.