Uploading multiple files to Amazon S3

Question

I need help on how to upload multiple files on Amazon S3. So basically I have three input fields for files upload, two inputs will take 10-20 pictures and last input is only one for one image and upload them to Amazon S3 when a form is submitted.

The form that I'm using for uploading images:

I have a bucket and everything, what I need is some kind of solution to upload multiple images to Amazon S3.

I'm using PHP as my backend and for now, images are stored on hosting when a form is submitted. But I will have more then 150gb of images uploaded every month and I need S3 for hosting those images.

When I connect the form with Amazon S3 and try to upload more than one image, I get this message "POST requires exactly one file upload per request.".

Answer 1

Here is the NodeJS code which will give you an idea on how to upload all the files and then send a response back to the UI when the upload is complete.

I am using promises and the promise.all() method which will resolve all promises.

I am also using multer for Node.JS which handles the files that I received from the UI.

app.post('/uploadMultipleFiles',upload.array('file', 10),function(req,res){
        var promises=[];
        for(var i=0;i<req.files.length;i++){
            var file = req.files[i];
            promises.push(uploadLoadToS3(file));
        }
        Promise.all(promises).then(function(data){
            res.send('Uploadedd');
        }).catch(function(err){
            res.send(err.stack);
        }) 
    })

    function uploadLoadToS3(ObjFile){

        var params={
            ACL :'public-read',
            Body : new Buffer(ObjFile.buffer),
            Bucket:'ascendon1',
            ContentType:ObjFile.mimetype,
            Key:ObjFile.originalname
        }
        return s3.upload(params).promise();
    }

Answer 2

S3 is highly scalable and distributed storage.

If you have those images locally in your machine you can simply use

aws s3 sync local_folder s3://bucket_name/

https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

cli takes cares of syncing the data.

You can also configure how much parallelism you want on the cli with the configuration settings.

https://docs.aws.amazon.com/cli/latest/topic/s3-config.html

You can also do this programmatically if that is going to be a continuous data movement.

EDIT1:

Only one file can be uploaded from the UI at one time.

You can sequence them via javascript and upload one at a time.

If you want to take it to the backend you can do so,

https://w3lessons.info/2013/09/06/jquery-multiple-file-upload-to-amazon-s3-using-php/

Hope it helps.