如何使用 Javascript 从 AWS S3 检索多个对象，然后在检索到它们时执行一个函数？

Question

The code below will retrieve at least one object (possibly two) from AWS S3.

I am using the AWS JS SDK and retrieve the objects from within a loop as there is noway to retrieve multiple objects with one request at this moment in time.

After the objects have been retrieved I want to do some image composition (the objects are images).

My problem is that the rest of my code executes before I have successfully retrieved the objects. I know this because objects remains unchanged when logged to the console.

How do I ensure I receive the objects from S3 before I attempt to carry out my other function to manipulate the images?

var app = require('../application');

exports.generate = function (req, res) {

    objects = {
        logo: req.body.logo,
    }

    if (!req.body.background.startsWith('#')) {
        objects.background = req.body.background;
    }

    for (type in objects) {
        var params = {
            Bucket: "my-bucket", 
            Key: objects[type]
        };
        app.s3.getObject(params, function(err, data) {
            if (err) {
                console.log(err, err.stack);
            }
            else {
                objects[type] = data;
            }
        });
    }

    if (objects.background) {
        gm(objects.logo).append(objects.background).write('temp.jpg', function() {
            console.log('Logo and background have been appended');
        });
    }

    console.log(objects);
    console.log('Finished');
}   

The console logs the following

{ logo: 'Original 106fm Logo #268390.jpg', background: 'test.jpg' }
Finished

When the images are retrieved the log should be showing the data body for each image.

Answer 1

Key is to use Promise Objects

app.get('/api/template', (req, res) => {

  let promises = [getS3Object(`templates/x.tpl`),
              getS3Object(`templates/y.tpl`),
              getS3Object(`templates/z.tpl`),
              getS3Object('templates/b.tpl')];

    return Promise.all(promises)
     .then((pres) => {
       const iterable = [0, 1, 2, 3];

       for (let value of iterable) {
        if (!pres[value].statusCode) {
         res.send(pres[value]);
       }
      }
     })
     .catch((res) => {
      console.log(`Error Getting Templates: ${res}`);
   });
});    


const getS3Object = key => {
return new Promise((resolve, reject) => {
  s3.getObject({
      Key: key
  }, (err, data) => {
      if (err){
        resolve(err)
      } else {
        resolve(data.Body)
      }
    })
  })
}

Answer 2

This happens because your console.log calls are executing prior to download files. What you can do is process files after receiving them.

app.s3.getObject(params, function(err, data) {
    if (err) {
      console.log(err, err.stack);
    }
    else {
      objects[type] = data;
      //Here you have the downloaded files. Do your processing here
      console.log(objects);          
    }
});

Asynchronous behavior is what you should read more about.

[Update] you can use following wrapper module to download multiple files

Download multiple files using aws s3