S3的Node.js / Awssum

Question

Hi there: I'm wondering if anyone knows a method for listing all files inside of an S3 bucket with node.js and Awssum, starting with the most recent file first.

By default, my code takes the first created file.

function getFileList(callback){
s3.ListObjects(coldBucket, function(err, data) {
    var fileList = getFilenames(data, coldBucketPath);
    callback(fileList);
});
};

Any help is much appreciated! Thanks.

Answer 1

this is Andy, creator of AwsSum.

That's correct, there is no way of asking S3 for the filenames in order of inserted time. However, you can get it using the LastModified.

If you'd like to see an example to get all the keys, see this example here:

• https://github.com/appsattic/node-awssum-scripts/blob/master/bin/amazon-s3-list.js

You could then store each item in an array, then sort it using LastModified. The underscore library contains a function that would be useful for this:

• http://documentcloud.github.com/underscore/#sortBy

Answer 2

I looking at http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGET.html , I don't believe s3 allows you to specify the sort order of the response.

However, you may be able to work around that by changing your file structure to include folders named after the date they were created, then you could do a couple of ListObjects requests with prefixes for recent days/months/etc.

For example, you could have year-month folder names so your file structure looks like this:

bucket_name/
    - 2012-3/
        - file2.jpg
        - file3.jpg
    - 2012-2/
        - file1.jpg

and then before you make your request, do this

var now = new Date();
coldBucket.Prefix = now.getFullYear() + "-" + now.getMonth();

And then if you wanted older stuff you'd have to do a followup request for previous months.

(If you have too many files even for that, you could try year-month-day )