使用nodejs和knox如何查看S3存储桶中的文件

Question

I upload files in many different ways to my s3 bucket.

In python i could use boto like this:

from boto.s3.connection import S3Connection

conn = S3Connection('access-key','secret-access-key')
bucket = conn.get_bucket('bucket')
for key in bucket.list():
    print key.name

In node I have used knox to connect to buckets to get urls, but how could I iterate through the keys in node to see all files in my bucket?

Answer 1

If your buckets get big, best stream those keys! Check out knox-copy :

var knoxCopy = require('knox-copy');

var client = knoxCopy.createClient({
  key: '<api-key-here>',
  secret: '<secret-here>',
  bucket: 'mrbucket'
});

client.streamKeys({
  // omit the prefix to list the whole bucket
  prefix: 'buckets/of/fun' 
}).on('data', function(key) {
  console.log(key);
});

Answer 2

You can do it with AwsSum. It is actively maintained and can perform ALL the S3 operations provided by Amazon.

• https://github.com/appsattic/node-awssum/

There is a fully featured example of exactly what you're looking for in the node-awssum-scripts repo. It gets the first 1000 keys, then keeps doing new requests using the 'marker' parameter to the operation until there are no more keys, so you may want to look at that:

• https://github.com/appsattic/node-awssum-scripts/blob/master/bin/amazon-s3-list.js

If you need any help, give me a shout on GitHub. Disclaimer: I'm chilts, author of Awssum. :)