Listing sub folders files path in an s3 bucket

Question

I have an s3 structure as follows:

s3bucketname -> List of first level keys -> List of second level keys -> List of third level keys -> Actual file.

What I need to do is that given the name of the s3bucket and an entry for the first level key and third level I need the path of all file that qualify this condition ie some kind of wildcard for second level keys

Can somebody point out how to do it in java?

Thanks

Answer 1

I would advise you to go through these links: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#listObjects%28com.amazonaws.services.s3.model.ListObjectsRequest%29

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectListing.html#getCommonPrefixes%28%29

Let's say you have a hierarchy like this in your bucket:

"foo/bar/baz/sample1.txt"
"foo/bar/bash/sample2.txt"
"foo/bar/bang/sample3.txt"
"foo/boo/baz/sample4.txt"
"foo/boo/bash/sample6.txt"
"foo/boo/sample5.txt"

And you know about the first-key which is "foo" and the third key say "baz", you can call listObjects() api with a prefix value of "foo/" and a delimiter value of "/", this will give you an ObjectListing with two entries in the common prefixes list that are "foo/bar/" and "foo/boo/". Then you can append the known third key say "bash" to each of these strings and get your all objects using listObjects() api again.

Answer 2

Quite frankly, the easiest way would be to get a list of all objects with the known prefix. Then, have your code look at the Keys that are returned to figure out which ones you want. This would involve fewer calls to AWS.