Amazon S3 - List ALL objects without hierarchy summaries

Question

For what it's worth, the Amazon S3 library is being used to perform operations on HCP. Perhaps that limits the version of Amazon S3 we can use or changes the interpretation of a folder. My understanding is, both Amazon S3 and HCP can acknowledge full paths as objects and ignore the folders, but I'm having difficulty getting my requests to do just that.

Say I have:

• something/over/here/object1
• something/over/there/object2
• somewhere/far/object3

and I want every object under something/. Here is my request attempt:

ListObjectsRequest listSomeObjects= new ListObjectsRequest()
.withBucketName("bucket")
.withDelimiter("/")
.withPrefix("something/")
.withMaxKeys("100") // maybe not necessary since it stops at 1000
.withMarker(null); // to be used after first request, I imagine
ObjectListing objectsReturned = this.hs3Client.listObjects(listSomeObjects);

objectsReturned gives me a few options:

• getCommonPrefixes() - if I include a delimiter, returns something/over. Otherwise, appears to be empty.
• getObjectSummaries() - With the delimiter, only returns objects immediately under something/ and itself. In this case, it only has itself. Without the delimiter, returns a list of object summaries for something/, something/over/, something/over/here, something/over/there/, something/over/here/object1, something/over/there/object2.
• getBucketName(), getMarker(), getMaxKeys(), getDelimiter(), getNextMarker(), getPrefix() - these don't appear to be anything I need

I only need a list of the full object names: something/over/here/object1, something/over/there/object2

Looping through all of getObjectSummaries() to filter the results would be a last resort. Using an incredible amount of small requests to account for the subfolders may not even be viable for our use case. Is there a way to get a list of full object names with a common prefix without going through all the keys in getObjectSummaries()?

Answer 1

Which amazon library version are you using?

I can tell you that i've been using

AmazonS3.listObjects(String bucketName, String prefix)

and i only received the list of existing matching keys. Only actual objects, not the "virtual folder" subpaths (using getObjectSummaries method as you do)

As far as i know that is equivalent to your code but without the delimiter (keeping aside maxkeys).

I used version 1.10.52 just in case it matters (would be surprised).

Answer 2

删除.withDelimiter("/")选项将使您获得前缀下的所有对象