使用Java SDK列出Amazon S3中的所有对象

Question

我正在列出存储桶中的所有对象。问题是当我列出所有对象时，一批中的最后一个对象再次在下一批对象中考虑，并在下一批中重复。为什么会发生，因为它只考虑了1000个批处理中的对象，并且不应考虑先前的批处理对象。

BasicAWSCredentials credentials = new BasicAWSCredentials("foo", "bar");
client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(http://(serviceEndpoint), null(signingRegion is null))
.withPathStyleAccessEnabled(true)
.withChunkedEncodingDisabled(true)
.build();
ObjectListing listing = client.listObjects( "bucketname");
System.out.println("Listing size "+listing.getObjectSummaries().size());
System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
System.out.println("At 999 index "+ listing.getObjectSummaries().get(999).getKey());
SomeFunction(listing);

while (listing.isTruncated()) {
    System.out.println("-----------------------------------------------");
    listing = client.listNextBatchOfObjects(listing);
    System.out.println("Listing size "+listing.getObjectSummaries().size());
    System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
    System.out.println("At 999 index "+ listing.getObjectSummaries().get(1000).getKey());
    someFunction(listing);
}

我的输出是：

Listing size 1000
At 0 index folder1/a.gz
At 999 index folder1/b.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/b.gz
At 1000 index folder1/d.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/d.gz
At 1000 index folder1/e.gz

如您所见，第一批999索引在第二批中被考虑（相同）为什么不应该发生？下一批使用1001个对象，包括前一个的最后一个对象，因为它应该给下1000个而不是1001。我解决这个。谢谢。

Answer 1

它不适合注释，因此我将代码放在这里：

BasicAWSCredentials credentials = new BasicAWSCredentials(accessKey, secretAccessKey);
AmazonS3 s3Client = AmazonS3ClientBuilder
        .standard()
        .withCredentials(new AWSStaticCredentialsProvider(credentials))
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("s3-us-west-1.amazonaws.com", region))
        .withPathStyleAccessEnabled(true)
        .withChunkedEncodingDisabled(true)
        .build();
ObjectListing listing = s3Client.listObjects(bucket);
int size = listing.getObjectSummaries().size();
if (size > 0) {
    System.out.println("Listing size " + size);
    System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
    System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
}

while(listing.isTruncated()) {
    System.out.println("-----------------------------------------------");
    listing = s3Client.listNextBatchOfObjects(listing);
    size = listing.getObjectSummaries().size();
    if (size > 0) {
        System.out.println("Listing size " + size);
        System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
        System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
    }
}

结果如下：

Listing size 1000
At 0 index file0001.txt
At 999 index file1000.txt
-----------------------------------------------
Listing size 1000
At 0 index file1001.txt
At 999 index file2000.txt
-----------------------------------------------
Listing size 1000
At 0 index file2001.txt
At 999 index file3000.txt
-----------------------------------------------
Listing size 100
At 0 index file3001.txt
At 99 index file3100.txt

我对代码的输出方式和输出大小做了一些更改。 顺便说一下，我正在使用1.11.330 SDK版本。