[英]List all objects in Amazon S3 using Java SDK
I am listing all objects in my bucket.The question is when i am listing all objects,the last object in a batch is again considering in next batch of object and its repeating for next batch.Why is it happening as it should consider only 1000 objects in a batch and should not consider the previous batch object. 我正在列出存储桶中的所有对象。问题是当我列出所有对象时,一批中的最后一个对象再次在下一批对象中考虑,并在下一批中重复。为什么会发生,因为它只考虑了1000个批处理中的对象,并且不应考虑先前的批处理对象。
BasicAWSCredentials credentials = new BasicAWSCredentials("foo", "bar");
client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(http://(serviceEndpoint), null(signingRegion is null))
.withPathStyleAccessEnabled(true)
.withChunkedEncodingDisabled(true)
.build();
ObjectListing listing = client.listObjects( "bucketname");
System.out.println("Listing size "+listing.getObjectSummaries().size());
System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
System.out.println("At 999 index "+ listing.getObjectSummaries().get(999).getKey());
SomeFunction(listing);
while (listing.isTruncated()) {
System.out.println("-----------------------------------------------");
listing = client.listNextBatchOfObjects(listing);
System.out.println("Listing size "+listing.getObjectSummaries().size());
System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
System.out.println("At 999 index "+ listing.getObjectSummaries().get(1000).getKey());
someFunction(listing);
}
My ouput is: 我的输出是:
Listing size 1000
At 0 index folder1/a.gz
At 999 index folder1/b.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/b.gz
At 1000 index folder1/d.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/d.gz
At 1000 index folder1/e.gz
As you can see the first batch 999 index is considered in second batch(same) why?It shouldn't happen right?And the next batch is taking 1001 objects including the last one from previous as it should give next 1000 not 1001.Help me solve this.Thank You. 如您所见,第一批999索引在第二批中被考虑(相同)为什么不应该发生?下一批使用1001个对象,包括前一个的最后一个对象,因为它应该给下1000个而不是1001。我解决这个。谢谢。
It doesn't fit in the note so I put my code here: 它不适合注释,因此我将代码放在这里:
BasicAWSCredentials credentials = new BasicAWSCredentials(accessKey, secretAccessKey);
AmazonS3 s3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("s3-us-west-1.amazonaws.com", region))
.withPathStyleAccessEnabled(true)
.withChunkedEncodingDisabled(true)
.build();
ObjectListing listing = s3Client.listObjects(bucket);
int size = listing.getObjectSummaries().size();
if (size > 0) {
System.out.println("Listing size " + size);
System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
}
while(listing.isTruncated()) {
System.out.println("-----------------------------------------------");
listing = s3Client.listNextBatchOfObjects(listing);
size = listing.getObjectSummaries().size();
if (size > 0) {
System.out.println("Listing size " + size);
System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
}
}
And here's the result: 结果如下:
Listing size 1000
At 0 index file0001.txt
At 999 index file1000.txt
-----------------------------------------------
Listing size 1000
At 0 index file1001.txt
At 999 index file2000.txt
-----------------------------------------------
Listing size 1000
At 0 index file2001.txt
At 999 index file3000.txt
-----------------------------------------------
Listing size 100
At 0 index file3001.txt
At 99 index file3100.txt
I changed your code a little on how to output sizes and indexes. 我对代码的输出方式和输出大小做了一些更改。 By the way I'm using 1.11.330 SDK version.
顺便说一下,我正在使用1.11.330 SDK版本。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.