All folder names are not coming in List objects call of Amazon S3

Question

enter image description here I'm using the following code to list all the objects present in the bucket of S3. But the problem is not all the folder names are coming in the call. I've attached the image for reference.

AmazonS3 s3Object = AmazonS3Utils.getS3Object(amazonS3ConnectionValues.regionName, amazonS3ConnectionValues.accessKey, amazonS3ConnectionValues.secretKey);
                s3Object.listBuckets();
                ListObjectsV2Result objectsList = s3Object.listObjectsV2(bucket);
                List<S3ObjectSummary> objectSummaries = objectsList.getObjectSummaries();
                for (S3ObjectSummary objectSummary : objectSummaries) {
                    Object[] newRow;
                    newRow = RowDataUtil.allocateRowData(data.outputRowMeta.size());
                    int index = 0;
                    newRow[index++] = objectSummary.getKey();
                    newRow[index++] = objectSummary.getETag();
                    newRow[index++] = objectSummary.getSize();
                    newRow[index++] = objectSummary.getLastModified();
                    newRow[index++] = objectSummary.getStorageClass();
                    putRow(data.outputRowMeta, newRow);

As shown in the image foldername "automationedge123/" is not shown in the response, other folder names such as "Test Folder/", "automationedge/" are shown in the output. I'm not understanding this behavior.

Answer 1

The listObjects result might be paginated. Make sure that you are not looking at a truncated result by looking at the return value of isTruncated() ( link ).

Here's a quote from the javadoc of listObjects(...)] ( link )

Because buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Always check the ObjectListing.isTruncated() method to see if the returned listing is complete or if additional calls are needed to get more results. Alternatively, use the AmazonS3Client.listNextBatchOfObjects(ObjectListing) method as an easy way to get the next page of object listings.