Delete a folder and its content AWS S3 java

Question

Is it possible to delete a folder(In S3 bucket) and all its content with a single api request using java sdk for aws. For browser console we can delete and folder and its content with a single click and I hope that same behavior should be available using the APIs also.

Answer 1

There is no such thing as folders in S3; There are simply files with slashes in the filenames.

Browser console will visualize these slashes as folders, but they're not real.

You can delete all files with the same prefix, but first you need to look them up with list_objects(), then you can batch delete them.

For code snippet using Java sdk please refer below doc

http://docs.aws.amazon.com/AmazonS3/latest/dev/DeletingMultipleObjectsUsingJava.html

Answer 2

You can specify keyPrefix in ListObjectsRequest.

For example, consider a bucket that contains the following keys:

• foo/bar/baz
• foo/bar/bash
• foo/bar/bang
• foo/boo

And you want to delete files from foo/bar/baz .

if (s3Client.doesBucketExist(bucketName)) {
        ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                .withBucketName(bucketName)
                .withPrefix("foo/bar/baz");

        ObjectListing objectListing = s3Client.listObjects(listObjectsRequest);

        while (true) {
            for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
                s3Client.deleteObject(bucketName, objectSummary.getKey());
            }
            if (objectListing.isTruncated()) {
                objectListing = s3Client.listNextBatchOfObjects(objectListing);
            } else {
                break;
            }
        }
    }

https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ListObjectsRequest.html

Answer 3

There is no option of giving a folder name or more specifically prefix in java sdk to delete files. But there is an option of giving array of keys you want to delete. Click for details . By using this, I have written a small method to delete all files corresponding to a prefix.

private AmazonS3 s3client = <Your s3 client>;
private String bucketName = <your bucket name, can be signed or unsigned>;

public void deleteDirectory(String prefix) {
    ObjectListing objectList = this.s3client.listObjects( this.bucketName, prefix );
    List<S3ObjectSummary> objectSummeryList =  objectList.getObjectSummaries();
    String[] keysList = new String[ objectSummeryList.size() ];
    int count = 0;
    for( S3ObjectSummary summery : objectSummeryList ) {
        keysList[count++] = summery.getKey();
    }
    DeleteObjectsRequest deleteObjectsRequest = new DeleteObjectsRequest( bucketName ).withKeys( keysList );
    this.s3client.deleteObjects(deleteObjectsRequest);
}

Answer 4

You can try this

void deleteS3Folder(String bucketName, String folderPath) {
    for (S3ObjectSummary file : s3.listObjects(bucketName, folderPath).getObjectSummaries()){
        s3.deleteObject(bucketName, file.getKey());
    }
}

Answer 5

You can try the below methods, it will handle deletion even for truncated pages:

   public List<String> listS3DirFiles(String dirPrefix) {
        ListObjectsV2Request s3FileReq = new ListObjectsV2Request()
                .withBucketName(bucket)
                .withPrefix(dirPrefix)
                .withDelimiter(PATH_SEPARATOR);
        List<String> filesList = new ArrayList<>();
        ListObjectsV2Result objectsListing;
        try {
            do {
                objectsListing = client.listObjectsV2(s3FileReq);
                for (S3ObjectSummary summary: objectsListing.getObjectSummaries()) {
                    filesList.add(summary.getKey());
                }

                s3FileReq.setContinuationToken(objectsListing.getNextContinuationToken());
            } while(objectsListing.isTruncated());
        } catch (SdkClientException e) {
            log.warn("Error while getting fetching list of files for dir '{}' from S3 bucket '{}' due to {}",
                    dirPrefix, bucket, e.getMessage());
            throw e;
        }

        return filesList;
    }

    public void deleteDirectoryContents(String directoryPrefix) {
        List<String> keysList = listS3DirFiles(directoryPrefix);

        if (keysList.isEmpty()) {
            log.warn("Given directory {} doesn't have any file", directoryPrefix);
            return;
        }

        DeleteObjectsRequest deleteObjectsRequest = new DeleteObjectsRequest(bucket).withKeys(String.valueOf(keysList));
        try {
            client.deleteObjects(deleteObjectsRequest);
        }  catch (SdkClientException e) {
            log.warn("Error while deleting list of files {} for dir '{}' from S3 bucket '{}' due to {}",
                    keysList.toString(), directoryPrefix, bucket, e.getMessage());
            throw e;
        }
    }

Answer 6

First you need to fetch all object keys starting with the given prefix:

public List<FileKey> list(String keyPrefix) {
  var objectListing = client.listObjects("bucket-name", keyPrefix);
  var paths =
      objectListing.getObjectSummaries().stream()
        .map(s3ObjectSummary -> s3ObjectSummary.getKey())
        .collect(Collectors.toList());

  while (objectListing.isTruncated()) {
    objectListing = client.listNextBatchOfObjects(objectListing);

    paths.addAll(
        objectListing.getObjectSummaries().stream()
            .map(s3ObjectSummary -> s3ObjectSummary.getKey())
            .toList());
  }

  return paths.stream().sorted().collect(Collectors.toList());
}

Then call deleteObjects:

client.deleteObjects(new DeleteObjectsRequest("bucket-name").withKeys(list("some-prefix")));