How to access AWS "subfolders" using ListObjects() in C++ SDK?

Question

I am trying to list the objects in the "noaa-goes16/GLM-L2-LCFA/2021/140/05" bucket. The AWS C++ SDK says that in order to list the objects, you need to use the following function:

bool ListObjects(const Aws::String& bucketName,
    const Aws::Client::ClientConfiguration& clientConfig) {
    Aws::S3::S3Client s3_client(clientConfig);

    Aws::S3::Model::ListObjectsRequest request;
    request.WithBucket(bucketName);

    auto outcome = s3_client.ListObjects(request);

    if (!outcome.IsSuccess()) {
        std::cerr << "Error: ListObjects: " <<
            outcome.GetError().GetMessage() << std::endl;
    }
    else {
        Aws::Vector<Aws::S3::Model::Object> objects =
            outcome.GetResult().GetContents();

        for (Aws::S3::Model::Object& object : objects) {
            std::cout << object.GetKey() << std::endl;
        }
    }
    return outcome.IsSuccess();
}

After passing the string to the function, I get the following error:

Aws::SDKOptions options;
Aws::InitAPI(options);
Aws::Client::ClientConfiguration clientConfig; 
std::string object = "noaa-goes16/GLM-L2-LCFA/2021/140/05";
ListObject(object, clientConfig); 
    

Output:

Error: ListObjects: The specified key does not exist. 
noaa-goes16/GLM-L2-LCFA/2021/140/05

I have tried to add a "/" to the bucket name, and it states again that it does not exist. If I just try "noaa-goes16" as the bucket name, it lists 1000 files that are not relevant to my application. How do I list the files in the "/05" subfolder using the AWS C++ SDK?

I tried to list the files in a subfolder of the noaa-goes16 bucket using the ListObjects() function in the AWS C++ SDK. I recieved an error that the subfolder I asked for does not exist. However, I know that the subfolder I am asking for does exist. I have tried to add a "/" to the end of the bucket, thinking this would resolve the error. This did not help. I have found that if I only put the main bucket object as the name ("noaa-goes16"), it works. However, I cannot list objects in a specific subfolder and would like to know how.

Answer 1

All, I have just figured it out. Hopefully, this thread helps others.

In order to get the subfolder, you need to use a method on the request object.

For example, in order to list all of the objects in the "GLM-L2-LCFA/2021/140/05" subfolder, add the Aws::String &prefix:

bool ListObjects(const Aws::String& bucketName, const Aws::String &prefix, 
    const Aws::Client::ClientConfiguration& clientConfig) {
    Aws::S3::S3Client s3_client(clientConfig);

    Aws::S3::Model::ListObjectsRequest request;
    request.WithBucket(bucketName).WithPrefix(prefix);

    auto outcome = s3_client.ListObjects(request);

    if (!outcome.IsSuccess()) {
        std::cerr << "Error: ListObjects: " <<
            outcome.GetError().GetMessage() << std::endl;
    }
    else {
        Aws::Vector<Aws::S3::Model::Object> objects =
            outcome.GetResult().GetContents();

        for (Aws::S3::Model::Object& object : objects) {
            std::cout << object.GetKey() << std::endl;
        }
    }
    return outcome.IsSuccess();

Now, pass the following:

Aws::Client::ClientConfiguration clientConfig;
std::string bucket = "noaa-goes16";
std::string prefix = "GLM-L2-LCFA/2021/140/05";
ListObjects(bucket, prefix, clientConfig);

Viola!