Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Trouble getting a file from IBM Cloud Object Storage

Question

I am trying to get a file from a bucket in IBM cloud object storage. For this, first I am trying to read all the available files in a bucket.

private static List<String> listBuckets(AmazonS3 cosClient) {
    final List<Bucket> bucketList = cosClient.listBuckets();
    List<String> bucketNames = new ArrayList<String>();
    for (final Bucket bucket : bucketList) {
        bucketNames.add(bucket.getName());
    }
    return bucketNames;
}

public InputStream getCOSFile(AmazonS3 cosClient, String bucketName, String objectName){
    List<String> bucketNames = listBuckets(cosClient);
    if (bucketNames.contains(bucketName)){
        LOGGER.info(bucketName+" exists");
        getBucketContentsV2(cosClient, bucketName, 2);
       }

Here, I get the message bucketName exists from within the if block. Also, the bucket does exist in my cloud account. But, getBucketContentsV2 gives me this error message: "The specified bucket does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Request ID: xxxxx)

Here's the method getBucketContentsV2, almost exactly as in the IBM cloud doc tutorials.

public static void getBucketContentsV2(AmazonS3 cosClient, String bucketName, int maxKeys) {
    System.out.printf("Retrieving bucket contents (V2) from: %s\n", bucketName);

    boolean moreResults = true;
    String nextToken = "";

    while (moreResults) {
        ListObjectsV2Request request = new ListObjectsV2Request()
                .withBucketName(bucketName)
                .withMaxKeys(maxKeys)
                .withContinuationToken(nextToken);

        ListObjectsV2Result result = cosClient.listObjectsV2(request);
        for(S3ObjectSummary objectSummary : result.getObjectSummaries()) {
            System.out.printf("Item: %s (%s bytes)\n", objectSummary.getKey(), objectSummary.getSize());
        }

        if (result.isTruncated()) {
            nextToken = result.getNextContinuationToken();
            System.out.println("...More results in next batch!\n");
        }
        else {
            nextToken = "";
            moreResults = false;
        }
    }
    System.out.println("...No more results!");
}

I also tried retrieving all files in a bucket with V1 code in the docs, and get the same NoSuchBucket error.

Here's the implementation of that method:

public static void listBuckets(AmazonS3 cosClient, String bucketName) {
    System.out.printf("Retrieving bucket contents from: %s\n", bucketName);
    ObjectListing objectListing = cosClient.listObjects(new ListObjectsRequest().withBucketName(bucketName));
    for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
        System.out.printf("Item: %s (%s bytes)\n", objectSummary.getKey(), objectSummary.getSize());
    }
    System.out.println();
}

Answer 1

So I have tried this code to replicate the error. I was able to list objects in the bucket. My first guest would be a configuration error.

String bucketName = "<BUCKET_NAME>";  // eg my-unique-bucket-name
String newBucketName = "<NEW_BUCKET_NAME>"; // eg my-other-unique-bucket-name
String apiKey = "<API_KEY>"; // eg "W00YiRnLW4k3fTjMB-oiB-2ySfTrFBIQQWanc--P3byk"
String serviceInstanceId = "<SERVICE_INSTANCE_ID"; // eg "crn:v1:bluemix:public:cloud-object-storage:global:a/3bf0d9003abfb5d29761c3e97696b71c:d6f04d83-6c4f-4a62-a165-696756d63903::"
String endpointUrl = "https://s3.us-south.cloud-object-storage.appdomain.cloud"; // this could be any service endpoint

Probably the endpointUrl is misconfigured. When you are getting the endpoint follow these steps:

1. Select the Endpoints menu from left
2. Select your Resiliency option. Mine was Regional
3. Select your Region
4. Copy the public endpoint.

Mine was like this: s3.eu-de.cloud-object-storage.appdomain.cloud