"The specified bucket does not exist" when trying to list objects in IBM Cloud Object Storage using Python

Question

Following IBM documentation , I'm trying to list objects in my COS bucket using Python and ibm-cos-sdk.

import ibm_boto3
from ibm_botocore.client import Config, ClientError

cos = ibm_boto3.resource("s3",
      ibm_api_key_id=params['API_KEY'],
      ibm_service_instance_id=params['COS_TGT_INSTANCE_CRN'],
      ibm_auth_endpoint=params['IAM_ENDPOINT'],
      config=Config(signature_version="oauth"),
      endpoint_url=params['COS_ENDPOINT']
)

for obj in cos.Bucket('sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f').objects.all():
  print(obj.key)

Which results in:

ibm_botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist.

I'm pretty sure the bucket does exist, because I can clearly see it in the output from

>>> for b in cos.buckets.all():
...  print(b.name)
... 
sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f

What am I doing wrong here?

Answer 1

The reason for the error is conceptual. You can see all buckets, but only can obtain details for buckets in the connected region. I ran into this a long time ago and solved it like this (tested then, not today):

def buckets_json():
   # Get a list of all bucket names from the response
   buckets = [bucket['Name'] for bucket in cos.list_buckets()['Buckets']]
   locs2=[{"name":name,"loc":loc} for name,loc in locations(buckets=buckets).iteritems()]
   return jsonify(buckets=locs2)

Another snippet which I found:

def locations(buckets):
   locs={}
   for b in buckets:
      try: 
         locs[b]=cos.get_bucket_location(Bucket=b)['LocationConstraint']
      except: 
         locs[b]=None
         pass
   return locs