UnrecognizedClientException after updating to new version of boto libraries

Question

After updating boto3 and botocore this code calling the describe_trails function of CloudTrail client objects now errors. The library versions that were changed were as follows:

boto3 : 1.9.44 -> 1.14.45

botocore : 1.12.44 -> 1.17.45

def get_region_list():
    '''
    Get the list of regions covered by CloudTrail from AWS
    '''
    return boto3.session.Session().get_available_regions(
        service_name='cloudtrail',
        partition_name='aws',
        allow_non_regional=False
    )


def generate_cloudtrail_clients(region_list, access_key, secret_key):
    '''
    Generates client objects that interact with CloudTrail in Amazon AWS.
    Each client object corresponds to a different region in region_list
    '''
    for region in region_list:
        yield boto3.client(
            'cloudtrail',
            aws_access_key_id=access_key,
            aws_secret_access_key=secret_key,
            region_name=region
        )
        
clients = generate_cloudtrail_clients(get_region_list(), access_key, secret_key)

for client in clients:
    print(client.describe_trails())

The error it gives me:

ClientError                               Traceback (most recent call last)
<ipython-input-31-31c1b228c022> in <module>
     30 
     31 for client in clients:
---> 32     print(client.describe_trails())

/opt/anaconda3/lib/python3.7/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    314                     "%s() only accepts keyword arguments." % py_operation_name)
    315             # The "self" in this scope is referring to the BaseClient.
--> 316             return self._make_api_call(operation_name, kwargs)
    317 
    318         _api_call.__name__ = str(py_operation_name)

/opt/anaconda3/lib/python3.7/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    633             error_code = parsed_response.get("Error", {}).get("Code")
    634             error_class = self.exceptions.from_code(error_code)
--> 635             raise error_class(parsed_response, operation_name)
    636         else:
    637             return parsed_response

ClientError: An error occurred (UnrecognizedClientException) when calling the DescribeTrails operation: The security token included in the request is invalid.

From what I've found looking up about this error is that it often occurs if the access_key, secret_access_key credentials do not have the correct permissions to access the relevant objects. In this case I do have the correct permissions, as I can access these objects on an older version of the boto libraries and it is clear in the attached permission JSON.

{
    "Effect": "Allow",
    "Action": [
        "cloudtrail:DescribeTrails"
    ],
    "Resource": "*"
}

Any idea what is going wrong here for this error to occur?

Answer 1

The error was caused by the new regions that are optionally available not being enabled.

The new boto libraries return a larger set of available regions than the older versions. If a client is created for one of the regions that is disabled, and then we call DescribeTrails with that client we get the UnrecognizedClientException error.