Boto3 and S3: For loop finishing after iterating through only 6 out of 200 S3 buckets in the list

Question

I'm trying to list and log all the S3 buckets PublicAccessBlockConfiguration in a list. The thing is, the loop finishes and prints 'No Public Access' after iterating over only 6 out of ~200 buckets.

I tested the code statically, writing the name of a bucket that I knew existed and had PublicAccessBlockConfiguration, and it worked.

But when iterating over the list, the same bucket doesn't show up. Why is that?

def check_bucket_access_block():
    try:
        for bucket in filtered_buckets:
            response = s3client.get_public_access_block(Bucket=bucket['Name'])
            for key, value in response['PublicAccessBlockConfiguration'].items():
                logger.info('Bucket Name: {}, {}: {}'.format(bucket['Name'], key, value))
    except botocore.exceptions.ClientError as e:
        if e.response['Error']['Code'] == 'NoSuchPublicAccessBlockConfiguration':
            print('\t no Public Access')
        else:
            print("unexpected error: %s" % (e.response))

check_bucket_access_block()

Answer 1

Your try/except block is outside of the for loop. Therefore when an error is generated, the loop is exited.

Try putting the try/except inside the for loop, something like this:

def check_bucket_access_block():
    for bucket in filtered_buckets:
        try:
            response = s3client.get_public_access_block(Bucket=bucket['Name'])
            for key, value in response['PublicAccessBlockConfiguration'].items():
                logger.info('Bucket Name: {}, {}: {}'.format(bucket['Name'], key, value))
        except botocore.exceptions.ClientError as e:
            if e.response['Error']['Code'] == 'NoSuchPublicAccessBlockConfiguration':
                print('\t no Public Access')
            else:
                print("unexpected error: %s" % (e.response))