Python / Boto3 / How can i get a tag list with pagination?

Question

I'd like to get a rds and each tag list using boto3 without 100 limits.

This is the code for getting the list of rds and each tag.

client = boto3.client('rds')
instances = client.describe_db_instances()['DBInstances']

for i in instances:
    db_instance_name = i['DBInstanceIdentifier']
    arn = i['DBInstanceArn']
    tags = client.list_tags_for_resource(ResourceName=arn)
    for item in tags['TagList']:
        if item['Key'] == 'Name':
            print(db_instance_name,item['Value'])

And this is the code for pagination.

def all_rds_instances(page_size=20):
    client = session.client('rds')
    marker = ""
    pool = []
    while True:
        for instance in pool:
            yield instance
        if marker is None:
            break
        result = client.describe_db_instances(MaxRecords=page_size, Marker=marker)
        marker = result.get("Marker")
        pool = result.get("DBInstances")

How can I combine these 2 codes?

Answer 1

You just need to change the for loop to iterate over your all_rds_instances generator.

Your script would look like:

import boto3

client = boto3.client('rds')

def all_rds_instances(page_size=20):
    marker = ''
    pool = []
    while True:
        for instance in pool:
            yield instance
        if marker is None:
            break
        result = client.describe_db_instances(MaxRecords=page_size, Marker=marker)
        marker = result.get('Marker')
        pool = result.get('DBInstances')


for i in all_rds_instances():
    db_instance_name = i['DBInstanceIdentifier']
    arn = i['DBInstanceArn']
    tags = client.list_tags_for_resource(ResourceName=arn)
    for item in tags['TagList']:
        if item['Key'] == 'Name':
            print(db_instance_name, item['Value'])

When you use the yield keyword your function becomes a generator and it works the same way as any iterable in python.

There is some cool answer about how generator works here