Boto3 Athena 没有显示所有表格

Question

trying to get a list of table names in Athena Table using BOTO3 python.

this is my code; I think my attempts to do paginator is not correct. Any help is appreciated

import boto3

client = boto3.client('glue')
responseGetDatabases = client.get_databases()
databaseList = responseGetDatabases['DatabaseList']

for databaseDict in databaseList:
    databaseName = databaseDict['Name']
    if "dbName_" in databaseName:
        print '\ndatabaseName: ' + databaseName
        responseGetTables = client.get_tables( DatabaseName = databaseName )
        paginator = client.get_paginator(['TableList'])
        for page in paginator:
            tableList = responseGetTables['TableList']
            for tables in tableList:
                print tables['Name'] 

Answer 1

The get_paginator function parameter must be the name of the operation. It looks like you're trying to paginate on the get_tables function so

    paginator = client.get_paginator(['TableList'])

should be:

    paginator = client.get_paginator('get_tables')

Once you have the paginator object, you need to call paginator.paginate to retrieve the iterator. You can send your database parameters like so:

    page_iterator = paginator.paginate(
        DatabaseName=databaseDict['Name'],
        PaginationConfig={
            'MaxItems': 123,
            'PageSize': 123,
            'StartingToken': 'string'
        }
    )

See the documention for this function here .

Now that you have the iterator, you can call a for loop by enumerating on it:

for page_index, page in enumerate(page_iterator):

Answer 2

Here's a full working example on how to do it using paginator.

Remember to provide region_name and database_name .

import boto3

region_name = '<PROVIDE_AWS_REGION_NAME>'
database_name = '<PROVIDE_YOUR_DATABASE_NAME>'
catalog_name = 'AwsDataCatalog'

athena = boto3.client('athena', region_name=region_name)

paginator = athena.get_paginator('list_table_metadata')
response_iterator = paginator.paginate(
    CatalogName=catalog_name,
    DatabaseName=database_name
)

table_names = []
for page in response_iterator:
    table_names.extend(
        (i['Name'] for i in page['TableMetadataList'])
    )

print(table_names)