DynamoDB query/scan only returns subset of items

Question

I noticed that DynamoDB query/scan only returns documents that contain a subset of the document, just the key columns it appears.

This means I need to do a separate Batch_Get to get the actual documents referenced by those keys.

I am not using a projection expression, and according to the documentation this means the whole item should be returned. 1

How do I get query to return the entire document so I don't have to do a separate batch get?

One example bit of code that shows this is below. It prints out found documents, yet they contain only the primary key, the secondary key, and the sort key.

t1 = db.Table(tname)

q = { 
    'IndexName': 'mysGSI',
    'KeyConditionExpression': "secKey= :val1 AND " \
                              "begins_with(sortKey,:status)",
    'ExpressionAttributeValues': {
        ":val1": 'XXX',
        ":status": 'active-',
    }
}
res = t1.query(**q)

for doc in res['Items']:
    print(json.dumps(doc))

Answer 1

This situation is discussed in the documentation for the Select parameter. You have to read quite a lot to find this, which is not ideal.

If you query or scan a global secondary index, you can only request attributes that are projected into the index. Global secondary index queries cannot fetch attributes from the parent table.

Basically:

• If you query the parent table then you get all attributes by default.

• If you query an LSI then you get all attributes by default - they're retrieved from the projection in the LSI if all attributes are projected into the index (so that costs nothing extra) or from the base table otherwise (which will cost you more reads).

• If you query or scan a GSI, you can only request attributes that are projected into the index. GSI queries cannot fetch attributes from the parent table.