DynamoDB QueryResultPage still returning results on bogus exclusive start key

Question

tldr; - When using a bogus LastEvaluatedKey with DynamoDB queries for pagination it still returns results in some cases.

I am implementing pagination for a fairly straight forward CRUD Repository. The implementation is based on: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.Pagination.html and
Pagination with DynamoDBMapper Java AWS SDK

I have a DynamoDB table and this query is running on a Global Secondary Index of that table.

Pagination is working fine, ie I have a 5000 records. I query and receive a set of 500 results and a LastEvaluatedKey. Using this key, I get the next set of 500 results etc.

This key is made up of:

• partition key "instanceId" which is always the same on subsequent page requests.
• range key "id" which is what changes on the next page request

Now I wrote a test to make sure that if a bogus LastEvaluatedKey is provided I should get zero results.

What is the actual behavior:

If I provide something like id = "rrrrrrrrrrrrr" I get zero results, as expected.
If I provide something like id = "aaaaaaaaaaaaa" I get 500 results!

What's worth noting is that the "id"'s are UUID strings, so the letter 'r' will not occur anywhere in any id.

My LastEvaluatedKey is made up like so (instanceId is the same for subsequent page queries):

        var startKeyMap = new HashMap<String, AttributeValue>();

        var idValue = new AttributeValue();
        idValue.setS(startKey);
        startKeyMap.put("id", idValue);

        var instanceIdValue = new AttributeValue();
        instanceIdValue.setS(instanceId);
        startKeyMap.put("instanceId", instanceIdValue);

        queryExpression.setExclusiveStartKey(startKeyMap);

I suspect what is happening is that because "id" is a sort key (in the GSI), the results are returned for anything greater than my bogus "aaaaaaaaaaaaaa". For "rrrrrrrrrrrr" it doesn't work because no keys would sort greater than 'rrrrrrrrrrrr'.

I would expect DDB to match exactly the exclusive start key, and return the next set of results from there but it seems like it is simply matching whatever comes close and returning whatever keys come after.

I also found: DynamoDB Global Secondary Index with Exclusive Start Key

In there the solution is to set the primary and range keys of both the table and the index as the ExclusiveStartKey. However, in my case both are there, they are just reversed:

On the table the id is primary, instanceId is secondary. On the GSI, the reverse is true.

Can someone explain what is happening or what I'm doing wrong?

Answer 1

Working as designed...

ExclusiveStartKey just means to start with key greater than whatever value you've passed in.

Exclusive, meaning use greater than as opposed to inclusive which would be greater than or equal.