DynamoDB - Newly put items are not reflecting in scan

Question

I have a problem with DynamoDB scan. I added new items to the table using putItem method.

[[AmazonClientManager ddb] putItem:request];

But when I try to fetch using scan using scan method, that item is not coming in the result.

DynamoDBScanResponse *response = [[AmazonClientManager ddb] scan:request];

I am getting the below response,

{Items: ( ),Count: 0,ScannedCount: 608,LastEvaluatedKey: {HashKeyElement: {S: U2575220130319062347000,N: (null),SS: ( ),NS: ( ),},RangeKeyElement: (null),},ConsumedCapacityUnits: 129,{requestId: 3GVT8PJGV4VB45IUPUA6KIN9URVV4KQNSO5AEMVJF66Q9ASUAAJG}}

But these items are showing in table, when I checked using AWS Console. Could anyone let me know what could be the issue?

Thanks.

Answer 1

Scan API is eventual consistent. Eventual consistent read may not return most recent changes. There is a slight delay (not more than a few seconds).

Query API allows consistent option. You may use Query if that's an option for you.

Another possibility is that you may not have finished processing the results of the scan yet -- Scan needs to be repeated until LastEvaluatedKey is null.

http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html

Answer 2

Setting ConsistentRead to true works for me:

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html#DDB-Scan-request-ConsistentRead

A Boolean value that determines the read consistency model during the scan:

If ConsistentRead is false, then the data returned from Scan might not contain the results from other recently completed write operations (PutItem, UpdateItem or DeleteItem).

If ConsistentRead is true, then all of the write operations that completed before the Scan began are guaranteed to be contained in the Scan response.

The default setting for ConsistentRead is false.

Answer 3

If anyone is having issues with consistent reads, meaning you are expecting them, but it does not seem to always work, here was my findings/solution...

1. Assuming you are using the AWS Java SDK - com.amazonaws:aws-java-sdk-dynamodb
2. First, we made a unit test that loops at high speed to be able to recreate the read inconsistency.
3. The test would read a specific row, update it, and read back the same row to assert the updated value.

Given there are 3 ways to fetch data:

• mapper.load()
• mapper.query()
• mapper.scan()

The "mapper.load()" uses the mapper's config that was set when the mapper was instantiated. For example - mapper.withConsistentReads(ConsistentReads.CONSISTENT).

The "mapper.query()" and "mapper.scan()" do not respect the mapper's config, rather they only use the DynamoDBQueryExpression and DynamoDBScanExpression inputs to determine consistent reads.

ANSWER: You need to use expression.setConsistentRead(true), even if your mapper has .withConsistentReads(ConsistentReads.CONSISTENT). Like this...

public PaginatedScanList<T> getAllRowsFor() {
    final DynamoDBScanExpression scanExpression = new DynamoDBScanExpression();
    scanExpression.setConsistentRead(true);
    return mapper.scan(myClass, scanExpression);
}

You can see the AWS Java source code here that shows the mapper's config is not used - com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper#createScanRequestFromExpression