AWS DynamoDB Lambda Scan with key

Question

I am currently making a lambda scan where I need to scan a table which currently is bigger than 1mb (the size limit of a single scan). However, my current experience doesnt allow me to rewrite the code for repetitive actions. This is the current call:

 const params = {
  TableName: 'product',
};
let items = []
try {
  const result = await dynamoDbLib.call("scan", params);
  items = result.Items;
  if (result.LastEvaluatedKey) {
    params.ExclusiveStartKey = result.LastEvaluatedKey;
    const newResult = await dynamoDbLib.call("scan", params);
    items.concat(newResult.Items)
  }
  callback(null, success(items));
} catch (e) {
  callback(null, failure({ status: false }));
}

So it fires two times, but it should keep on going until the last key is undefined. Any help would be much appreciated!

Greetings Bram

Answer 1

Create a recursive function that will stop when the LastEvaluatedKey is undefined. It will keep on collecting the results of the scan until the LastEvaluatedKey is undefined and then it will return you the results of the scan.

async function scanTillEnd(params, items){
     const result = await dynamoDbLib.call("scan", params);
     items.concat(result.Items)
     //check if they are more items to get
     if (result.LastEvaluatedKey) {
       params.ExclusiveStartKey = result.LastEvaluatedKey;
       //scan the table again
       return await scanTillEnd(params, items)
    }
    else
       //have scanned till the end, return the results
       return items
}
const params = {
    TableName: 'product',
};
let items = []
let results = scanTillEnd(params,items)