AWS-SDK DynamoDB NodeJS性能不好？

Question

I'm testing the dynamoDB API with the AWS-SDK for NodeJS. But I have some strange behavior, dynamoDB seems to be slower to respond and I don't understand why.

From the AWS Console, I set "100" Read capacity units to "MyTable"

I made some benchmarks :

For 10 calls the services seems to respond in less then 1 sec; 100 calls (1,5 sec), 1000 calls (4,5 secs), 5000 calls (30 secs)

See below for my code :

var http = require('http');
var https = require('https');
var AWS = require("aws-sdk");

var start = new Date();

http.globalAgent.maxSockets = 100000;
https.globalAgent.maxSockets = 100000;

AWS.config.update({
  accessKeyId: 'MyAccessKey',
  secretAccessKey: 'MySecretKey',
  region: 'MyRegion'
});

var dynamodbDoc = new AWS.DynamoDB.DocumentClient();

var startedDate = new Date()
console.log('Start');
for (var i = 0; i < 5000; i++) {
    getVolume("604275e8-17de-4582-bbc2-1b5eaf3b0a4c");
}   
console.log('Over ' + (new Date() - startedDate) + ' ms');

function getVolume(key) {
    var params = {
        TableName: "MyTable",
        KeyConditionExpression: "#id = :key",
        ExpressionAttributeNames: {
            "#id": "id"
        },
        ExpressionAttributeValues: {
            ":key": key
        }
    };
    var requestStarted = new Date()
    dynamodbDoc.query(params, function (err, data) {
        if (err) {
            console.error("Unable to query. Error:", JSON.stringify(err, null, 2));
        } else {
            console.log("DynamoDB : " + (new Date() - requestStarted));
        }
    });
}

With the 5000 calls, the "for" statement is over after 3 seconds but the first dynamoDB respond after 30 seconds, why ?

Thank you in advance,

Answer 1

You are sending 5000 calls to DynamoDB virtually simultaneously. First, there is probably some CPU and network bottlenecks on the computer running the test keeping this from performing as well as you want. Second, if this is really what you intend to be testing then you would need to increase the write capacity of DynamoDB to something much higher.

If you are trying to perform the calls to DynamoDB one at a time, then you need to wait for the callback to finish before executing the next call.

Answer 2

thank you for your answer.

Currently I have a nodejs web server (with restify) and when I did a load test on it (more than 1000 req/s), dynamodb seems to be slow to respond, more than 60 seconds and of course some request have a timeout.

With my code I would like to understand why dynamodb seems to be slow. I'm looking for the bottleneck in my code.