node.js: expensive request

Question

Hi, everyone! I need some help in my first app:

I'm creating an application with express+node.js as the background. There is no database. I'm using 3rd-party solution with some functions, that doing calculations, instead.

Front

50 objects. Every object has one unique value: random number. At start I have all these objects, I need to calculate some values for every object and position it on the form based on the calculated results.

Each object sends: axios.get('/calculations?value=uniqueValue') and I accumulate the results in an array. When array.length will be equal 50 I will compare array elements to each other and define (x, y) coordinates of each object. After that, objects will appear on the form.

Back

let value = uniqueValue; // an unique value received from an object
let requests = [];
for (let i = 0; i < 1500; i++) { // this loop is necessary due to application idea
      requests.push(calculateData(value)); // 3rd-party function
      value += 1250;
    }
let result = await Promise.all(requests);
let newData = transform(result); // here I transform the calculated result and then return it.
return newData

Calculations for one object cost 700 ms. All calculations for all 50 objects cost ≈10 seconds. 3rd-party function receives only one value at the same time, but works very quickly. But the loop for (let i = 1; i < 1500; i++) {…} is very expensive.

Issues

1. 10 seconds is not a good result, user can't wait so long. May be I should change in approach for calculations?
2. Server is very busy while calculating, and other requests (eg axios.get('/getSomething?params=something') are pending.

Any advice will be much appreciated!

Answer 1

You can make the call in chunks of data using async.eachLimit


var values = [];
for (let i = 0; i < 1500; i++) { // this loop is necessary due to application idea
      values.push(value);
      value += 1250;
    }
     var arrayOfItemArrays = _.chunk(values, 50);

      async.eachLimit(arrayOfItemArrays, 5, eachUpdate, function(err, result){let 
      newData = transform(result); 
      return newData ;
});

function eachUpdate(req_arr, cb){
  var result = []
  req_arr.forEach(fucntion(item){
     calculateData(item).then((x){
     result.push(x);
  });
 cb(result);
}