Firebase RangeError: Maximum call stack size exceeded

Question

Below is a firebase read query performance test from this SO question, I just changed it to my own firebase instance and removed sequential test leaving only the parallel one.

The problem is that any time I run this code and I make the for loop i < 12 resulting in an array of length: 32,768 , I keep getting an error saying that call stack exceeded: RangeError: Maximum call stack size exceeded but I can't figure out what is causing this error... I tried testing it on both localhost and via cloud functions but no success.

(The referenced SO test works at i<12 but then also breaks if I change it to 13) .

function loadVideosParallel(videoIds, callback) {
  Promise.all(
    videoIds.map((id) => {
      return admin.database().ref("test").child(id).once('value')
        .then((snapshot) => {
          return snapshot;
        })
    })
  ).then((r) => callback());
}

function loadVideosForUser(loadVideosParallel, callback) {
  admin.database().ref("test").once('value').then((snapshot) => {
    var videoKeys = Object.keys(snapshot.val());

    for (var i=0; i < 13; i++) {
      videoKeys = videoKeys.concat(videoKeys);
    }

    loadVideosParallel(videoKeys, function() {
      if (callback) callback();
    });
  })
}

let start = Date.now();
loadVideosForUser(loadVideosParallel, function() {
  const endTime = (Date.now() - start)+'ms';
  console.log('parallel done after '+ endTime +'ms');
});

2017-12-20T23:26:28.527ZE helloWorld: RangeError: Maximum call stack size exceeded at T (/user_code/node_modules/firebase-admin/lib/database/database.js:139:366) at /user_code/node_modules/firebase-admin/lib/database/database.js:190:120 at ta (/user_code/node_modules/firebase-admin/lib/database/database.js:45:40) at ua (/user_code/node_modules/firebase-admin/lib/database/database.js:45:89) at Ag (/user_code/node_modules/firebase-admin/lib/database/database.js:190:93) at /user_code/node_modules/firebase-admin/lib/database/database.js:195:260 at Oc (/user_code/node_modules/firebase-admin/lib/database/database.js:85:363) at Oc (/user_code/node_modules/firebase-admin/lib/database/database.js:85:449) at Oc (/user_code/node_modules/firebase-admin/lib/database/database.js:85:449) at Nc (/user_code/node_modules/firebase-admin/lib/database/database.js:85:316) 2017-12-20T23:26:29.758741829ZD helloWorld: Function execution took 5107 ms, finished with status: 'crash'

Anybody have any insight? I tried looking at the /user_code/node_modules/firebase-admin/lib/database/database.js file but it's minified so can't really tell what's going on.

Answer 1

You'r queueing up more than 32,768 read operations, which is overflowing one of the stacks in the SDK. Loading that many items without any flow control in your code seems like a bad idea, so I'm actually quite happy that the SDK cops out. Consider loading the items in batches that are more reasonable.

In general though: try to limit loading thousands of items from the database. If you need that many items for an aggregation operation, you're much better off keeping the aggregated value explicitly in the database and updating it with every write operation.