Weird memory usage on Node.js

Question

This simple code stores 1 million strings (100 chars length) in an array.

function makestring(len) {
    var s = '';
    while (len--) s = s+'1';
    return s;
}

var s = '';
var arr = [];
for (var i=0; i<1000000; i++) {
    s = makestring(100);
    arr.push(s);
    if (i%1000 == 0) console.log(i+' - '+s);
}

When I run it, I get this error:

(...)
408000 - 1111111111111111111 (...)
409000 - 1111111111111111111 (...)
FATAL ERROR: JS Allocation failed - process out of memory

That's strange 1 million * 100 are just 100 megabytes.

But if I move the s = makestring(100); outside the loop...

var s = makestring(100);
var arr = [];
for (var i=0; i<1000000; i++) {
    arr.push(s);
    if (i%1000 == 0) {
        console.log(i+' - '+s);
    }
}

This executes without errors!

Why? How can I store 1 Million objects in node?

Answer 1

In the moment you move the String generation outside the loop, you basically just create one String and push it a million times into the array.

Inside the array, however, just pointers to the original String are used, which is far less memory consuming then saving the String a million times.

Answer 2

Your first example builds 1000000 strings.

In your second example, you're taking the same string object and adding it to your array 1000000 times. (it's not copying the string; each entry of the array points to the same object)

V8 does a lot of things to optimize string use. For example, string concatenation is less expensive (in most cases) than you think. Rather than building a whole new string, it will typically opt to connect them ia linked list fashion under the covers.