Memory management when converting redis keys

Question

I need to convert keys from type string to hash. The name of all keys is in the set list:of:keys .

My current implementation looks like this:

var rdbc = require("redis").createClient(6379, '127.0.0.1');
rdbc.smembers("list:of:keys", function(err, strings){
    strings.forEach(function(string, index, strings){
        rdbc.get(string, function(err, result){
            rdbc.del(string);
            rdbc.hset(string, "foo", result);
        });
    });
});

My attempt works. But when list:of:keys contains many values memory usage grows a lot.

1. Are there memory efficient structures to go through many keys? (especially strings.forEach(… seems inefficient)

2. How do I inform the garbage collector in node.js to clean up after each rdbc.del/rbdc.hset operation?

Answer 1

1. I don't know how the redis driver for node.js handles this, but if they are clever they use some the cursors redis offers to loop through the results. This means the do not fetch all results at the beginning, they fetch them when you access them.

2. The garbage collection is handled by the V8 (the underlying JavaScript engine of node.js) There is plenty of documentation on how it works (just search for "Garbage Collection V8"):

   • https://developers.google.com/v8/design#garb_coll
   • Node.js and V8 garbage collection
   • garbage collection with node.js
   • Node.js' garbage collector