nodejs and sql queries - concurrent access

Question

I'm discovering nodejs and its asynchronous system and I'm doing a query in MySQL with the node-mysql package. I'm doing something like that:

• iterating a collection
• if the item is not in the DB, I insert
• if the item is in the DB, I update

So my code look like that:

var stuff = ['a', 'b', 'a'];
for(var i=0;i<stuff.length;i++){
    connection.query("SELECT COUNT(*) AS count FROM table WHERE column = ?", [stuff[i]],
    (function(err, rows){
        if(rows.count == 0){
            connection.query("INSERT INTO table ...");
        } else {
            connection.query("UPDATE table SET ...");
        }
    }).bind(this)
}

But I'm wondering, because of asynchronous mode, if sometimes there is a problem with this pattern. What is the behavior here ?

• "SELECT WHERE column = 'a'" ==> count == 0
• "SELECT WHERE column = 'b'" ==> count == 0
• "SELECT WHERE column = 'a'" ==> count == 0
• "INSERT 'a'"
• "INSERT 'b'"
• " INSERT 'a'" ==> unexpected behavior

or

• "SELECT WHERE column = 'a'" ==> count == 0
• "INSERT 'a'"
• "SELECT WHERE column = 'b'" ==> count == 0
• "INSERT 'b'"
• "SELECT WHERE column = 'a'" ==> count == 1
• " UPDATE 'a'" ==> expected behavior !!

I hope you will understand my problem, sorry for my bad english, it's a real handicap..

Thank you.

Answer 1

mysql has a insert or update command, http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html This will insert if it doesn't exist, or update if it does.

As for insert/update with this async behavior, take a look at the async lib to run things like this in series instead. With your code example above, you can't be sure which will run/finish first IIRC.

It also depends on what exactly you are trying to update. If you are just going to count something, and need to increment the count, order doesn't matter. Again, this all depends on what you actually want to insert/update.

Answer 2

I find that recursion often works great for this kind of challenge. I use it all the time - it seems simpler than resorting to async for such a straight-forward problem.

Something like this:

var stuff = ['a', 'b', 'a'];

processStuff(stuff);

function processStuff(theStuff) {

    if (theStuff.length) == 0 {
        // Done all items - either return result to client or execute some callback here
        // res.end('Completed');
        return;
    }

    var thisItem = theStuff.shift();

    connection.query('SELECT * FROM table WHERE column = ' + connection.escape(thisItem), function(err, rows) {
        if (rows.length == 0) {
            connection.query("INSERT INTO table ...", function(err, result) {
                processStuff(theStuff);
            });
        } else {
            connection.query("UPDATE table SET ...", function(err, result) {
                processStuff(theStuff);
            });
        }
    });
}

This will keep your queries and updates/inserts flowing one after another, and yet it is still asynchronous. I find this easier to read as well, and easy to modify.