Mongoose findOneAndUpdate in a for loop using set if extist or put if doesn't

Question

I am having a problem with concurrency trying to use findOneAndUpdate with the mongoose in a for a loop. Don' t focus on the example given but only on its method. Let' s say i have this schema:

var PlayerSchema = new Schema({
    playername: String,
    playerstats:[{
        matchNumber: Number,
        goals: {
            type: Number,
            default: 0
        }
    }]
})

Assuming that I have a year's array and I would like to update goals with another array created with a for loop iteration, what I am looking to do is to have a for loop for x = 0 to 5 where x is the matchNumber and scores[x] represents the number of goals for that match (for the last year). I would like to look in the Player document if the matchNumber exists and if it exists I would use findOneAndUpdate with $set to set scores[x] but it doesn' t exist I would like to create it with scores[x] using $push. Let' s give code for the easiest understanding ("Maradona" will be our player name):

function iterateYears(){
    years = ["2019", "2020"]
    for (var k = 0; k < years.lenght; k++) {
        updateScores(k)
    }
}
function updateScores(k) {
    var scores = []
    for (var y = 0; y < 6; y++) {
        scores.splice(y, 0, (y * 10) + k)
    }
    //at this point we would have the first time scores = [0, 10, 20, 30, 40] and second time scores = [1, 11, 21, 31, 41]
    for (var x = 0; x < 6; x++) {
        Player.findOneAndUpdate({playername: "Maradona", 'playerstats.matchNumber': x}, {$set: {'playerstats.$.goals': scores[x]}}, function(err, matchfound) {
            if (err) {
                console.log("Err found")
            } else {
                if (matchfound == null) {
                    Player.findOneAndUpdate({playername: "Maradona"}, {$push: {playerstats:{matchNumber: x, goals: scores[x]}}}).exec();
                }
            }
        })
    }
}

What I would have at the end is:

{
    playername: "Maradona",
    playerstats:[
        {
            matchNumber: 0,
            goals: 1
        },
        {    
            matchnumber: 1,
            goals: 11
        },
        {
            matchNumber: 2,
            goals: 21
        },
        {
            matchNumber: 3,
            goals: 31
        },
        {
            matchNumber: 4,
            goals: 41
        }
}

but what i get is something like:

{
    playername: "Maradona",
    playerstats:[
        {
            matchNumber: 0,
            goals: 1
        },
        {
            matchNumber: 0,
            goals: 0
        },
        {    
            matchnumber: 1,
            goals: 11
        },
        {    
            matchnumber: 1,
            goals: 10
        },
        {
            matchNumber: 2,
            goals: 21
        },
        {
            matchNumber: 2,
            goals: 20
        },
        {
            matchNumber: 3,
            goals: 31
        },
        {
            matchNumber: 3,
            goals: 30
        },
        {
            matchNumber: 4,
            goals: 41
        },
        {
            matchNumber: 4,
            goals: 40
        }
}

or a mix of them. This happens because of the code will attempt to make the findOneAndUpdate with $set and when the matchNumber is not found (matchfound == null) instead of executing the findOneAndUpdate with $push to create it, it will keep on iterating so will try to do the same with the value of k = 1 (the second year to clarify) and will need to move again to findOneAndUpdate with $push because of it will not be found and so the value with scores = (x * 10) + k is created and after that, it will come back and make the findOneAndUpdate with $push and scores = x * 10. I understand that this is its normal working because of mono threading but I really need to have the desired output. Thanks to all and sorry for the long reading.

Answer 1

The problem there is that findOneAndUpdate is asynchronous. Each iteration through the for loop calls Player.findOneAndUpdate and passes it a callback function. The findOneAndUpdate call returns immediately, and the loop moves on to the next iteration.

At some point in the future, the query to the mongo completes and the callback is called with the return value, since there are multiple asynchronous calls staged, the order they complete is not strictly defined.

You might try using await with each call to findOneAndUpdate to make them synchronous, which will serialize the requests.

If you are using MongoDB 4.2, you can use the pipeline form of update to be able to do both insert and update array elements in an single statement:

.update(
         {playername: "Maradona"},
         [{$set:{
             playerstats:{
                 $concatArrays:[
                     {$filter:{
                         input:"$playerstats",
                         cond:{$ne:["$$this.matchNumber",x]}
                     }},
                     [{matchNumber: x, goals: scores[x] }]
                 ]
             }
          }}]
)

In order to make sure these are applied in the correct order, you could build and array of all of the operations to apply, like:

[
  {updateOne:{
    filter: {playername: "Maradona"},
    update: [{$set:{
             playerstats:{
                 $concatArrays:[
                     {$filter:{
                         input:"$playerstats",
                         cond:{$ne:["$$this.matchNumber",0]}
                     }},
                     [{matchNumber: 0, goals: 1 }]
                 ]
             }
          }}]
 }},
 {updateOne:{
    filter: {playername: "Maradona"},
    update: [{$set:{
             playerstats:{
                 $concatArrays:[
                     {$filter:{
                         input:"$playerstats",
                         cond:{$ne:["$$this.matchNumber",1]}
                     }},
                     [{matchNumber: 1, goals: 11 }]
                 ]
             }
          }}]
 }}

and then pass that array to bulkWrite with the ordered=true option.

Also note that if you are able to build an array of all of the scores that need to be updated, you could use a similar statement to update all of the scores for a single player in a one operation, which should perform a bit better.