Node.js and for loops acting weird

Question

I am working on a project for my internship, where they are using node.js and mongoDB in their back-end. This set up really doesn't work so well at the moment, because we in fact have relational data and mongoDB really isn't the best for that.

My problem is, I have to get some data from the DB that require a "join" on two tables. MongoDB doesn't allow this, so I am trying to do it manually. I have to find a gig, and for each job type (JobTypeLine) the gig has, find the info about staff members invited. RolodexMembersInterested is an array that contains IDs which allows us to find the staff info in the RolodexMember "table".

The json structures look like this:

    {
        _id: {type: Schema.ObjectId},
        CreatedAt: {type: Date, default: Date.now()},
        EventName: {type: String, required: true},
        Headline: {type: String, required: true},
        Date: {type: Date, required: false},
        EndDate: {type: Date, required: false},
        Address: {type: String, required: false},
        Description: {type: String, required: true},
        CompanyName: {type: String, required: true},
        CompanyID: {type: String, required: true},
        Exposed: {type: Boolean, default: false},
        HashTags: {type: []},
        JobTypeLine: {type: [
            {
                JobType: {type: String, required: true},
                Briefing: {type: String, required: false},
                Quantity: {type: Number, required: true},
                StartTime: {type: String, required: true},
                EndTime: {type: String, required: true},
                NumberOfHours: {type: String, required: false},
                HourlySalary: {type: Number, required: false},
                RolodexMembersInvited: {type: [], default: []},
                RolodexMembersAssigned: {type: [], default: []},
                RolodexMembersInterested: {type: [], default: []}
            }
        ]},
    },

    {
        _id:{type:Schema.ObjectId},
        companyId: {type: String, required: true},
        userId: {type: String, required: false},
        userEmail: {type: String, required: true},
        comment: {type: String, required: false},
        payRate: {type: Number, required: false},
        name: {type: String, required: false},
        phone: {type: String, required: false},
        pictureUrl: {type: String, required: false},
        isHot: {type: Boolean, required: false},
        isInPaySystem: {type: Boolean, required: false},
        numberOfPreviousGigs: {type: Number, required: true, default: 0},
        previousGigs: [],
        otherAttributes: [
            {
                attributeName: {type: String, required: false },
                attributeValue: {type: String, required: false }
            }
        ],
    })

So I wrote a method to "simulate" my join :

    exports.getMembersInvitedToGig = function(req,res) {

        var jobTypes = [];
        var rolodexInvited = [];
        gigsModel.findOne({_id: req.body.gigId}, function(err, gig) {
            if(err) {
                console.log(err);
                res.send(404, "Couldn't find the gig with id: " + req.body.gigId);
            } else {
                for(var i=0;i<gig.JobTypeLine.length;i++){
                    rolodexInvited = [];
                    for(var j=0;j<gig.JobTypeLine[i].RolodexMembersInvited.length;j++) {
                        var userId = gig.JobTypeLine[i].RolodexMembersInvited[j];
                        getUserData(userId, function(data,success) {
                            if (success) {
                                rolodexInvited[j] = data;
                            }
                        });
                    }
                    jobTypes[i] = rolodexInvited;
                }
                res.send(200,jobTypes);
            }
        });
    }

    function getUserData (userId, callback) {
        var RolodexItem = mongoose.model('RolodexItems');
        RolodexItem.findOne({ _id: userId }, function (err, doc) {
            if (err) {
                console.log("There was an error in finding the rolodex item: " + err);
            } else {
                    callback(doc,true);
            }
        });
    }

My problem is that the method return and empty array filled other empty arrays (as many as I have jobTypes). I know the getUserData function works just fine and finds the right staff info etc. the problem is, it seems like node doesn't wait for the data to be returned inside the for(j) loop and just continues on (and eventually ends gets to the res.send(200,jobTypes) while jobTypes is actually still empty. I have tried adding a callback to the getUserData to solve that problem, but it didn't change anything.

(Now I know using a relational database would solve the problem all together, but I don't really get a say in this, so I have to make this work as it is). There might also be some really bad practices in there, as I just based myself on functions previously made by other people, and I have noticed they weren't really good at it. I have never worked with node.js previously though, so I haven't had time to look as some proper practices/etc. (Feel free to point out anything that is badly written, I'd love to learn the "good" way to do it)

Answer 1

The problem is that your getUserData function is asynchronous (because findOne is asynchronous). So your for loop does not wait for your getUserData function to complete. You should use the async library (have a look to the each function) : https://github.com/caolan/async