MongoDB: Push related data from another collection

Question

I have one collection that consists of generated report data.

Products:

{
  location: 'Spain',
  month: 5,
  year: 2015,
  name: 'Cup',
  price: 100.32,
  type: 1
},
  ...

Then I have the important data:

Report:

{
  location: 'Spain',
  month: 5,
  year: 2015,
  stdPrice: 110.22,
  products : [] // Here is where I'd like to insert related data (location, month, year) 
//from Products
}

The data should be stored back into the DB.

Something like:

 products.forEach(function(product){
   report.forEach(function(data){
    if(product['location'] === data['location'] && product['month'] === data['month'] && product['year'] === data['year']){
       data['products'].push(product);
     }
   });
 });

Anyone know how to achieve this? I was thinking map-reduce would be a good way to do it. I'd like to be able to do this programatically with mongoose as well.

Thanks!

Answer 1

I was able to make this work myself. I did not execute manually after x operations because I didn't get it working myself.

Each group of operations can have at most 1000 operations. If a group exceeds this limit, MongoDB will divide the group into smaller groups of 1000 or less. For example, if the bulk operations list consists of 2000 insert operations, MongoDB creates 2 groups, each with 1000 operations.

Source

    var bulk = mongoose.model('Report').collection.initializeOrderedBulkOp();

    mongoose.model('Product').find({}).exec(function (error, doc) {
        doc.forEach(function (value, index) {
            if (index % 500 === 0) {
                console.log('Current index: ', index);
            }
            bulk.find({
                location: value._doc.location,
                month: {$lte: value._doc.month}
            }).updateOne({
                "$push": {
                    "products": {
                        "location": doc.location,
                        "month": doc.month,
                        "year": doc.year,
                        "name": doc.name,
                        "price": doc.price,
                        "type": doc.type
                    }
                }
            });
        });
        console.log('Bulk Executing');
        bulk.execute(function (err, results) {
            if (err)
                console.error(err);
            else
                console.log(results.toJSON());
        });
    });

Credit to @yogesh for the MongoDB code.

Answer 2

Use mongo bulk operation as below :

 var bulk = db.report.initializeOrderedBulkOp(),
    count = 0;
 db.products.find().forEach(function(doc) {
    bulk.find({
        "location": doc.location,
        "month": doc.month
    }).updateOne({
        "$push": {
            "products": {
                "location": doc.location,
                "month": doc.month,
                "year": doc.year,
                "name": doc.name,
                "price": doc.price,
                "type": doc.type
            }
        }
    });
    count += 2;
    if(count % 500 == 0) {
        bulk.execute();
        bulk = db.report.initializeOrderedBulkOp();
    }
 });
 if(count % 500 !== 0) bulk.execute();

for ref check this