Iterating through array produced by MongoDB aggregation query

Question

Good afternoon all,

I am having a really tough time working with aggregation queries in MongoDB 3.4. I have a problem that is asking me to do push the results of my aggregation query into an empty array called categories which I have been able to do successfully using this code:

var categories = [];

    database.collection("item").aggregate([{
        $group : {
            _id : "$category",
             num : {$sum : 1}
         }},
        {$sort:{_id:1}}]).toArray(function(err, data){

            categories.push(...data);
            callback(categories);
            console.log(categories);
        })

    }

categories looks like this:

[ { _id: 'Apparel', num: 6 },
{ _id: 'Books', num: 3 },
{ _id: 'Electronics', num: 3 },
{ _id: 'Kitchen', num: 3 },
{ _id: 'Office', num: 2 },
{ _id: 'Stickers', num: 2 },
{ _id: 'Swag', num: 2 },
{ _id: 'Umbrellas', num: 2 } ]

Next I have the following task:

  In addition to the categories created by your aggregation query, include a document for category "All" in the array of categories passed to the callback. The "All" category should contain the total number of items across all categories as its value for "num". The most efficient way to calculate this value is to iterate through the array of categories produced by your aggregation query, summing counts of items in each category. 

The problem is that it seems like inside my .toArray() method the data parameter sometimes acts like an array and sometimes not. For example if I wanted to add perhaps just the value of the num key to the categories array like so: categories.push(...data["num"]) I get an error stating undefined is not iterable .

Since I cannot iterate over each data.num key I cannot extract it's value and add it to a running total of all data.num values.

What am I not understanding about what is going on here?

Answer 1

You don't need to use application logic to group data, mongoDB aggregation is made for this task. Add another $group to your query with a new field All that $sum your $num field and $push all documents to a new field called categories :

db.item.aggregate([{
    $group: {
        _id: "$category",
        num: { $sum: 1 }
    }
}, { $sort: { _id: 1 } }, {
    $group: {
        _id: 1,
        All: { $sum: "$num" },
        categories: {
            $push: {
                _id: "$_id",
                num: "$num"
            }
        }
    }
}])

It gives :

{
    "_id": 1,
    "All": 23,
    "categories": [{
        "_id": "Swag",
        "num": 2
    }, {
        "_id": "Office",
        "num": 2
    }, {
        "_id": "Stickers",
        "num": 2
    }, {
        "_id": "Apparel",
        "num": 6
    }, {
        "_id": "Umbrellas",
        "num": 2
    }, {
        "_id": "Kitchen",
        "num": 3
    }, {
        "_id": "Books",
        "num": 3
    }, {
        "_id": "Electronics",
        "num": 3
    }]
}

For consuming the output, data is an array , to access the first element use data[0] :

var categories = [];

database.collection("item").aggregate([{
    $group: {
        _id: "$category",
        num: { $sum: 1 }
    }
}, { $sort: { _id: 1 } }, {
    $group: {
        _id: 1,
        All: { $sum: "$num" },
        categories: {
            $push: {
                _id: "$_id",
                num: "$num"
            }
        }
    }
}]).toArray(function(err, data) {

    var totalCount = data[0]["All"];
    console.log("total count is " + totalCount);

    categories = data[0]["categories"];

    for (var i = 0; i < categories.length; i++) {
        console.log("category : " + categories[i]._id + " | count : " + categories[i].num);
    }
})

Answer 2

What I wanted to achieve was pushing or unshifting as we'll see in a moment an object that looked like this into my categories array:

var allCategory = {
      _id: "All",
      num: [sum of all data.num values]
}

I ended up messing with .reduce() method and used it on the categories array. I got lucky through some console.log -ing and ended up making this:

var categories = [];

    database.collection("item").aggregate([{
        $group : {
            _id : "$category",
             num : {$sum : 1}
         }},
        {$sort:{_id:1}}]).toArray(function(err, data){
            categories.push(...data);
            var sum = categories.reduce(function(acc, val){
                // console.log(acc, val["num"])
                return acc + val["num"]
            },0);

            var allCategory = {
                _id: "All",
                num: sum
            }
            categories.unshift(allCategory)
            callback(categories);
        })

First I use a spread operator to push all the objects from data into categories . Then declare sum which runs .reduce() on categories returning the accumulation of val["num"] which is really data.num (console log is life). I create the allCategory document/object then use .unshift to place it at the beginning of my categories array (this placement was a requirement) then use my callback.

I think it's a hacky way of accomplishing my goal and I had to go through some trial and error as to the correct order of methods and variables in my .toArray() . Yet it worked and I learned something. Thanks for the help @Bertrand Martel .