MongoDB aggregate two collections, return additional field as count

Question

(See edit below)

I am trying to aggregate data from two separate collections within the same MongoDB database.

The "accounts" collection contains user information (cleansed):

{ 
    _id: ObjectId("5c0d64a4224a2900108c005f"),
    "username" : "mike22", 
    "email" : "mike22@<domain>.com",
    "country" : GB,
    "created" : ISODate("2018-11-26T23:37:49.051Z") 
},
{ 
    _id: ObjectId("5a0d64a4527h2880108c0445"),
    "username" : "mike23", 
    "email" : "mike23@<domain>.com", 
    "country" : DE,
    "created" : ISODate("2018-11-26T23:37:49.051Z")
},
{ 
    _id: ObjectId("5a3334a45zzz2884448c0445"),
    "username" : "mike24", 
    "email" : "mike24@<domain>.com", 
    "country" : DE,
    "created" : ISODate("2018-11-26T23:37:49.051Z")
}

The "devices" collection contains device definitions for all users. A user is likely to have many devices defined in this collection and many users devices are in this collection.

A single device within this collection is defined as follows:

{ 
    "_id" : ObjectId("5c10138c73bbe0001018e415"), 
    "capabilities" : [ 
        "BrightnessController", 
        "PowerController" 
    ], 
    "displayCategories" : [ 
        "LIGHT" 
    ], 
    "friendlyName" : "Test1", 
    "description" : "Test device 1", 
    "reportState" : true, 
    "username" : "mike22", 
    "endpointId" : 11, 
    "__v" : 0 
},
{ 
    "_id" : ObjectId("5c10138c73bbe0001018e415"), 
    "capabilities" : [ 
        "PowerController" 
    ], 
    "displayCategories" : [ 
        "SWITCH" 
    ], 
    "friendlyName" : "Test2", 
    "description" : "Test device 2", 
    "reportState" : true, 
    "username" : "mike23", 
    "endpointId" : 12, 
    "__v" : 0 
},
{ 
    "_id" : ObjectId("5c10138c73bbe0001018e415"), 
    "capabilities" : [ 
        "PowerController" 
    ], 
    "displayCategories" : [ 
        "SMARTPLUG" 
    ], 
    "friendlyName" : "Test3", 
    "description" : "Test device 3", 
    "reportState" : true, 
    "username" : "mike22", 
    "endpointId" : 13, 
    "__v" : 0 
}

I'm able to use the aggregate below to show me a count of device per-user:

db.accounts.aggregate([
    { 
        $lookup: {
            from : "devices",
            localField : "username",
            foreignField : "username",
            as : "userdevs"
        },
    },
    { $unwind:"$userdevs" },
    { $group : { _id : "$username", count : { $sum : 1 } } }
 ])

Example output from the data/ aggregate above:

{ "_id" : "mike22", "count" : 2 },
{ "_id" : "mike23", "count" : 1 }

(Note user with no devices is now missing/ should be there with a zero count?!)

However, I want to return all fields for each user plus a new field which shows me the count of devices they have in the "devices" collection. The output I am looking for is as below:

{ 
    "_id" : ObjectId("5c0d64a4224a2900108c005f"),
    "username" : "mike22", 
    "email" : "mike22@<domain>.com",
    "country" : GB,
    "created" : ISODate("2018-11-26T23:37:49.051Z"),
    "countDevices": 2
},
{ 
    "_id" : ObjectId("5a0d64a4527h2880108c0445"),
    "username" : "mike23", 
    "email" : "mike23@<domain>.com", 
    "country" : DE,
    "created" : ISODate("2018-11-26T23:37:49.051Z"),
    "countDevices": 1
},
{ 
    "_id" : ObjectId("5a0d64a4527h2880108c0445"),
    "username" : "mike24", 
    "email" : "mike24@<domain>.com", 
    "country" : DE,
    "created" : ISODate("2018-11-26T23:37:49.051Z"),
    "countDevices": 0
}

---

Edit 16/12: So I am nearly there with the aggregate below. Zero-count users are missing though.

use users
    db.accounts.aggregate([
        { 
            $lookup: {
                from : "devices",
                localField : "username",
                foreignField : "username",
                as : "userdevs"
            },
        },
        { $unwind: "$userdevs"},
        { $group : { _id : {
            _id: "$_id",
            username: "$username",
            email: "$email",
            country: "$country",
            region: "$region",
        },
        countDevices : { $sum : 1 } } }
     ])

---

2nd Edit 16/12:

I have found the aggregate needed below:

db.accounts.aggregate([
  { "$lookup": {
    "from": "devices",
    "let": { "username": "$username" },
    "pipeline": [
      { "$match": {
        "$expr": { "$eq": [ "$$username", "$username" ] }
      }},
      { "$count": "count" }
    ],
    "as": "deviceCount"    
  }},
  { "$addFields": {
    "countDevices": { "$sum": "$deviceCount.count" }
  }}
])

Answer 1

First of All, you can flatten the answer you have got with a projection like below:

{ $project : {
        _id : '$_id._id',
        username : '$_id.username',
        email : '$_id.email',
        country : '$_id.country',
        region : '$_id.region',
        countDevices: 1
    }
}

add this after the $group in your pipeline, you will get your result as you wanted in the question.

About zero-count users, there is a way to handle this in database using mongoDB, as explained in detail here but I do not recommend it, its better that you handle this kind of problem client side.

Answer 2

As-per second edit, the aggregate I used is as below:

db.accounts.aggregate([
  { "$lookup": {
    "from": "devices",
    "let": { "username": "$username" },
    "pipeline": [
      { "$match": {
        "$expr": { "$eq": [ "$$username", "$username" ] }
      }},
      { "$count": "count" }
    ],
    "as": "deviceCount"    
  }},
  { "$addFields": {
    "countDevices": { "$sum": "$deviceCount.count" }
  }}
])