Mongo aggregate several documents into one

Question

I'm trying to merge a few complicated documents via the mongodb.collection.aggregate() command.

Let's say that I want to merge x of the collection's document (in the following example: x=2):

[
  {
    "_id": 1,
    "Data": {
      "children": {
        "1": {
          "name": "appear_only_in_first_doc",
          "cost": 1,
          "revenue": 4.5,
          "grandchildren": {
            "1t9dsqdqdvoj8pdppxjk": {
              "cost": 0,
              "revenue": 1.5
            }
          }
        },
        "2": {
          "name": "appear_in_both_docs",
          "cost": 2,
          "revenue": 7,
          "grandchildren": {
            "jesrdt5qwef2222dgt": {
              "cost": 1,
              "revenue": 3
            },
            "klh352hk5367kf": {
              "cost": 2,
              "revenue": 7
            }
          }
        }
      }
    }
  },
  {
    "_id": 2,
    "Data": {
      "children": {
        "2": {
          "name": "appear_in_both_docs___but_diff_name",
          "cost": 9,
          "revenue": 7,
          "grandchildren": {
            "aaaaaaaaa": {
              "cost": 3,
              "revenue": 2
            },
            "jesrdt5qwef2222dgt": {
              "cost": 6,
              "revenue": 5
            }
          }
        },
        "3": {
          "name": "appear_only_in_last_doc",
          "cost": 4,
          "revenue": 2,
          "grandchildren": {
            "cccccccccccc": {
              "cost": 4,
              "revenue": 2
            }
          }
        }
      }
    }
  }
]

Challenges:

1. The keys under the "children" and "grandchildren" keys are dynamic and unknown while writing the query.
2. If a child or grandchild appears only in one doc (eg "1", "3", "1t9dsqdqdvoj8pdppxjk", "klh352hk5367kf", "aaaaaaaaa" and "cccccccccccc") - it should also appear in the final result.
3. If a child appears in more than one docs (eg "2" and "jesrdt5qwef2222dgt") - it should appear as one in the final result. The fields "cost" and "revenue" should be summed, and the last "name" field should be taken.

I've seen the following solutions:

1. unionWith - Irrelevant, unions 2 different collections.
2. merge - Irrelevant, cannot sum values of fields that appear in more than once (it takes the last instead).
3. mergeObjects - Irrelevant, cannot sum values of fields that appear in more than once (it takes the last instead).

The final result should look like this:

{
  "Data": {
    "children": {
      "1": {
        "name": "appear_only_in_first_doc",
        "cost": 1,
        "revenue": 4.5,
        "grandchildren": {
          "1t9dsqdqdvoj8pdppxjk": {
            "cost": 0,
            "revenue": 1.5
          }
        }
      },
      "2": {
        "name": "appear_in_both_docs___but_diff_name",
        "cost": 11,
        "revenue": 14,
        "grandchildren": {
          "aaaaaaaaa": {
            "cost": 3,
            "revenue": 2
          },
          "jesrdt5qwef2222dgt": {
            "cost": 7,
            "revenue": 8
          },
          "klh352hk5367kf": {
            "cost": 2,
            "revenue": 7
          }
        }
      },
      "3": {
        "name": "appear_only_in_last_doc",
        "cost": 4,
        "revenue": 2,
        "grandchildren": {
          "cccccccccccc": {
            "cost": 4,
            "revenue": 2
          }
        }
      }
    }
  }
}

Answer 1

This is little lengthy process, might be there will be some easy one, I am just sharing the process,

• $project to convert children to array format (k,v)
• $unwind deconstruct children array
• $group by children key and do sum of cost and revenue , and get last name using $last
• $unwind deconstruct grandchildren array
• $addFields to convert grandchildren to array format (k,v)
• $unwind deconstruct grandchildren array

db.collection.aggregate([
  { $project: { "Data.children": { $objectToArray: "$Data.children" } } },
  { $unwind: "$Data.children" },
  {
    $group: {
      _id: "$Data.children.k",
      name: { $last: "$Data.children.v.name" },
      cost: { $sum: "$Data.children.v.cost" },
      revenue: { $sum: "$Data.children.v.revenue" },
      grandchildren: { $push: "$Data.children.v.grandchildren" }
    }
  },
  { $unwind: "$grandchildren" },
  { $addFields: { grandchildren: { $objectToArray: "$grandchildren" } } },
  { $unwind: "$grandchildren" },

• $group by children key and grandchildren key, count the sum of cost and revenue of grandchildren

  {
    $group: {
      _id: {
        ck: "$_id",
        gck: "$grandchildren.k"
      },
      cost: { $first: "$cost" },
      revenue: { $first: "$revenue" },
      name: { $first: "$name" },
      grandchildren_cost: { $sum: "$grandchildren.v.cost" },
      grandchildren_revenue: { $sum: "$grandchildren.v.revenue" }
    }
  },

• $group by children key and re-construct grandchildren array

  {
    $group: {
      _id: "$_id.ck",
      cost: { $first: "$cost" },
      revenue: { $first: "$revenue" },
      name: { $last: "$name" },
      grandchildren: {
        $push: {
          k: "$_id.gck",
          v: {
            cost: "$grandchildren_cost",
            revenue: "$grandchildren_revenue"
          }
        }
      }
    }
  },

• $group by null and re-construct children array and convert grandchildren to object from (k,v) array using $arrayToObject

  {
    $group: {
      _id: null,
      children: {
        $push: {
          k: "$_id",
          v: {
            name: "$name",
            cost: "$cost",
            revenue: "$revenue",
            grandchildren: { $arrayToObject: "$grandchildren" }
          }
        }
      }
    }
  },

• $project to convert children to object using $arrayToObject

  {
    $project: {
      _id: 0,
      "Data.children": { $arrayToObject: "$children" }
    }
  }
])

Playground