MongoDB $sort aggregation

Question

I have data base with user's activities and I'd like to count active users and number of activities that they made for each month, then sort result first by year, and each year sort by month!

I've got query:

query = {
        "activities": {
            "$exists": 1
        },
        "activities.started": {
            "$exists": 1,
            "$type": MONGODB_DATE_TYPE,
            "$gte": datetime(2011, 6, 1),
            "$lte": datetime(2013, 10, 1)
        }
    }

Then, I made pipeline like this:

pipeline = [
        {'$project': {
            '_id': 1,
            'activities': 1
        }},
        {'$unwind': "$activities"},
        {'$match': query},
        {'$group': {
            '_id': {"y": {"$year": "$activities.started"},
                    "m": {"$month": "$activities.started"}},
            'users': {'$addToSet': "$_id"},
            'activities_count': {"$sum": 1},
        }},
        {"$sort": {
            "_id.y": 1,
            "_id.m": 1,
        }}
    ]

    results = col.aggregate(pipeline)
    results = results.get("result", [])

But, this $sort operation doesn't work good, It did not sort by _id.y and _id.m , only by _id.m !

I've been following this link for mongodb $sort and there is a example that shows that's possible to sort by two values there .

So, my question is : How can I order result first by year, then each year by month?

EDIT

For example when I put pdb below result variable and type function:

for res in results : print res["_id"]

I got this results :

{u'y': 2012, u'm': 1}
{u'y': 2013, u'm': 1}
{u'y': 2012, u'm': 2}
{u'y': 2013, u'm': 2}
{u'y': 2012, u'm': 3}
{u'y': 2013, u'm': 3}
{u'y': 2012, u'm': 4}
{u'y': 2013, u'm': 4}
{u'y': 2012, u'm': 5}
{u'y': 2013, u'm': 5}
{u'y': 2011, u'm': 6}
{u'y': 2012, u'm': 6}
{u'y': 2013, u'm': 6}
{u'y': 2011, u'm': 7}
{u'y': 2012, u'm': 7}
{u'y': 2013, u'm': 7}
{u'y': 2011, u'm': 8}
{u'y': 2012, u'm': 8}
{u'y': 2013, u'm': 8}
{u'y': 2011, u'm': 9}
{u'y': 2012, u'm': 9}
{u'y': 2013, u'm': 9}
{u'y': 2011, u'm': 10}
{u'y': 2012, u'm': 10}
{u'y': 2011, u'm': 11}
{u'y': 2012, u'm': 11}
{u'y': 2011, u'm': 12}
{u'y': 2012, u'm': 12}

Answer 1

Like @JohnnyHK mentioned in the comments python dictionaries (unlike js objects) are unordered. If you want to use sorting on curosr you would use something like this db.foo.find().sort([("foo", 1), ("bar": 1)])) but this syntax is not supported in aggregation $sort . You can use bson.son.SON object or collections.OrderedDict instead:

from bson.son import SON

pipeline = [
        {'$project': {
            '_id': 1,
            'activities': 1
        }},
        {'$unwind': "$activities"},
        {'$match': query},
        {'$group': {
            '_id': {"y": {"$year": "$activities.started"},
                    "m": {"$month": "$activities.started"}},
            'users': {'$addToSet': "$_id"},
            'activities_count': {"$sum": 1},
        }},
        {"$sort": SON([
            ("_id.y", 1),
            ("_id.m", 1)
        ])}
    ]

EDIT

Actually I think that in your case this should be enough:

pipeline = [
       {'$project': {
           '_id': 1,
           'activities': 1
       }},
       {'$unwind': "$activities"},
       {'$match': query},
       {'$group': {
           '_id': {"y": {"$year": "$activities.started"},
                   "m": {"$month": "$activities.started"}},
           'users': {'$addToSet': "$_id"},
           'activities_count': {"$sum": 1},
       }},
       {"$sort": {"_id": 1}}
   ]

If sort filed is document MonogoDB seems to perform sorting field by field. Order of the fields in a document can change during updates and in the general case it wouldn't work. Here however order of fields is defined in the $group phase and document are not modified after that so it shouldn't be a problem.

Sorting by embedded document - shell example:

> db.bar.insert({foobar: {foo: 2012, bar: 1}})
> db.bar.insert({foobar: {foo: 2012, bar: 5}})
> db.bar.insert({foobar: {foo: 2012, bar: 3}})
> db.bar.insert({foobar: {foo: 2010, bar: 5}})
> db.bar.insert({foobar: {foo: 2010, bar: 1}})
> db.bar.insert({foobar: {foo: 2013, bar: 5}})
> db.bar.insert({foobar: {foo: 2013, bar: 3}})
> db.bar.find({}, {_id: 0}).sort({foobar: 1})
{ "foobar" : { "foo" : 2010, "bar" : 1 } }
{ "foobar" : { "foo" : 2010, "bar" : 5 } }
{ "foobar" : { "foo" : 2012, "bar" : 1 } }
{ "foobar" : { "foo" : 2012, "bar" : 3 } }
{ "foobar" : { "foo" : 2012, "bar" : 5 } }
{ "foobar" : { "foo" : 2013, "bar" : 3 } }
{ "foobar" : { "foo" : 2013, "bar" : 5 } }