PyMongo组通过多个键

Question

With PyMongo, group by one key seems to be ok:

results = collection.group(key={"scan_status":0}, condition={'date': {'$gte': startdate}}, initial={"count": 0}, reduce=reducer)

results:

{u'count': 215339.0, u'scan_status': u'PENDING'} {u'count': 617263.0, u'scan_status': u'DONE'}

but when I try to do group by multiple keys I get an exception:

results = collection.group(key={"scan_status":0,"date":0}, condition={'date': {'$gte': startdate}}, initial={"count": 0}, reduce=reducer)

How can I do group by multiple fields correctly?

Answer 1

If you are trying to count over two keys then while it is possible using .group() your better option is via .aggregate() .

This uses "native code operators" and not the JavaScript interpreted code as required by .group() to do the same basic "grouping" action as you are trying to achieve.

Particularly here is the $group pipeline operator:

result = collection.aggregate([
    # Matchn the documents possible
    { "$match": { "date": { "$gte": startdate } } },

    # Group the documents and "count" via $sum on the values
    { "$group": {
        "_id": {
            "scan_status": "$scan_status",
            "date": "$date"
        },
        "count": { "$sum": 1 }
    }}
])

In fact you probably want something that reduces the "date" into a distinct period. As in:

result = collection.aggregate([
    # Matchn the documents possible
    { "$match": { "date": { "$gte": startdate } } },

    # Group the documents and "count" via $sum on the values
    { "$group": {
        "_id": {
            "scan_status": "$scan_status",
            "date": {
                "year": { "$year": "$date" },
                "month": { "$month" "$date" },
                "day": { "$dayOfMonth": "$date" }
            }
        },
        "count": { "$sum": 1 }
    }}
])

Using the Date Aggregation Operators as shown here.

Or perhaps with basic "date math":

import datetime
from datetime import date

result = collection.aggregate([
    # Matchn the documents possible
    { "$match": { "date": { "$gte": startdate } } },

    # Group the documents and "count" via $sum on the values
    # use "epoch" "1970-01-01" as a base to convert to integer
    { "$group": {
        "_id": {
            "scan_status": "$scan_status",
            "date": {
                "$subtract": [
                    { "$subtract": [ "$date", date.fromtimestamp(0) ] },
                    { "$mod": [
                        { "$subtract": [ "$date", date.fromtimestamp(0) ] },
                        1000 * 60 * 60 * 24
                    ]}
                ]
            }
        },
        "count": { "$sum": 1 }
    }}
])

Which will return integer values from "epoch" time instead of a compisite value object.

But all of these options are better than .group() as they use native coded routines and perform their actions much faster than the JavaScript code you need to supply otherwise.