How to run an aggregate query via mongo_go_driver that has javascript in it?

Question

I'm using the mongo-go-driver ( https://godoc.org/github.com/mongodb/mongo-go-driver/mongo ) and I'm trying to do the equivalent of

db.getCollection('mycollection').aggregate([
    { $lookup: {
        from: "anothercollection",
        localField: "_id",
        foreignField: "foreignID",
        as: "matched_docs"
    }},
    { $match: { "matched_docs": { $eq: [] } } },
    { $project: { "matched_docs": 0 } },
    { $match: {"dateTimeGMT":{$lt: (new Date(Date.now()-1000*60*60*24)).toISOString()}} }
])

I can't figure out how to put the Javascript commands in using this method.

pipeline := bson.NewArray(
    bson.VC.DocumentFromElements(
        bson.EC.SubDocumentFromElements( 
        //yada, yada, yada...
cursor, err := collection.Aggregate(ctx, pipeline)

(In general, I dislike this method, anyway. I want to be able to design queries in Robo 3T and copy them to my code just like I do with MySQL Workbench and PHP)

This method yields an empty *bson.Array in pipeline

pipelineJSON := `[
    { $lookup: {
        from: "anothercollection",
        localField: "_id",
        foreignField: "interactionID",
        as: "matched_docs"
    }},
    { $match: { "matched_docs": { $eq: [] } } },
    { $project: { "matched_docs": 0 } },
    { $match: {"dateTimeGMT":{$lt: (new Date(Date.now()-1000*60*60*24)).toISOString()}} }
]`
pipeline, err = bson.ParseExtJSONArray(pipelineJSON)

I'd really love it if there were a way to send Mongo a command as a string (like I was typing it into Robo 3T) and get a *mongo.Cursor back. Is there a better driver (that is still supported by someone) that I should use instead? Do I need to code my own?

Thanks!

Answer 1

I can't figure out how to put the Javascript commands in using this method.

Although I understand this is not the preferred method for your use case, this is how to construct the aggregation pipeline as bson.D using mongo-go-driver v1.0:

pipeline := mongo.Pipeline{
    {{"$lookup", bson.D{
        {"from", "tour"}, 
        {"localField", "_id"}, 
        {"foreignField", "foreignID"}, 
        {"as", "matched_docs"},
    }}},
    {{"$match", bson.D{
        {"matched_docs", bson.D{
            {"$eq", bson.A{}}},
        }, 
    }}},
    {{"$project", bson.D{
        {"matched_docs", 0}, 
    }}},        
    {{"$match", bson.D{
        {"dateTimeGMT", bson.D{
            {"$lt", time.Now().AddDate(0, 0, -1).UTC().Format(time.RFC3339)},
            }, 
        },
    }}},  
}

{ $project: { "matched_docs": 0 } },

You can also combined the two $match into a single pipeline stage instead, and then attach $project at the end. For example:

db.collection.aggregate([
    { "$lookup":{
        "from":"anothercollection",
        "localField":"_id",
        "foreignField":"foreignID",
        "as":"matched_docs"}
    }, 
    { "$match": { "matched_docs": { "$eq": [] }, 
                  "datetimegmt": { "$lt": (new Date(Date.now()-1000*60*60*24)).toISOString() } 
                } 
    }, 
    { "$project": { "matched_docs": 0 } }
]);

{ $match: {"dateTimeGMT":{$lt: (new Date(Date.now()-1000*60*60*24)).toISOString()}} }

Based on the $match value, it looks like you're storing date as string instead of a Date object. I would suggest to store dates as the proper Date objects for better index performance.

Using ParseExtJSONArray DOES work, but, unlike Mongo Shell, you cannot include JS or unquoted aggregation stages / expression operators

mongo shell provides some convienient methods/types ie ObjectID() , etc to construct MongoDB Extended JSON . The mongo shell is more than just a JavaScript shell.

If your aim just to evaluate a JavaScript expression, you can utilise a JavaScript interpreter for Go (otto). A rough example would be:

// Note the unquoted fields, as JSON is JavaScript native. 
raw := `[
    { "$lookup": {
        from: "anothercollection",
        localField: "_id",
        foreignField: "foreignID",
        as: "matched_docs"
    }},
    { $match: { "matched_docs": { $eq: [] },
                "dateTimeGMT":{$lt: (new Date(Date.now()-1000*60*60*24)).toISOString() }, 
              }
    },
    { $project: { "matched_docs": 0 } },
]`
vm := otto.New()
// Evaluate JS expression
jsvalue, err := vm.Eval(raw)

// Export to Go interface{}
output, err := jsvalue.Export()

// Convert interface{} to bson.Document bytes 
bsonbytes, err := bson.Marshal(output)

// Convert bson.Document bytes to bson.Document
pipeline, err := bson.UnmarshalDocument(bsonbytes)

Note as mentioned above, there are some objects that will not be recognised by a normal JavaScript interpreter ie ObjectId()

I want to be able to design queries in Robo 3T and copy them to my code just like I do with MySQL Workbench and PHP

Although currently does not support Go, worth mentioning that MongoDB Compass has a feature to Export Query to Language . Current version (1.15) supports Java, Node, C# and Python3. Hopefully Go would be in the future.

Answer 2

Using ParseExtJSONArray DOES work, but, unlike Mongo Shell, you cannot include JS or unquoted aggregation stages / expression operators. Note that the ISO date for the comparison is calculated in golang and the aggregation stages and aggregtion expression operators have been surrounded by quotes.

pipelineJSON := fmt.Sprintf(`[
    { "$lookup": {
        "from": "another_collection",
        "localField": "_id",
        "foreignField": "interactionID",
        "as": "matched_docs"
    }},
    { "$match": { "matched_docs": { "$eq": [] } } },
    { "$project": { "matched_docs": 0 } },
    { "$match": {"dateTimeGMT":{"$lt": "%s"}} }
]`, time.Now().AddDate(0, 0, -1).UTC().Format(time.RFC3339))
pipeline, err = bson.ParseExtJSONArray(pipelineJSON)

With the driver I asked about, that's the answer.