InvalidPipelineOperator 用于在 mongodb 中使用 golang 进行聚合和过滤

Question

I have a mongodb collection with the following structure:

{
        "_id" : ObjectId("5e2af47006d5b7820876cc34"),
        "uuid" : "6ec5245e-d512-4496-995d-9a7d1073ff80",
        "beaconid" : "fc775907-f442-43ca-9b86-78fede5f9218",
        "locations" : [
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T13:43:12.895Z")
                },
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T13:47:09.066Z")
                },
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T15:03:02.770Z")
                },
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T15:23:36.891Z")
                }
}
{
     ......
}

I am currently developing an API to get the all the locations with in a particular time range using mongodb's aggregation and filtering functions in golang.

For example: I would like to retrieve all values from ISODate("2020-01-24T13:45:00.066Z") to ISODate("2020-01-24T15:10:00.770Z"), which only retrieve the middle two locations, the output should be following:

{
        "_id" : ObjectId("5e2af47006d5b7820876cc34"),
        "uuid" : "6ec5245e-d512-4496-995d-9a7d1073ff80",
        "beaconid" : "fc775907-f442-43ca-9b86-78fede5f9218",
        "locations" : [
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T13:47:09.066Z")
                },
                {
                        "longitude" : 464.4,
                        "latitude" : 34.8,
                        "establish" : ISODate("2020-01-24T15:03:02.770Z")
                }
}

To achieve this, I have referenced the following two webpages for making the pipeline for the aggregation and filtering:

https://github.com/simagix/mongo-go-examples/blob/5a04bab8b677e7b160dbe4c327f7ac68efb83ba5/examples/aggregate_reduce_test.go

How to write bson form of mongo query in golang?

At first I have trouble using the github's mdb.MongoPipeline(pipeline), so I decide to use the github's pipeline format and use it in the form of second answer of stack overflow of mongo.Pipeline{...}

The resulting pipeline becomes:

new_pipeline = mongo.Pipeline{
    {{"$match", bson.D{
         {"beaconID", r.URL.Query()["beaconid"][0]},
    }}},
    {{"$project", bson.D{
         {"locations", bson.D{
              {"$filter", bson.D{
                  {"input", "$locations"},
                  {"as", "locations"},
                  {"cond", bson.D{
                       {"$and", bson.D{
                           {"$lte", bson.D{{"locations.establish", r.URL.Query()["rangeend"][0]}}},
                           {"$gtd", bson.D{{"locations.establish", r.URL.Query()["rangebegin"][0]}}},
                       }},
                  }},
              }},
          }},
     }}},
     {{"$unwind", "$locations"}},
}

opts := options.Aggregate()
cursor, err := collection.Aggregate(context.TODO(), new_pipeline, opts)

However, by running the program there is an error which I don't know how to solve:

(Location15983) An object representing an expression must have exactly one field: { $lte: { $$locations.establish: new Date(1580122004573) }, $gtd: { $$locations.establish: new Date(1578826004573) } }

And then when trying to debug by testing different pipeline cases, this pipeline causes another problem:

...
{"as", "locations"},
      {"cond", bson.D{
            {"$lte", bson.D{{"$$locations.establish", r.URL.Query()["rangeend"][0]}}},
      }},
}},
...

Error is: (InvalidPipelineOperator) Unrecognized expression '$$locations.establish'

Any ideas why these two error happen? And how to fix it? Thank You.

Answer 1

Any ideas why these two error happen?

You're getting those errors because of the missing array operator. The operator $and requires an array, which should be represented as bson.A .

And how to fix it?

Please see the following pipeline:

// Create a date object using time.Parse()
lteDate, err := time.Parse(time.RFC3339, "2020-01-24T15:10:00.770Z")
if err!=nil {log.Fatal(err)}
gteDate, err := time.Parse(time.RFC3339, "2020-01-24T13:45:00.066Z")
if err!=nil {log.Fatal(err)}

// MongoDB Aggregation Pipeline Stages
matchStage := bson.D{
    {"$match", bson.D{
        {"beaconid", "fc775907-f442-43ca-9b86-78fede5f9218"},
    }},
}

projectStage := bson.D{
    {"$project", bson.D{
        {"locations", bson.D{
            {"$filter", bson.D{
                {"input", "$locations"}, 
                {"as", "locations"}, 
                {"cond", bson.D{
                    {"$and", bson.A{
                            bson.D{{"$lte", bson.A{"$$locations.establish", lteDate}}},
                            bson.D{{"$gte", bson.A{"$$locations.establish", gteDate}}},
                    }},
                }}, 
            }}, 
        }}, 
    }}, 
}

unwindStage := bson.D{
    {"$unwind", "$locations"},
}
pipeline := mongo.Pipeline{matchStage, projectStage, unwindStage}

The snippet above is written using the current version of mongo-go-driver (v1.2.x)