当某个日期小于新的Date（）时，如何从mongodb集合中删除文档

Question

I'm trying to delete (while importing new documents) some documents from my MongoDB collection, when some date in the document is smaller than new Date() .

Everytime I import a document, I check, if there is a document with the same STATION_ID that needs to be deleted.

Now I'm doing it like this using mongoose:

function doTheStuff(stationid, time, callback) {
  OwnMetarDoc.find({
    STATION_ID: stationid[0]
  }, function(err, entry) {
    var isTaf = false;
    if (entry[0] != undefined) {
      if (entry[0].TIME) {
        var entryObjectTimerec = entry[0].TIME;
        var isTaf = true;
      };
      var d = new Date;
      var month = d.getUTCMonth();
      var year = d.getUTCFullYear();
      var timeRecConverted = new Date(year, month, parseInt(entryObjectTimerec.substring(0, 2)),
          parseInt(entryObjectTimerec.substring(2, 4)), parseInt(entryObjectTimerec.substring(4, 6)));
      //The relevant part
      if (timeRecConverted.getTime() < new Date() && isTaf == true) {
        OwnMetarDoc.remove({
          _id: entry[0]._id
        }, function(err, result) {
          if (err) {
            console.log(err);
          } else {
            console.log('old taf entry removed');
          };
        });
      }
      if (typeof callback === 'function') {
          callback()
      };
    };
  })
};

But this takes way too long when i have over 100'000 entries and multiple callbacks.

Does anybody knows if there is a way to achieve this quickly and automatically?

I know how to do it when I have to delete a document that is older than fe eight hours within the Schema like this:

var ownMetarSchema = new Schema({
  STATION_ID: String,
  TIMERECASDATE: {
    type: Date,
    expires: 28800
  }
})

But I don't know how to do it with my previous example.

ps: I'm relatively new to Javascript and you may need some more information, just ask me please.

Answer 1

If I understand, you want to update your document. You should read the mongo docs. Here : http://docs.mongodb.org/manual/tutorial/modify-documents/ You can just replace all value with your new value. Hope that helps.

Answer 2

I would assume that you would have used the TTL part, if you know how long your weather forecasts would be. Since you haven't used TTL , that means your forecast could be one day in the future or even one month (just for the discussion).

You are finding all the documents with STATION_ID and the parsing through it. Instead, you can do the following:

If you have a field date with the exact timestamp, then just run

db.collection.remove({'date':{'$lt': new Date()} })

http://docs.mongodb.org/manual/reference/method/db.collection.remove/

This will delete all the documents where the date is less than the current date. If you want, you can add the station_id field also. But for weather, I don't think that is necessary.

I would recommend to do this delete in a separate thread/cron job, instead of doing it when you are pushing the data.