TTL in Mongo DB using C# stop working after expiry index date field is stored as string

Question

I am using the Time To Live option to set the expiry of items in MONGO DB. Previously it was working because the field CreatedAt format used to be BSON Date Format but later we needed to convert our whole object into JSON and then back to BSON before saving it to MONGO DB. After that conversion, CreatedAt is stored as string format and the problem starts with TTL.

DB Configuration:

 private void InitializeSessionCollection(MongoDbOptions dbOptions)
        {
        var collection = _database.GetCollection<Session>(nameof(Session));

        var indexKeysDefinition = Builders<Session>.IndexKeys.Descending(l => l.CreatedAt);
        var indexOptions = new CreateIndexOptions
        {
            ExpireAfter = dbOptions.SessionTimeout,
            Name = "TimeToLiveIndex"
        };
        var model = new CreateIndexModel<Session>(indexKeysDefinition, indexOptions);

        using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)))
        {
            collection.Indexes.CreateOne(model, cancellationToken: cts.Token);
        }
    }

Previously DB stored format: (TTL working with it)

{
  "_id": {
    "$binary": {
      "base64": "RqcDGqCmokyxipJ+jTi7Sw==",
      "subType": "03"
    }
  },
  "CreatedAt": {
    "$date": {
      "$numberLong": "1659538489113"
    }
  }
}

Currently DB stored format:(TTL not working with it)

{
  "_id": {
    "$oid": "62ea84638919f5904e6b6bfd"
  },
 "CreatedAt": "2022-08-03T19:17:03.8851421+05:30"
}

Using the following generic code to save the object into DB:

 public async Task AddDocumentAsync(TDocument document)
        {
            var bsonDocument = SerializeDocument(document);
            await _collection.InsertOneAsync(bsonDocument).ConfigureAwait(false);
        }
        private BsonDocument SerializeDocument(TDocument document)
        {
            var jsonDocument = JsonConvert.SerializeObject(document);
            return BsonSerializer.Deserialize<BsonDocument>(jsonDocument);
        }

Please help me into resolving this issue? Thanks in advance.

Answer 1

In order to use a TTL index, you need a datetime property. For a lot of other reasons, it is preferrable to use datetime properties instead of strings.

You can update the documents in a script with an aggregation pipeline like this:

db.collection.update({
  "CreatedAt": {
    $type: "string"
  }
},
[
  {
    $set: {
      "CreatedAt": {
        $toDate: "$CreatedAt"
      }
    }
  }
],
{
  multi: true
})

You can try the script with this playground .

When storing data, you can use the .NET DateTime directly. MongoDB will store this as a UTC datetime.