AzureDocumentDB MongoDB protocol support: Failing to create TTL index

Question

I'm currently using Spring Data MongoDB to abstract MongoDB operations, and using an Azure DocumentDB database with MongoDB protocol support. I've also run into this using the latest MongoDB Java driver by itself.

There is an issue with setting a TTL index in this process.

I'm receiving the following exception.

`Caused by: com.mongodb.CommandFailureException: { "serverUsed" : "****-****-test.documents.azure.com:****" , "_t" : "OKMongoResponse" , "ok" : 0 , "code" : 2 , "errmsg" : "The 'expireAfterSeconds' option has invalid value. Ensure to provide a nonzero positive integer, or '-1'` which means never expire." , "$err" : "The 'expireAfterSeconds' option has invalid value. Ensure to provide a nonzero positive integer, or '-1' which means never expire."}
at com.mongodb.CommandResult.getException(CommandResult.java:76)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:140)
at com.mongodb.DBCollectionImpl.createIndex(DBCollectionImpl.java:399)
at com.mongodb.DBCollection.createIndex(DBCollection.java:597)

This is a simple representation of the POJO that I'm using.

public class Train{
   @JsonProperty
   private String id;

   @JsonProperty("last_updated")
   @Indexed(expireAfterSeconds = 1)
   private Date lastUpdated;

   // Getters & Setters
   .
   .
   .
}

This was my initial approach of initializing the index (via the @Indexed annotation).

I've also attempted to initialize the index via:

mongoTemplate.indexOps(collection.getName())
                .ensureIndex(new Index("last_updated", Sort.Direction.DESC)
                .expire(1, TimeUnit.SECONDS));

Both ways of setting the index throw that same execption.

I've also seen an error saying that it can only be done on the '_ts' field. I think this is due to Azure DocumentDB using the '_ts' field for it's own TTL operation. So I've tried the following with the same results:

• Added a new field, Long '_ts', to the pojo and tried with the annotation.
• Attempted to set the index via the ensureIndex method with '_ts' field.
• Did the same things above but changing the type of '_ts' to Date.

I'm new to these technologies (DocumentDB and MongoDB), so I'm probably missing something obvious.

Any thoughts?

Answer 1

Revisiting my question I had posted a while back to reply with the solution that I figured out.

Note that DocumentDB has been renamed to CosmosDB since I posted this question.

There was an issue with type casting in either the Spring framework or the CosmosDB/DocumentDB platform side. Despite documentation saying it needs an integer, you actually need to pass it a double value.

I'm using something along the lines of the following and it works

dcoll.createIndex(new BasicDBObject("_ts", 1)
                    , new BasicDBObject("expireAfterSeconds", 10.0));

Answer 2

This works for me.

db.coll.createIndex( { "_ts": 1 }, { expireAfterSeconds: 3600 } )

Answer 3

Using Spring Boot and Spring Data in a ContextListener :

private final MongoTemplate mongoTemplate;

@EventListener(ContextRefreshedEvent.class)
public void ensureIndexes() {
  mongoTemplate.indexOps(YourCollection.class)
    .ensureIndex(
      new Index()
        .on("_ts", Sort.Direction.ASC)
        .expire(30, TimeUnit.SECONDS)
    );
}

The Azure console is a little confusing in the "Settings" options for the collection, it says "To enable time-to-live (TTL) for your collection/documents, create a TTL index" - it will say this even if you do already have such an index created.

Confirmed working January 2023 with Spring Boot 2.7.6 and Azure CosmosDB.

Answer 4

According to the blog DocumentDB now supports Time-To-Live (TTL) & the section Setting TTL on a document of the offical tutorial Expire data in DocumentDB collections automatically with time to live, the TTL setting on Azure DocumentDB is different from MongoDB, although Azure support do the operater on DocumentDB via MongoDB drive & spring-data in Java.

The TTL should be defined as a property on DocumentDB to set a nonzero positive integer value, so please try to change your code as below.

public class Train{
   @JsonProperty
   private String id;

   @JsonProperty("last_updated")
   private Date lastUpdated;

   /*
    * Define a property as ttl and set the default value 1.
    */
   @JsonProperty("ttl")
   private int expireAfterSeconds = 1;

   // Getters & Setters
   .
   .
   .
}

Hope it helps. Any concern, please feel free to let me know.

---

Update : Notice the below content at https://docs.microsoft.com/en-us/azure/documentdb/documentdb-time-to-live#configuring-ttl .

By default, time to live is disabled by default in all DocumentDB collections and on all documents.

So please first enable the feature TIME TO LIVE on Azure portal as the figure below, or follow the above link to enable it programmatically.