Upsert Cosmos item TTL using Azure Data Factory Copy Activity

Question

I have a requirement to upsert data from REST API to Cosmos DB and also maintain the item level TTL for particular time interval.

I have used ADF Copy activity to copy the data but for TTL, used additional custom column at source side with hardcoded value 30.

Noticed that time interval (seconds) updating as string instead of integer. Hence failing with the below error.

Details Failure happened on 'Sink' side. ErrorCode=UserErrorDocumentDBWriteError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Documents failed to import due to invalid documents which violate some of Cosmos DB constraints: 1) Document size shouldn't exceeds 2MB; 2) Document's 'id' property must be string if any, and must not include the following charaters: '/', '', '?', '#'; 3) Document's 'ttl' property must not be non-digital type if any.,Source=Microsoft.DataTransfer.DocumentDbManagement,'

ttl Mapping between Custom column to cosmos DB

When i use ttl1 instead of ttl, it is getting success and value stored as string.

Any suggestion please?

Answer 1

Yes, that's the issue with additional columns in Copy activity. Even of you set it to int, it will change to string at the source.

The possible workaround is to create a Cosmos DB trigger in Azure function and add 'TTL' there.