how to set retention period for azure storage tables?

Question

I have a azure table used for metric data collection and I want to set some retention period for eg: if retention period is 7 then table should have last 7 day data in it.

Is there any option available.

Answer 1

I have a azure table used for metric data collection and I want to set some retention period for eg: if retention period is 7 then table should have last 7 day data in it.

Is there any option available.

As of today, No such thing is available for Azure Tables (or for Blobs and Files). You would need to roll something out of your own. One possible solution would be to write a background job (either as WebJob or Azure Function) that runs periodically and finds the entities that have been created before "x" days (7 in your example) and delete those entities.

Answer 2

There's no way in Azure storage to specify auto-archival. What I typically do with my storage tables, is create monthly storage tables. Data gets inserted according to its timestamp into appropriate "Orders201701" or "Orders201612", etc. tables. Then purge old tables when they contain data that's no longer relevant.

You can make your data access layer abstract the consumer away from this complexity by auto-joining results from queries that span multiple months.

If you're dealing with native Azure diagnostics storage, you're out of luck. However, you can just delete those tables once in a while and Azure will recreate them.

Do keep in mind that manually delete data in tables is pretty laborious task - you need to download the Partition Keys/Row Keys first, then execute delete commands in a loop of up to 100 batched rows at the same time. Pretty inefficient.

Answer 3

You can make a new table each week. Your reading layer should abstract away this thing. And you can delete old tables entirely which is cheaper then fetching keys and deleting insicidual rows