How to store data incrementallyfrom azure SQL database to azure blob storage using azure data factory,based on some conditions:date need to be updated

Question

i need to store some data from sql server to azure blob storage in which the data in sql server will be updated, and the time stamp will be mentioned along with it and also whenever i store it in storage it will be recording some timestamp with it. so based on the condition if the timestamp in sql server is greater than timestamp in blob storage i need to migrate that data i tried migrating by moving the data from sql server to storage account, but there might be some changes in future in the sql server so based on that how to migrate the new data periodically

Answer 1

I would suggest you go through some Pipeline template

Perhaps...

If you do have timestamp column in your source database to identify the new or updated rows, but do not want to create any external control table to achive delta copy, you can go to "copy data tool" to get a pipeline, which use trigger scheduled time as a variable to read the new rows only from source database.

View documentation from MS for detailed walkthrough