Azure 数据工厂增量加载，无需更改本地数据库

Question

I am trying to use Azure Data Factory to perform an incremental load on a database without using a watermark or change tracking technology. I do not have the rights to add watermarks to tables, I can only read data from the target database. The database system does not have an ability to enable change tracking technology. It is also a very large database, which is why I want to be able to incrementally load changes rather than dropping the entire database and re-uploading it every night.

Is there a way to only upload the changes without altering the on-premises database or am I SOL?

I am connecting to an old Sybase database on premises and uploading data to an Azure SQL Server Database.

Answer 1

I would suggest use Data Flow. It provide options 'upsert' for you to allow insert or update the data in Azure SQL database. We don't need to drop the entire database and re-uploading it every night.

Ref here : Sink transformation