简体   繁体   中英

Data warehousing (ETL): Scheduled data migration from MySQL to PostgreSQL

I've been assigned task of data warehousing for Reporting and data analysis. Let me first explain what we are going to do.

Step 1 . Replicate production server MySQL database.

Step 2 . Scheduled ETL : Read replicated database ( MySQL ) and push data to PostgreSQL .

Now I need your help on Step 2.

Note: I want saveOrUpdate functionality. If id is available then update it or save it. Data will be picked up based on modified date.

So is there any tool available for scheduled data push in PostgreSQL ?, Considering my requirements.

If there ain't any tool available then which programming language I should use for ETL? And other pointers you can provide me to achieve this.

Asked same question https://dba.stackexchange.com/questions/203460/data-warehousing-etl-scheduled-data-migration-from-mysql-to-postgresql on dba.stackexchange.com but I guess it has low userbase so posting it here.

On aws you have DMS . I don't know if you can use it with external services but it's works pretty well.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM