简体   繁体   English

如何仅提取 AWS RDS(Postgres 数据库)中的更改数据(CDC)并将更改数据转储到另一个 RDS postgres 实例?

[英]How to extract only change data (CDC) in AWS RDS (Postgres database) and dump change data to another RDS postgres instance?

Is there a way to capture only change data in RDS Postgres instance and write those change records to another RDS Postgres DB instance for later use?有没有办法只捕获 RDS Postgres 实例中的更改数据并将这些更改记录写入另一个 RDS Postgres 数据库实例以供以后使用?

I have looked on few option like using Debezium but it doesn't seems to be helpful as I don't have to actually write to streams.我看过几个选项,比如使用 Debezium,但它似乎没有帮助,因为我不必实际写入流。

My requirement is that: 1) Load all the existing data to another DB at once 2) Capture the change data (delta) and store in a database (with same replica) and process the change data.我的要求是:1)一次将所有现有数据加载到另一个数据库2)捕获更改数据(增量)并存储在数据库中(具有相同的副本)并处理更改数据。

I have followed below steps to implement above scenario:我已按照以下步骤实施上述方案:

1) Written a Spring batch job which connects to RDS and does a full load once. 1) 编写了一个 Spring 批处理作业,它连接到 RDS 并执行一次完整加载。 2) Once the full load is completed, we added logic to export WAL logs for CDC data and then put the data to stream to process further. 2) 完全加载完成后,我们添加逻辑导出CDC数据的WAL日志,然后将数据放入stream进一步处理。

Not the very best solution but this solution worked well for the above scenario.不是最好的解决方案,但该解决方案适用于上述情况。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM