Can I configure AWS RDS to only stream INSERT operations to AWS DMS?

Question

My requirement is to stream only INSERTs on a specific table in my db to a Kinesis data stream.

I have configured this pipeline in my AWS environment:

RDS Postgres 13 -> DMS (Database Migration Service) -> KDS (Kinesis Data Stream)

This setup works correctly but it processes all changes, even UPDATEs and DELETEs, on my source table.

What I've tried:

• Looking for config options in the Postgres logical decoding plugin. DMS uses the test_decoding PG plugin which does not accept options to include/exclude data changes by operation type.
• Looking at the DMS selection and filtering rules. Still didn't see anything that might help.

Of course I could simply ignore records originated from non-INSERT operations in my Kinesis consumer, but this doesn't look like a cost-efficient implementation.

Is there any way to meet my requirements using these AWS services (RDS -> DMS -> Kinesis)?

Answer 1

Well DMS does not have this capability.

If you want only INSERT to be send to Kinesis in that case you can have a lambda function on every INSERT of RDS. Lambda function can be configured as trigger for INSERT.

You can invoke lambda only for INSERT and write to Kinesis directly.

Cost wise also this will be less. In DMS you are paying for Replication instance even when not in use.

For detailed reference Stream changes from Amazon RDS for PostgreSQL using Amazon Kinesis Data Streams and AWS Lambda