How are consumers setup in Active - Active Kafka setup

Question

We are having Active Active Kafka cluster setup with topic renaming using Mirror Maker 2.0 as specified in https://strimzi.io/blog/2020/03/30/introducing-mirrormaker2/ . I believe topic such as us-email are setup as follows:

dc1

1. us-email
2. us-email-dc2 (mirror of dc2)

dc2

1. us-email
2. us-email-dc1 (mirror of dc1)

Producers can publish to their local DC's and both clusters would contain data of both the DC's. So far so good.

Consumer app would subscribe to wild card topic (us-email-*) to read data of both DC's. If that's the case, Do I setup a consumer to read from their respective DC's? In this case, there will be duplicate message read for reach message due to mirroring. OR it is recommended to point a single consumer group to a single DC only at a time to prevent duplication? If yes, if a single DC fails, how will the failover happen?

Answer 1

Does consumers in both data centers have to point to single DC

Consumers cannot read from more than one list of bootstrap servers, so yes

there is manual failover?

Not clear what you mean by manual.

1. If the Mirror or destination brokers fail, then consumer stops reading anything
2. If the source is down, then the mirroring stops, leading back to (1)

consumers in both DC's will get replicated messages as well

Mirroring doesn't guarantee exactly once delivery

Answer 2

Automatic failover is not possible. Whenever one dc fails, you have to update the consumer to read from other dc manually. Also about consumer offsets, I am not sure if they sync and they let you continue or treat the consumer as new consumer-group.