How to add new Application consumers on the fly in Kafka

Question

I am working on an event driven flow architecture and I'd like to have as many independent applications consuming from the same topic independently (not in the same consumer group). To do that I'd create one consumer group for each new application but I'd like to do that automatically through a system. Is that possible and is that the best approach?

By the way we are coding with Python.

Thanks.

Answer 1

All you need to do is change the value of the group.id property while creating the consumer. You can do this either manually in each application or you can generate a random string using the UUID generator in Python