Starting Zookeeper and Kafka servers from Java application

Question

I am using a Java application which start a consumer reading Kafka topics. Every time I need to start the consumer application, I have to start the Zookeeper and Kafka servers with commands in cmd. Is it possible to start/stop them with small Java programs? Thank you.

Answer 1

Please see this topic, it's about starting Zookeeper in Java:

Best way to start zookeeper server from java program

directing to this:

Is it possible to start a zookeeper server instance in process, say for unit tests?

Answer 2

If you need this for something like testing the I think that the better way is using the Debezium library which provides a KafkaCluster embedded implementation. You can see how we use it in the Vert.x Kafka Client project for unit testing here :

https://github.com/vert-x3/vertx-kafka-client/blob/master/src/test/java/io/vertx/kafka/client/tests/KafkaClusterTestBase.java

Take a look around that ;)

Answer 3

To start up a Kafka cluster for your tests, you can use the following wrapper: https://www.testcontainers.org/modules/kafka/

See also this project which integrates Kafka further in the context of a Spring application: https://github.com/vspiliop/embedded-kafka-cluster