How to add / copy a custom jar file into KafkaMirrorMaker?

Question

I'm developing some custom topic name mapping and a jar file has been produced out of it.

And since I'm using MirrorMaker v1, these variables have also been added KAFKA_MIRRORMAKER_MESSAGE_HANDLER and KAFKA_MIRRORMAKER_MESSAGE_HANDLER_ARGS inside the KafkaMirrorMaker yaml file.

But I don't know how to physically add this custom jar file into the KafkaMirrorMaker pod. I have checked the CRD of the KafkaMirrorMaker but can't find any clue yet.

So, is there a way to let KafkaMirrorMaker download some file(s) / artifact(s) and include the jar file(s) into the classpath so that the custom MessageHandler can find it?

The helm install command is used to deploy the Mirror Maker. The apiVersion of the KafkaMirrorMaker I'm using currently: kafka.strimzi.io/v1beta2

Answer 1

Based on the strimzi tag, I assume you use Strimzi's Mirror Maker v1? To add your own JAR, you would need to build a custom container image.

You could modify the Strimzi project sources and build everything from scratch (you can add your JAR as a dependency into the 3rd party libs in `docker-images/kafka/...). But that is rather complicated as you build the whole project.

The easiest way is to just write your own Dockerfile and use the existing Strimzi image as a base image. For example:

FROM quay.io/strimzi/kafka:0.26.0-kafka-3.0.0
USER root:root
COPY ./my-jar.jar /opt/kafka/libs/my-jar.jar
USER 1001

You can build this Dockerfile and push it into your own Docker registry (Docker Hub, Quay, whetever you use). You should make sure the FROM uses the right image based on the Strimzi version you use and KAfka version you use.

And once you have it, you have to tell Strimzi to use this image. You can do that either using the .spec.image option in the KafkaMirrorMaker custom resource. Or you can just change the environment variable STRIMZI_KAFKA_MIRROR_MAKER_IMAGES in the Strimzi Cluster Operator deployment and update the images which should be used there.