Kafka Producer Exception NoClassDefFoundError

Question

I have some problem with kafka Producer, but i dont know how i can solved it

My Maven Dependency:

  <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.10</artifactId>
    <version>0.10.1.1</version>
</dependency>

If i create:

 Producer<String, byte[]> producer = createKafkaProducer();

I become Exception:

java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer
at de.dienes.opitz.node.NodesValue.onSubscriptionValue(NodesValue.java:120)
at org.eclipse.milo.opcua.sdk.client.subscriptions.OpcUaMonitoredItem.onValueArrived(OpcUaMonitoredItem.java:176)
at org.eclipse.milo.opcua.sdk.client.subscriptions.OpcUaSubscriptionManager.lambda$null$28(OpcUaSubscriptionManager.java:547)
at org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$PollAndExecute.run(ExecutionQueue.java:107)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Producer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 common frames omitted

Idea where is the problem?

Answer 1

org.apache.kafka.clients.producer.Producer is in the kafka-clients artifact. You should use

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>0.10.2.1</version>
</dependency>

kafka_2.10 is the artifact for the broker. You don't need it if you're just writing a producer/consumer (except for integration testing your producer/consumer against a test cluster).

Answer 2

if i use

        <dependency>
           <groupId>org.apache.kafka</groupId>
           <artifactId>kafka-clients</artifactId>
           <version>0.10.2.1</version>
        </dependency>

I become the new Error: KafkaProducer Claas not found.

If i doing this with my java Client and create a Producer, error:

java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/KafkaProducer

I testet it on the server with console producer and console consumer, it works! But with java client become a Exception. I have no idea more

Answer 3

This happened to me too. I solved it using BootStrapExtansionSchema like this :

java -Xbootclasspath/a:tyrus.jar -jar MyJar.jar

You can add many libraries seperated by :

java -Xbootclasspath/a:/usr/local/kafka/libs/kafka-clien-1.0.0.jar:tyrus.jar -jar MyJar.jar

Or you can simply package everything together, using

maven-assembly-plugin

add this to your pom.xml:

  <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                 <executions>
                     <execution>
                         <phase>package</phase>
                         <goals>
                             <goal>single</goal>
                         </goals>
                     </execution>
                 </executions>
                 <configuration>
                     <descriptorRefs>
                         <descriptorRef>jar-with-dependencies</descriptorRef>
                     </descriptorRefs>
                    <archive>
                         <manifest>
                             <mainClass>pakageName.MainClassName</mainClass>
                         </manifest>
                     </archive>
                 </configuration>
             </plugin>