java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer

Question

I'm trying to use Kafka:

import java.util.Properties;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;

public class SimpleProducer {
    public static void main(String[] args) {
        Properties props = new Properties();
        Producer<String, String> producer = new KafkaProducer<String, String>(props);
    }
}

But getting the following error:

java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer

build.gradle:

...

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'
    compile group: 'org.apache.kafka', name: 'kafka-clients', version: '2.2.0'
}

...

Answer 1

I had tried with the following dependency in gradle.

compile group: 'org.apache.kafka', name: 'kafka-clients', version: '2.1.0'

Also, I provide below the code snippet for which I tested.

import java.util.Properties;
import org.apache.kafka.clients.producer.Callback;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;

public class Producer {

    public static void main(String[] args){
        Properties properties = new Properties();
        properties.put("bootstrap.servers", "192.168.119.139:9092");
        properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        KafkaProducer kafkaProducer = new KafkaProducer(properties);
        TestCallback callback = new TestCallback();
        try{
            for(int i = 0; i < 100; i++){
                System.out.println("----->"+i);
                kafkaProducer.send(new ProducerRecord("test", Integer.toString(i), "test message - " + i ));
            }
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            kafkaProducer.close();
        }
    }


    private static class TestCallback implements Callback {
        @Override
        public void onCompletion(RecordMetadata recordMetadata, Exception e) {
            if (e != null) {
                System.out.println("Error while producing message to topic :" + recordMetadata);
                e.printStackTrace();
            } else {
                String message = String.format("sent message to topic:%s partition:%s  offset:%s", recordMetadata.topic(), recordMetadata.partition(), recordMetadata.offset());
                System.out.println(message);
            }
        }
    }
}

I have tested this code using Eclipse IDE. Also remember, apache kafka client also downloads the following dependencies.

• zstd-jni-1.3.5-4.jar
• lz4-java-1.5.0.jar
• snappy-java-1.1.7.2.jar
• slf4j-api-1.7.25.jar

If you want to run using command you have to run by specifying the -classpath with a set of jar files. I provide an example below.

java.exe -Dfile.encoding=UTF-8 -classpath somelocation/a.jar;somelocation/b.jar;somelocation/c.jar

If it still does not solve your problem, provide the application structure, post code details so that others can help you.

Answer 2

I ran:

java -jar TestProejct-1.0-SNAPSHOT.jar -cp "D:\Software\kafka_2.12-2.2.0\libs\kafka-clients-2.2.0.jar"

There were several problems:

1. java doesn't aceept both -jar and -cp, so I had to include my jar in the classpath itself in addition to Kafka.
2. I had to specify the main class I wanted to run.
3. There was more than one jar to import from Kafka, so I had to specify * insead of kafka-clients-2.2.0.jar.

This solved the problem:

java -cp "D:\Software\kafka_2.12-2.2.0\libs\*;TestProejct-1.0-SNAPSHOT.jar" SimpleProducer