NoSuchMethodError：KafkaConsumer.subscribe

Question

I use the following dependencies:

val akkaVersion = "2.4.9"
val kafkaVersion = "0.10.0.1"

dependencies =  ...
  "com.typesafe.akka" %% "akka-actor" % akkaVersion,
  "com.typesafe.akka" %% "akka-cluster" % akkaVersion,
  "com.softwaremill.reactivekafka" %% "reactive-kafka-core" % "0.10.0"   
        excludeAll(
          ExclusionRule("org.slf4j", "log4j-over-slf4j"),
          ExclusionRule("org.apache.kafka", "kafka_2.11")),
   "org.apache.kafka" % "kafka_2.11" % kafkaVersion,
   "org.apache.kafka" % "kafka-clients" % kafkaVersion,
   "org.apache.kafka" % "connect-json" % kafkaVersion

And there is actor:

class ExpKafkaActor extends Actor {

   override def preStart(): Unit = {
    super.preStart()
    val kafka = new ReactiveKafka()

    val publisher = kafka.consume(ConsumerProperties(
      bootstrapServers = "localhost:9092",
      topic = "someTopicName",
      groupId = "groupName",
      valueDeserializer = new StringDeserializer()
    ))

    Source.fromPublisher(publisher).map(m => {
      val message = ProducerMessage(m.value().toUpperCase)
      log.info(s"handle message ${m.value()}")
      message
    })
   }
   ...
}

Then I try start the above actor and the following exception raises:

java.lang.NoSuchMethodError:
 org.apache.kafka.clients.consumer.KafkaConsumer.subscribe

I found the following issue , but it doesn't help. How can I resolve this API conclict?

Answer 1

This issue is due to Kafka client API incompatibility. Please use the right dependency as part of your project pom.xml:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>{kafka.version}</version>
</dependency>

ex: for Kafka 0.10.2.0, use the following one
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>0.10.2.0</version>
</dependency>