[英]Spring Kafka Test with Confluent Kafka Avro Serializer cannot find ZKClientConfig
我在启动嵌入式 Kafka 进行集成测试时遇到了困难。 显示此错误:
原因:org.springframework.beans.factory.BeanCreationException:创建名为“embeddedKafka”的bean时出错:调用init方法失败; 嵌套异常是 java.lang.NoClassDefFoundError: org/apache/zookeeper/client/ZKClientConfig
这是我的最小样本。 只要我需要进一步自动装配 Spring bean,我就需要使用@ExtendWith(SpringExtension.class)
。
@ExtendWith(SpringExtension.class)
@EmbeddedKafka(
partitions = 1,
bootstrapServersProperty = "spring.kafka.bootstrap-servers")
public class KafkaTest {
@Test
public void test(EmbeddedKafkaBroker broker) {
var brokerList = broker.getBrokersAsString();
log.debug("Embedded Kakfa topics {}", broker.getTopics());
}
}
我用:
我猜这里发生了依赖不一致,但是,这很奇怪,因为我相信 Spring Boot parent 可以实现依赖兼容性。
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.0.RELEASE</version>
<relativePath/>
</parent>
当然,我在下面的pom.xml中列出了相关的依赖项:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>5.2.3</version>
</dependency>
<!-- TEST SCOPE -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.6.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
我试图排除org.apache.kafka:kafka-clients
在org.springframework.kafka:spring-kafka-test
和io.confluent:kafka-avro-serializer
没有运气和相同的错误。
<exclusion>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
</exclusion>
简短的回答:
这些依赖是冲突的:
io.confluent:kafka-avro-serializer:5.2.3
org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE
org.springframework.boot:spring-boot-starter-parent:2.3.0.RELEASE
前者使用org.apache.zookeeper:zookeeper:jar:3.4.13:compile
应该排除:
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>5.2.3</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
</exclusions>
</dependency>
长答案:
检查mvn dependency:tree
before 显示kafka-avro-serializer
包含令人惊讶的org.apache.zookeeper
依赖项,而spring-kafka-test
没有:
为kafka-avro-serializer提取
[INFO] +- io.confluent:kafka-avro-serializer:jar:5.2.3:compile
[INFO] | +- io.confluent:kafka-schema-registry-client:jar:5.2.3:compile
[INFO] | +- io.confluent:common-config:jar:5.2.3:compile
[INFO] | \- io.confluent:common-utils:jar:5.2.3:compile
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.13:compile
[INFO] | | +- jline:jline:jar:0.9.94:compile
[INFO] | | +- org.apache.yetus:audience-annotations:jar:0.5.0:compile
[INFO] | | \- io.netty:netty:jar:3.10.6.Final:compile
[INFO] | \- com.101tec:zkclient:jar:0.10:compile
为spring-kafka-test提取
[INFO] \- org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE:test
[INFO] +- org.apache.kafka:kafka-clients:jar:test:2.5.0:test
[INFO] +- org.apache.kafka:kafka-streams:jar:2.5.0:test
[INFO] | +- org.apache.kafka:connect-json:jar:2.5.0:test
[INFO] | | \- org.apache.kafka:connect-api:jar:2.5.0:test
[INFO] | \- org.rocksdb:rocksdbjni:jar:5.18.3:test
[INFO] +- org.apache.kafka:kafka-streams-test-utils:jar:2.5.0:test
[INFO] +- org.apache.kafka:kafka_2.12:jar:2.5.0:test
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.11.0:test
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.11.0:test
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.8:test
[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.11.0:test
[INFO] | +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:test
[INFO] | +- com.yammer.metrics:metrics-core:jar:2.2.0:test
[INFO] | +- org.scala-lang.modules:scala-collection-compat_2.12:jar:2.1.3:test
[INFO] | +- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.9.0:test
[INFO] | +- org.scala-lang:scala-library:jar:2.12.10:test
[INFO] | +- org.scala-lang:scala-reflect:jar:2.12.10:test
[INFO] | +- com.typesafe.scala-logging:scala-logging_2.12:jar:3.9.2:test
[INFO] | \- commons-cli:commons-cli:jar:1.4:test
[INFO] +- org.apache.kafka:kafka_2.12:jar:test:2.5.0:test
[INFO] \- org.junit.jupiter:junit-jupiter-api:jar:5.6.2:test
[INFO] +- org.apiguardian:apiguardian-api:jar:1.1.0:test
[INFO] +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] \- org.junit.platform:junit-platform-commons:jar:1.6.2:test
spring-kafka-test
缺少org.apache.zookeeper:zookeeper
因此测试无法连接到 Zookeeper 并显示错误:
原因:org.springframework.beans.factory.BeanCreationException:创建名为“embeddedKafka”的bean时出错:调用init方法失败; 嵌套异常是 java.lang.NoClassDefFoundError: org/apache/zookeeper/client/ZKClientConfig
从kafka-avro-serializer
中排除这种依赖可以解决问题(请参阅简短答案)。 mvn dependency:tree
的输出确认org.apache.zookeeper:zookeeper
已正确排除在kafka-avro-serializer
之外并包含在spring-kafka-test
中。
为kafka-avro-serializer提取:
+- io.confluent:kafka-avro-serializer:jar:5.2.3:compile
[INFO] | +- io.confluent:kafka-schema-registry-client:jar:5.2.3:compile
[INFO] | +- io.confluent:common-config:jar:5.2.3:compile
[INFO] | \- io.confluent:common-utils:jar:5.2.3:compile
[INFO] | \- com.101tec:zkclient:jar:0.10:compile
为spring-kafka-test提取:
\- org.springframework.kafka:spring-kafka-test:jar:2.5.0.RELEASE:test
[INFO] +- org.apache.kafka:kafka-clients:jar:test:2.5.0:test
[INFO] +- org.apache.kafka:kafka-streams:jar:2.5.0:test
[INFO] | +- org.apache.kafka:connect-json:jar:2.5.0:test
[INFO] | | \- org.apache.kafka:connect-api:jar:2.5.0:test
[INFO] | \- org.rocksdb:rocksdbjni:jar:5.18.3:test
[INFO] +- org.apache.kafka:kafka-streams-test-utils:jar:2.5.0:test
[INFO] +- org.apache.kafka:kafka_2.12:jar:2.5.0:test
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.11.0:test
[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.11.0:test
[INFO] | | \- com.thoughtworks.paranamer:paranamer:jar:2.8:test
[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.11.0:test
[INFO] | +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:test
[INFO] | +- com.yammer.metrics:metrics-core:jar:2.2.0:test
[INFO] | +- org.scala-lang.modules:scala-collection-compat_2.12:jar:2.1.3:test
[INFO] | +- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.9.0:test
[INFO] | +- org.scala-lang:scala-library:jar:2.12.10:test
[INFO] | +- org.scala-lang:scala-reflect:jar:2.12.10:test
[INFO] | +- com.typesafe.scala-logging:scala-logging_2.12:jar:3.9.2:test
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.5.7:test
[INFO] | | +- org.apache.zookeeper:zookeeper-jute:jar:3.5.7:test
[INFO] | | +- org.apache.yetus:audience-annotations:jar:0.5.0:test
[INFO] | | +- io.netty:netty-handler:jar:4.1.49.Final:test
[INFO] | | | +- io.netty:netty-common:jar:4.1.49.Final:test
[INFO] | | | +- io.netty:netty-resolver:jar:4.1.49.Final:test
[INFO] | | | +- io.netty:netty-buffer:jar:4.1.49.Final:test
[INFO] | | | +- io.netty:netty-transport:jar:4.1.49.Final:test
[INFO] | | | \- io.netty:netty-codec:jar:4.1.49.Final:test
[INFO] | | \- io.netty:netty-transport-native-epoll:jar:4.1.49.Final:test
[INFO] | | \- io.netty:netty-transport-native-unix-common:jar:4.1.49.Final:test
[INFO] | \- commons-cli:commons-cli:jar:1.4:test
[INFO] +- org.apache.kafka:kafka_2.12:jar:test:2.5.0:test
[INFO] \- org.junit.jupiter:junit-jupiter-api:jar:5.6.2:test
[INFO] +- org.apiguardian:apiguardian-api:jar:1.1.0:test
[INFO] +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] \- org.junit.platform:junit-platform-commons:jar:1.6.2:test
我想在上一个答案中添加一些信息,因为我今天遇到了类似的问题。
主要问题是 kafka-client 2.5.0 依赖于 Zookeeper 3.5 而不再是 Zookeeper 3.4(有关兼容性矩阵,请参见https://spring.io/projects/spring-kafka )
ZkClientConfig class 已在 Zookeeper 3.5.2 中引入,因此任何低于此的 Zookeeper 版本都会引发此错误。 出于测试目的,您需要删除任何已经存在且低于 3.5.2 的 zookeeper 依赖项,并将 3.5.2 后添加到您的测试依赖项中。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.