卡夫卡連接 java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;

Question

我正在嘗試在 AWS 實例上設置 kafka-connect-cassandra。

我在connect-avro-distributed.properties文件中設置了plugin.path ：

plugin.path=/home/ubuntu/kafka_2.11-1.0.0/plugins

我有kafka-connect-cassandra-1.0.0-1.0.0-all.jar在：

/home/ubuntu/kafka_2.11-1.0.0/plugins/lib

這是回溯：

[2018-02-18 10:28:33,268] INFO Kafka Connect distributed worker initializing ... (org.apache.kafka.connect.cli.ConnectDistributed:60)
[2018-02-18 10:28:33,278] INFO WorkerInfo values: 
    jvm.args = -Xmx256M, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/var/log/kafka, -Dlog4j.configuration=file:/etc/kafka/connect-log4j.properties
    jvm.spec = Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_151, 25.151-b12
    jvm.classpath = /home/ubuntu/kafka_2.11-1.0.0/plugins:/usr/share/java/kafka/jackson-jaxrs-json-provider-2.9.1.jar:/usr/share/java/kafka/paranamer-2.7.jar:/usr/share/java/kafka/rocksdbjni-5.7.3.jar:/usr/share/java/kafka/jackson-jaxrs-base-2.9.1.jar:/usr/share/java/kafka/jetty-servlets-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-clients-1.0.0-cp1.jar:/usr/share/java/kafka/xz-1.5.jar:/usr/share/java/kafka/commons-lang3-3.1.jar:/usr/share/java/kafka/jetty-security-9.2.22.v20170606.jar:/usr/share/java/kafka/httpclient-4.5.2.jar:/usr/share/java/kafka/jackson-core-asl-1.9.13.jar:/usr/share/java/kafka/connect-json-1.0.0-cp1.jar:/usr/share/java/kafka/connect-runtime-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-container-servlet-2.25.1.jar:/usr/share/java/kafka/commons-collections-3.2.1.jar:/usr/share/java/kafka/connect-transforms-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-common-2.25.1.jar:/usr/share/java/kafka/zookeeper-3.4.10.jar:/usr/share/java/kafka/scala-library-2.11.11.jar:/usr/share/java/kafka/jackson-core-2.9.1.jar:/usr/share/java/kafka/argparse4j-0.7.0.jar:/usr/share/java/kafka/maven-artifact-3.5.0.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-test-sources.jar:/usr/share/java/kafka/commons-codec-1.9.jar:/usr/share/java/kafka/kafka-connect-cassandra-1.0.0-1.0.0-all.jar:/usr/share/java/kafka/httpcore-4.4.4.jar:/usr/share/java/kafka/hk2-utils-2.5.0-b32.jar:/usr/share/java/kafka/connect-api-1.0.0-cp1.jar:/usr/share/java/kafka/javassist-3.21.0-GA.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-sources.jar:/usr/share/java/kafka/support-metrics-client-4.0.0.jar:/usr/share/java/kafka/commons-compress-1.8.1.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-scaladoc.jar:/usr/share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/share/java/kafka/jersey-media-jaxb-2.25.1.jar:/usr/share/java/kafka/kafka-streams-1.0.0-cp1.jar:/usr/share/java/kafka/zkclient-0.10.jar:/usr/share/java/kafka/hk2-locator-2.5.0-b32.jar:/usr/share/java/kafka/slf4j-api-1.7.25.jar:/usr/share/java/kafka/support-metrics-common-4.0.0.jar:/usr/share/java/kafka/kafka.jar:/usr/share/java/kafka/jersey-server-2.25.1.jar:/usr/share/java/kafka/jackson-module-jaxb-annotations-2.9.1.jar:/usr/share/java/kafka/jetty-io-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-log4j-appender-1.0.0-cp1.jar:/usr/share/java/kafka/avro-1.8.2.jar:/usr/share/java/kafka/jackson-annotations-2.9.1.jar:/usr/share/java/kafka/guava-20.0.jar:/usr/share/java/kafka/hk2-api-2.5.0-b32.jar:/usr/share/java/kafka/lz4-java-1.4.jar:/usr/share/java/kafka/reflections-0.9.11.jar:/usr/share/java/kafka/commons-digester-1.8.1.jar:/usr/share/java/kafka/slf4j-log4j12-1.7.25.jar:/usr/share/java/kafka/jersey-client-2.25.1.jar:/usr/share/java/kafka/commons-lang3-3.5.jar:/usr/share/java/kafka/jackson-mapper-asl-1.9.13.jar:/usr/share/java/kafka/javax.annotation-api-1.2.jar:/usr/share/java/kafka/snappy-java-1.1.4.jar:/usr/share/java/kafka/javax.inject-2.5.0-b32.jar:/usr/share/java/kafka/jackson-databind-2.9.1.jar:/usr/share/java/kafka/jetty-http-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-streams-examples-1.0.0-cp1.jar:/usr/share/java/kafka/plexus-utils-3.0.24.jar:/usr/share/java/kafka/metrics-core-2.2.0.jar:/usr/share/java/kafka/connect-file-1.0.0-cp1.jar:/usr/share/java/kafka/kafka-tools-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-guava-2.25.1.jar:/usr/share/java/kafka/commons-logging-1.2.jar:/usr/share/java/kafka/jetty-server-9.2.22.v20170606.jar:/usr/share/java/kafka/validation-api-1.1.0.Final.jar:/usr/share/java/kafka/jetty-continuation-9.2.22.v20170606.jar:/usr/share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/share/java/kafka/httpmime-4.5.2.jar:/usr/share/java/kafka/log4j-1.2.17.jar:/usr/share/java/kafka/jopt-simple-5.0.4.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-javadoc.jar:/usr/share/java/kafka/javax.ws.rs-api-2.0.1.jar:/usr/share/java/kafka/jersey-container-servlet-core-2.25.1.jar:/usr/share/java/kafka/jetty-servlet-9.2.22.v20170606.jar:/usr/share/java/kafka/jetty-util-9.2.22.v20170606.jar:/usr/share/java/kafka/commons-validator-1.4.1.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1.jar:/usr/share/java/kafka/javax.inject-1.jar:/usr/share/java/kafka/commons-beanutils-1.8.3.jar:/usr/share/java/kafka/javassist-3.20.0-GA.jar:/usr/share/java/kafka/aopalliance-repackaged-2.5.0-b32.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-test.jar:/usr/share/java/confluent-common/zookeeper-3.4.10.jar:/usr/share/java/confluent-common/common-metrics-4.0.0.jar:/usr/share/java/confluent-common/build-tools-4.0.0.jar:/usr/share/java/confluent-common/zkclient-0.10.jar:/usr/share/java/confluent-common/slf4j-api-1.7.25.jar:/usr/share/java/confluent-common/common-utils-4.0.0.jar:/usr/share/java/confluent-common/netty-3.10.5.Final.jar:/usr/share/java/confluent-common/common-config-4.0.0.jar:/usr/share/java/confluent-common/jline-0.9.94.jar:/usr/share/java/confluent-common/log4j-1.2.17.jar:/usr/share/java/kafka-serde-tools/kafka-json-serializer-4.0.0.jar:/usr/share/java/kafka-serde-tools/paranamer-2.7.jar:/usr/share/java/kafka-serde-tools/xz-1.5.jar:/usr/share/java/kafka-serde-tools/jackson-core-asl-1.9.13.jar:/usr/share/java/kafka-serde-tools/jackson-core-2.9.1.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-converter-4.0.0.jar:/usr/share/java/kafka-serde-tools/commons-compress-1.8.1.jar:/usr/share/java/kafka-serde-tools/avro-1.8.2.jar:/usr/share/java/kafka-serde-tools/jackson-annotations-2.9.1.jar:/usr/share/java/kafka-serde-tools/kafka-schema-registry-client-4.0.0.jar:/usr/share/java/kafka-serde-tools/kafka-avro-serializer-4.0.0.jar:/usr/share/java/kafka-serde-tools/jackson-mapper-asl-1.9.13.jar:/usr/share/java/kafka-serde-tools/jackson-databind-2.9.1.jar:/usr/share/java/kafka-serde-tools/snappy-java-1.1.1.3.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-json-provider-2.9.1.jar:/usr/bin/../share/java/kafka/paranamer-2.7.jar:/usr/bin/../share/java/kafka/rocksdbjni-5.7.3.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-base-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-servlets-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-clients-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/xz-1.5.jar:/usr/bin/../share/java/kafka/commons-lang3-3.1.jar:/usr/bin/../share/java/kafka/jetty-security-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/httpclient-4.5.2.jar:/usr/bin/../share/java/kafka/jackson-core-asl-1.9.13.jar:/usr/bin/../share/java/kafka/connect-json-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/connect-runtime-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-2.25.1.jar:/usr/bin/../share/java/kafka/commons-collections-3.2.1.jar:/usr/bin/../share/java/kafka/connect-transforms-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-common-2.25.1.jar:/usr/bin/../share/java/kafka/zookeeper-3.4.10.jar:/usr/bin/../share/java/kafka/scala-library-2.11.11.jar:/usr/bin/../share/java/kafka/jackson-core-2.9.1.jar:/usr/bin/../share/java/kafka/argparse4j-0.7.0.jar:/usr/bin/../share/java/kafka/maven-artifact-3.5.0.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-test-sources.jar:/usr/bin/../share/java/kafka/commons-codec-1.9.jar:/usr/bin/../share/java/kafka/kafka-connect-cassandra-1.0.0-1.0.0-all.jar:/usr/bin/../share/java/kafka/httpcore-4.4.4.jar:/usr/bin/../share/java/kafka/hk2-utils-2.5.0-b32.jar:/usr/bin/../share/java/kafka/connect-api-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/javassist-3.21.0-GA.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-sources.jar:/usr/bin/../share/java/kafka/support-metrics-client-4.0.0.jar:/usr/bin/../share/java/kafka/commons-compress-1.8.1.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-scaladoc.jar:/usr/bin/../share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/bin/../share/java/kafka/jersey-media-jaxb-2.25.1.jar:/usr/bin/../share/java/kafka/kafka-streams-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/zkclient-0.10.jar:/usr/bin/../share/java/kafka/hk2-locator-2.5.0-b32.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.25.jar:/usr/bin/../share/java/kafka/support-metrics-common-4.0.0.jar:/usr/bin/../share/java/kafka/kafka.jar:/usr/bin/../share/java/kafka/jersey-server-2.25.1.jar:/usr/bin/../share/java/kafka/jackson-module-jaxb-annotations-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-io-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-log4j-appender-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/avro-1.8.2.jar:/usr/bin/../share/java/kafka/jackson-annotations-2.9.1.jar:/usr/bin/../share/java/kafka/guava-20.0.jar:/usr/bin/../share/java/kafka/hk2-api-2.5.0-b32.jar:/usr/bin/../share/java/kafka/lz4-java-1.4.jar:/usr/bin/../share/java/kafka/reflections-0.9.11.jar:/usr/bin/../share/java/kafka/commons-digester-1.8.1.jar:/usr/bin/../share/java/kafka/slf4j-log4j12-1.7.25.jar:/usr/bin/../share/java/kafka/jersey-client-2.25.1.jar:/usr/bin/../share/java/kafka/commons-lang3-3.5.jar:/usr/bin/../share/java/kafka/jackson-mapper-asl-1.9.13.jar:/usr/bin/../share/java/kafka/javax.annotation-api-1.2.jar:/usr/bin/../share/java/kafka/snappy-java-1.1.4.jar:/usr/bin/../share/java/kafka/javax.inject-2.5.0-b32.jar:/usr/bin/../share/java/kafka/jackson-databind-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-http-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-streams-examples-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/plexus-utils-3.0.24.jar:/usr/bin/../share/java/kafka/metrics-core-2.2.0.jar:/usr/bin/../share/java/kafka/connect-file-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/kafka-tools-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-guava-2.25.1.jar:/usr/bin/../share/java/kafka/commons-logging-1.2.jar:/usr/bin/../share/java/kafka/jetty-server-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/validation-api-1.1.0.Final.jar:/usr/bin/../share/java/kafka/jetty-continuation-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/bin/../share/java/kafka/httpmime-4.5.2.jar:/usr/bin/../share/java/kafka/log4j-1.2.17.jar:/usr/bin/../share/java/kafka/jopt-simple-5.0.4.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-javadoc.jar:/usr/bin/../share/java/kafka/javax.ws.rs-api-2.0.1.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-core-2.25.1.jar:/usr/bin/../share/java/kafka/jetty-servlet-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/jetty-util-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/commons-validator-1.4.1.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/javax.inject-1.jar:/usr/bin/../share/java/kafka/commons-beanutils-1.8.3.jar:/usr/bin/../share/java/kafka/javassist-3.20.0-GA.jar:/usr/bin/../share/java/kafka/aopalliance-repackaged-2.5.0-b32.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-test.jar:/usr/bin/../share/java/confluent-support-metrics/*:/usr/share/java/confluent-support-metrics/*
    os.spec = Linux, amd64, 4.4.0-1049-aws
    os.vcpus = 2
 (org.apache.kafka.connect.runtime.WorkerInfo:71)
[2018-02-18 10:28:33,279] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed:69)
[2018-02-18 10:28:33,290] INFO Loading plugin from: /home/ubuntu/kafka_2.11-1.0.0/plugins/lib (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:179)
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
    at org.reflections.Reflections.expandSuperTypes(Reflections.java:380)
    at org.reflections.Reflections.<init>(Reflections.java:126)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:258)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:201)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:193)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:153)
    at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
    at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:70)

jvm.classpath的第一個條目是我在plugin/lib擁有kafka-connect-cassandra.jar的位置。

番石榴.jars：

這些是番石榴罐的路徑。 我應該將 kafka-connect-cassandra.jar 放在哪里還是應該刪除這些罐子中的任何一個？

/usr/share/java/kafka-connect-elasticsearch/guava-18.0.jar
/usr/share/java/kafka-connect-storage-common/guava-14.0.1.jar
/usr/share/java/kafka/guava-20.0.jar
/home/ubuntu/cassandra/apache-cassandra-3.11.1/lib/guava-18.0.jar

請幫幫我。

Answer 1

Connect 框架目前帶有guava-20.0.jar以便使用 Reflections 並能夠掃描您的plugin.path以獲取 Connect 插件。

如果您使用其 bin 腳本之一啟動 Connect，則 Connect 框架的依賴項將添加到您的 CLASSPATH 中： connect-distributed或connect-standalone 。

但是，不知何故，舊版本的番石榴首先駐留在您的 CLASSPATH 中（可能是因為您已將 CLASSPATH 設置為包含此類依賴項），然后 Connect 無法使用番石榴版本< 20掃描您的plugin.path 。 這就是你的異常所表明的。 該 Connect 無法使用它在您的 CLASSPATH 中找到的番石榴版本掃描plugin.path 。

你的plugin.path應該包含你的插件，每個都在它自己的目錄中。 例如，類似於上面的內容：

plugin.path=/home/ubuntu/connect with

/home/ubuntu/connect/kafka_2.11-1.0.0/和

/home/ubuntu/connect/kafka-connect-cassandra-1.0.0-1.0.0-all.jar

根據您的示例，上面的布局用於表明插件不應包含其他插件，但不建議將您的plugin.path設置為用戶的主目錄。

Answer 2

這是一個類路徑問題。 看起來您的類路徑中可能有一個不兼容的番石榴版本？ 如果您的插件路徑在它擁有的任何 jar 中都不包含此方法，則這是連接器打包問題。 如果是，那么您可能有兩個版本。 作為第一步，使用 find 命令仔細檢查該插件路徑，以檢查消息中該類的所有 jar。 最終，您需要弄清楚連接器期望的依賴項的版本，並將該版本和僅該版本放入插件路徑中。