Spring XD无法找到RabbitMQ ConnectionFactory的类定义

Question

我正在使用1.3.1-RELEASE版本从xd-singlenode模式切换到分布式模式。

我在一个Docker容器中运行Spring XD，其Dockerfile与此处的容器类似 。 根据XD指南 ，如果要使用RabbitMQ作为数据传输，则需要配置servers.yml文件。 我在一个单独的Docker容器中运行RabbitMQ，并将这两个容器链接在一起作为docker组成部分的一部分。

简而言之，我已经配置了所有内容（我相信是正确的），但是当我去在Spring XD中运行多个流时（那是RabbitMQ起作用的时候，因为它将充当容器之间的消息传输），Spring XD会抛出以下内容例外：

java.lang.NoClassDefFoundError: org/springframework/amqp/rabbit/connection/ConnectionFactory
    at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_131]
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_131]
    at java.lang.Class.getConstructors(Class.java:1651) ~[na:1.8.0_131]
    at org.springframework.boot.BeanDefinitionLoader.isComponent(BeanDefinitionLoader.java:276) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.BeanDefinitionLoader.load(BeanDefinitionLoader.java:158) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.BeanDefinitionLoader.load(BeanDefinitionLoader.java:135) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.BeanDefinitionLoader.load(BeanDefinitionLoader.java:127) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.SpringApplication.load(SpringApplication.java:615) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:139) [spring-boot-1.2.3.RELEASE.jar:1.2.3.RELEASE]
    at org.springframework.xd.dirt.plugins.spark.streaming.MessageBusConfiguration.createApplicationContext(MessageBusConfiguration.java:86) [spring-xd-dirt-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.xd.dirt.plugins.spark.streaming.MessageBusSender.start(MessageBusSender.java:105) [spring-xd-dirt-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:58) [spring-xd-spark-streaming-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:53) [spring-xd-spark-streaming-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:206) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:206) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:806) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:806) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1498) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1498) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.scheduler.Task.run(Task.scala:64) [spark-core_2.10-1.3.1.jar:1.3.1]
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) [spark-core_2.10-1.3.1.jar:1.3.1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.lang.ClassNotFoundException: org.springframework.amqp.rabbit.connection.ConnectionFactory
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_131]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_131]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[na:1.8.0_131]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_131]
    ... 26 common frames omitted

那么，有什么用呢？ Spring XD是否没有附带必需的RabbitMQ依赖项？

我什至尝试将spring-messaging jar手动添加到/opt/spring-xd/xd/lib ，其中包含的依赖项没有成功。 这里发生了什么？

编辑：我将把我的servers.yml文件放入额外的上下文中。 也许有人可以看到我在哪里做错了什么：

spring:
  profiles: container
xd:
  transport: rabbit
  messagebus:
    local:
      queueSize:                   2147483647
      polling:                     1000
      executor:
        corePoolSize:              0
        maxPoolSize:               200
        queueSize:                 2147483647
        keepAliveSeconds:          60
    rabbit:
      compressionLevel:            1
            # bus-level property, applies only when 'compress=true' for a stream module
            # See java.util.zip.Deflater; 1=BEST_SPEED, 9=BEST_COMPRESSION, ...
      longStringLimit:             8192
            # Headers longer than this will not be converted to String and will be a
            # DataInputStream - such headers will NOT be properly converted back on output.
      default:
        ackMode:                   AUTO
             # Valid: AUTO (container acks), NONE (broker acks), MANUAL (consumer acks).
             # Upper case only.
             # Note: MANUAL requires specialized code in the consuming module and is unlikely to be
             # used in an XD application. For more information, see
             # http://docs.spring.io/spring-integration/reference/html/amqp.html#amqp-inbound-ack
        autoBindDLQ:               false
        backOffInitialInterval:    1000
        backOffMaxInterval:        10000
        backOffMultiplier:         2.0
        batchBufferLimit:          10000
        batchingEnabled:           false
        batchSize:                 100
        batchTimeout:              5000
        compress:                  false
        concurrency:               1
        deliveryMode:              PERSISTENT
        durableSubscription:       false
        maxAttempts:               3
        maxConcurrency:            1
        prefix:                    xdbus.
             # prefix for queue/exchange names so policies (ha, dle etc.) can be applied
        prefetch:                  1
        replyHeaderPatterns:       STANDARD_REPLY_HEADERS,*
        republishToDLQ:            false
             # When false, normal rabbitmq dlq processing; when true, republish to the DLQ with stack trace
        requestHeaderPatterns:     STANDARD_REQUEST_HEADERS,*
        requeue:                   true
        transacted:                false
        txSize:                    1

#    redis:
#      headers:
            # comma-delimited list of additional header names to transport
#      default:
            # default bus properties, if not specified at the module level
#        backOffInitialInterval:    1000
#        backOffMaxInterval:        10000
#        backOffMultiplier:         2.0
#        concurrency:               1
#        maxAttempts:               3
    kafka:
      brokers:                                 kafka:9092
      zkAddress:                               zookeeper:2181
      mode:                                    embeddedHeaders
      offsetManagement:                        kafkaTopic
      headers:
             # comma-delimited list of additional header names to transport
      socketBufferSize:                        2097152
      offsetStoreTopic:                        SpringXdOffsets
      offsetStoreSegmentSize:                  25000000
      offsetStoreRetentionTime:                60000
      offsetStoreRequiredAcks:                 1
      offsetStoreMaxFetchSize:                 1048576
      offsetStoreBatchBytes:                   16384
      offsetStoreBatchTime:                    1000
      offsetUpdateTimeWindow:                  10000
      offsetUpdateCount:                       0
      offsetUpdateShutdownTimeout:             2000
      default:
        batchSize:                 16384
        batchTimeout:              0
        replicationFactor:         1
        concurrency:               1
        requiredAcks:              1
        compressionCodec:          none
        queueSize:                 8192 # must be a power of 2
        maxWait:                   100
        fetchSize:                 1048576
        minPartitionCount:         1
        durableSubscription:       false
        syncProducer:              false
        syncProducerTimeout:       5000
---
#Config for admin
spring:
  profiles: admin
xd:
  transport: rabbit
  messagebus:
    local:
      queueSize:                   2147483647
      polling:                     1000
      executor:
        corePoolSize:              0
        maxPoolSize:               200
        queueSize:                 2147483647
        keepAliveSeconds:          60
    rabbit:
      compressionLevel:            1
            # bus-level property, applies only when 'compress=true' for a stream module
            # See java.util.zip.Deflater; 1=BEST_SPEED, 9=BEST_COMPRESSION, ...
      longStringLimit:             8192
            # Headers longer than this will not be converted to String and will be a
            # DataInputStream - such headers will NOT be properly converted back on output.
      default:
        ackMode:                   AUTO
             # Valid: AUTO (container acks), NONE (broker acks), MANUAL (consumer acks).
             # Upper case only.
             # Note: MANUAL requires specialized code in the consuming module and is unlikely to be
             # used in an XD application. For more information, see
             # http://docs.spring.io/spring-integration/reference/html/amqp.html#amqp-inbound-ack
        autoBindDLQ:               false
        backOffInitialInterval:    1000
        backOffMaxInterval:        10000
        backOffMultiplier:         2.0
        batchBufferLimit:          10000
        batchingEnabled:           false
        batchSize:                 100
        batchTimeout:              5000
        compress:                  false
        concurrency:               1
        deliveryMode:              PERSISTENT
        durableSubscription:       false
        maxAttempts:               3
        maxConcurrency:            1
        prefix:                    xdbus.
             # prefix for queue/exchange names so policies (ha, dle etc.) can be applied
        prefetch:                  1
        replyHeaderPatterns:       STANDARD_REPLY_HEADERS,*
        republishToDLQ:            false
             # When false, normal rabbitmq dlq processing; when true, republish to the DLQ with stack trace
        requestHeaderPatterns:     STANDARD_REQUEST_HEADERS,*
        requeue:                   true
        transacted:                false
        txSize:                    1

#    redis:
#      headers:
            # comma-delimited list of additional header names to transport
#      default:
            # default bus properties, if not specified at the module level
#        backOffInitialInterval:    1000
#        backOffMaxInterval:        10000
#        backOffMultiplier:         2.0
#        concurrency:               1
#        maxAttempts:               3
    kafka:
      brokers:                                 kafka:9092
      zkAddress:                               zookeeper:2181
      mode:                                    embeddedHeaders
      offsetManagement:                        kafkaTopic
      headers:
             # comma-delimited list of additional header names to transport
      socketBufferSize:                        2097152
      offsetStoreTopic:                        SpringXdOffsets
      offsetStoreSegmentSize:                  25000000
      offsetStoreRetentionTime:                60000
      offsetStoreRequiredAcks:                 1
      offsetStoreMaxFetchSize:                 1048576
      offsetStoreBatchBytes:                   16384
      offsetStoreBatchTime:                    1000
      offsetUpdateTimeWindow:                  10000
      offsetUpdateCount:                       0
      offsetUpdateShutdownTimeout:             2000
      default:
        batchSize:                 16384
        batchTimeout:              0
        replicationFactor:         1
        concurrency:               1
        requiredAcks:              1
        compressionCodec:          none
        queueSize:                 8192 # must be a power of 2
        maxWait:                   100
        fetchSize:                 1048576
        minPartitionCount:         1
        durableSubscription:       false
        syncProducer:              false
        syncProducerTimeout:       5000

---
#  Rabbit MQ Properties
#
#  NOTE: sslProperties is mutually exclusive with keyStore, keyStorePassphrase, trustStore, trustStorePassphrase.
#  if you set inline properties, values in the properties file location given by 'sslProperties' will be ignored.
#
spring:
  rabbitmq:
   addresses: my-domain:5672
   adminAddresses: http://my-domain:15672
   nodes: rabbit@my-domain
   username: myusername
   password: mypassword
   virtual_host: /
   useSSL: false
   sslProperties:
   ssl:
     keyStore:
     keyStorePassphrase:
     trustStore:
     trustStorePassphrase:

编辑2：好像在AdminServerApplication或ContainerServerApplication启动时， xd-admin和xd-container脚本没有将messagebus jar添加到$CLASSPATH 。 我不确定这是否是罪魁祸首。

Answer 1

根据您选择的消息总线，所需的jar位于xd / lib下的子文件夹中。

../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/amqp-client-3.6.0.jar
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/http-client-1.0.0.RELEASE.jar
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/spring-amqp-1.5.4.RELEASE.jar
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/spring-integration-amqp-4.2.5.RELEASE.jar
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/spring-rabbit-1.5.4.RELEASE.jar
../../Downloads/spring-xd-1.3.1.RELEASE/xd/lib/messagebus/rabbit/spring-xd-messagebus-rabbit-1.3.1.RELEASE.jar

运行时在解析配置时将该目录添加到类路径。

Answer 2

用于xd-admin和xd-container的Spring XD启动脚本不会在启动它们各自的应用程序之前将任何messagebus jar添加到类路径中 。 他们只添加直接位于/lib的jars和位于lib/hadoop27/的hadoop jars。

我可以通过修改脚本以在类路径上包含必要的jar来解决此问题：

RABBIT_LIB=$APP_HOME/lib/messagebus/rabbit
if [ -d "$RABBIT_LIB" ]; then
  for k in "$RABBIT_LIB"/*.jar; do
    CLASSPATH="$CLASSPATH":"$k"
  done
fi