使用Flume将数据从kafka提取到HDFS :: ConfigurationException：必须指定Bootstrap服务器

Question

我正在尝试使用从kafka源到hdfs的水槽接收数据。 以下是我的flume conf文件。

flume1.sources = kafka-source-1
flume1.channels = hdfs-channel-1
flume1.sinks = hdfs-sink-1
flume1.sources.kafka-source-1.type = org.apache.flume.source.kafka.KafkaSource
flume1.sources.kafka-source-1.bootstrap.servers = localhost:9092
flume1.sources.kafka-source-1.zookeeperConnect = localhost:2181
flume1.sources.kafka-source-1.topic = MYNEWSFEEDS
flume1.sources.kafka-source-1.batchSize = 100
flume1.sources.kafka-source-1.channels = hdfs-channel-1
flume1.channels.hdfs-channel-1.type   = memory
flume1.sinks.hdfs-sink-1.channel = hdfs-channel-1
flume1.sinks.hdfs-sink-1.type = hdfs
flume1.sinks.hdfs-sink-1.hdfs.writeFormat = Text
flume1.sinks.hdfs-sink-1.hdfs.fileType = DataStream
flume1.sinks.hdfs-sink-1.hdfs.filePrefix = test-events
flume1.sinks.hdfs-sink-1.hdfs.useLocalTimeStamp = true
flume1.sinks.hdfs-sink-1.hdfs.path = hdfs://quickstart.cloudera:8020/tmp
flume1.sinks.hdfs-sink-1.hdfs.rollCount=100
flume1.sinks.hdfs-sink-1.hdfs.rollSize=0
flume1.channels.hdfs-channel-1.capacity = 10000
flume1.channels.hdfs-channel-1.transactionCapacity = 1000

我正在使用以下命令来运行水槽代理：

sudo flume-ng agent --name flume1 --conf-file '/etc/flume-ng/conf/flafka.conf' Dflume.root.logger=TRACE,console

但我得到以下错误：

18/03/12 16:49:18错误node.AbstractConfigurationProvider：由于配置org.apache.flume.conf.ConfigurationException期间的错误，已删除源kafka-source-1：必须在org.apache中指定引导服务器。 org.apache.flume.source.BasicSourceSemantics.configure（BasicSourceSemantics.java:65）处的flume.source.kafka.KafkaSource.doConfigure（KafkaSource.java:330）在org.apache.flume.source.AbstractPollableSource.configure（AbstractPollableSource。 org.apache.flume.conf.Configurables.configure（Configurables.java:41）处org.apache.flume.node.AbstractConfigurationProvider.loadSources（AbstractConfigurationProvider.java:326）处org.apache.flume.node处的java：63） org.apache.flume.node.PollingPropertiesFileConfigurationProvider $ FileWatcherRunnable.run。 ），位于java.util。 java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 301（ScheduledThreadPoolExecutor.java:178）处的并发.FutureTask.runAndReset（FutureTask.java:304）在java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run：301。 ）的java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1145）的java.util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:615）的java.lang.Thread.run（Thread.java： 745）

虽然，我在conf文件中指定了Bootstrap服务器，但仍然给出相同的错误。 尝试了许多排列组合，但没有成功。

Answer 1

根据官方的JavaDoc ，您应该替换

flume1.sources.kafka-source-1.bootstrap.servers = localhost:9092

同

flume1.sources.kafka-source-1.kafka.bootstrap.servers = localhost:9092