Apache Kafka放棄了我的制作人和消費者

Question

我對Apache Kafka有一個重大的不可預測的問題。 我正在使用Kafka 0.8，經過幾個小時的運行后，它開始丟棄我的應用程序。 特別是制片人。

這種行為是不可預測的，我無法一致地重現它。

• 我的操作系統是Mac OS
• 我使用的是Java 7（Producer是一個Web應用程序）
• 我在9092和9093上的同一個盒子上運行兩個主題和兩個服務器

我在Zookeeper上看到以下錯誤

EndOfStreamException: Unable to read additional data from client sessionid 0x14d1b94a22d001b, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
    at java.lang.Thread.run(Thread.java:745)
[2015-05-03 20:35:37,988] INFO Closed socket connection for client /0:0:0:0:0:0:0:1:60236 which had sessionid 0x14d1b94a22d001b (org.apache.zookeeper.server.NIOServerCnxn)
[2015-05-03 20:35:37,989] INFO Closed socket connection for client /127.0.0.1:60204 which had sessionid 0x14d1b94a22d0016 (org.apache.zookeeper.server.NIOServerCnxn)
[2015-05-03 20:35:37,989] WARN Ignoring unexpected runtime exception (org.apache.zookeeper.server.NIOServerCnxnFactory)
java.nio.channels.CancelledKeyException
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
    at java.lang.Thread.run(Thread.java:745)

這是我在9093服務器上看到的錯誤

java.lang.IllegalStateException: Kafka scheduler has not been started
    at kafka.utils.KafkaScheduler.ensureStarted(KafkaScheduler.scala:114)
    at kafka.utils.KafkaScheduler.shutdown(KafkaScheduler.scala:86)
    at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:350)
    at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1108)
    at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1107)
    at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1107)
    at kafka.utils.Utils$.inLock(Utils.scala:535)
    at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1107)
    at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
    at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)

我究竟做錯了什么？

Answer 1

日志表明zookeeper暫時斷開連接，增加了zookeeper超時值