無法在 Kafka 開發盒上啟動多個 Zookeeper 服務

Question

我正在嘗試在我的開發計算機上的同一主機上啟動 3 個 zookeeper 服務。 這顯然不是我在生產中會做的事情，我這樣做是為了探索我的測試/開發計算機中的容錯性和 Kafka 依賴性。

我已經在我的開發計算機上安裝了 Kafka 2.5.0，並且能夠在同一主機上成功設置 3 個 Kafka 服務和 1 個 Zookeeper 服務。 使用kafka package附帶的zookeeper腳本和package。

嘗試設置 3 個 zookeeper 服務時問題開始... 我做了以下設置 3 個 zookeeper 服務，但我無法成功啟動服務。 我有 3 個配置文件：

config/zookeeper.properties
config/zookeeper1.properties
config/zookeeper2.properties

config/zookeeper.properties 的內容是：

dataDir=/tmp/zookeeper
clientPort=2181
maxClientCnxns=0
admin.enableServer=false
initLimit=5
syncLimit=2
server.1=localhost:2888:3888
server.2=localhost:2889:3889
server.3=localhost:2890:3890

在config/zookeeper1.properties我有clientPort=2182和dataDir=/tmp/zookeeper1

在config/zookeeper2.properties我有clientPort=2183和dataDir=/tmp/zookeeper2

還創建了文件 /tmp/zookeeper/myid、/tmp/zookeeper1/myid、/tmp/zookeeper2/myid 並分別輸入了 id 值文本 1、2、3。

從命令行啟動 3 個動物園管理員時，它們啟動正常：

$ sudo bin/zookeeper-server-start.sh config/zookeeper.properties
$ sudo bin/zookeeper-server-start.sh config/zookeeper1.properties
$ sudo bin/zookeeper-server-start.sh config/zookeeper2.properties

我還可以通過以下方式查看領導者和追隨者是誰：

$ echo srvr | nc localhost 2181 | grep Mode
Mode: follower
$ echo srvr | nc localhost 2182 | grep Mode
Mode: leader
$ echo srvr | nc localhost 2183 | grep Mode
Mode: follower

但是當我嘗試將它們設置為系統服務時，我無法正確啟動它們......這是我擁有的單元文件：

$ cat /etc/systemd/system/zookeeper.service
[Unit]
Description=zookeeper
After=syslog.target network.target
[Service]
Type=simple
User=kafka
Group=kafka
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh
[Install]
WantedBy=multi-user.target

$ cat /etc/systemd/system/zookeeper1.service
[Unit]
Description=zookeeper 1
After=syslog.target network.target
[Service]
Type=simple
User=kafka
Group=kafka
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper1.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop1.sh
[Install]
WantedBy=multi-user.target

$ cat /etc/systemd/system/zookeeper2.service
[Unit]
Description=zookeeper 2
After=syslog.target network.target
[Service]
Type=simple
User=kafka
Group=kafka
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper2.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop2.sh
[Install]
WantedBy=multi-user.target

在嘗試開始他們之后

$ sudo systemctl daemon-reload
$ sudo systemctl enable  zookeeper
$ sudo systemctl enable  zookeeper1
$ sudo systemctl enable  zookeeper2
$ sudo systemctl start zookeeper
$ sudo systemctl start zookeeper1
$ sudo systemctl start zookeeper2

我沒看到他們跑...

在系統日志我看到這個：

May 17 03:56:20 melly-dev2 kafka-server-start.sh: [2020-05-17 03:56:20,039] INFO Opening socket connection to server localhost/127.0.0.1:2183. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
May 17 03:56:20 melly-dev2 kafka-server-start.sh: [2020-05-17 03:56:20,040] INFO Socket error occurred: localhost/127.0.0.1:2183: Connection refused (org.apache.zookeeper.ClientCnxn)

這是我在sudo journalctl -u zookeeper.service上看到的 wgat：

[2020-05-17 06:33:33,096] INFO Notification time out: 6400 (org.apache.zookeeper.server.quorum.FastLeaderElection)
[2020-05-17 06:33:39,497] WARN Cannot open channel to 2 at election address localhost/127.0.0.1:3889 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:650)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:707)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:735)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:910)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1247)
[2020-05-17 06:33:39,497] WARN Cannot open channel to 3 at election address localhost/127.0.0.1:3890 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:650)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:707)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:735)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:910)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1247)
[2020-05-17 06:33:39,497] INFO Notification time out: 12800 (org.apache.zookeeper.server.quorum.FastLeaderElection)

如何設置/查找 zookeeper 日志文件，如何讓 zookeeper 作為服務成功啟動？

Answer 1

[2020-05-17 07:02:35,248] ERROR Unable to access datadir, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain) 
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Cannot write to data directory /tmp/zookeeper1/version-2 

---

我懷疑用戶melly-dev2在/tmp/zookeeper/下寫入日志。

此外，請確保將dataDir更改為永久位置（即不在/tmp/下），因為一旦您的機器關閉，所有內容都將丟失。

Answer 2

我的程序中缺少的步驟是：

sudo chown -R kafka:kafka /tmp/zookeeper
sudo chown -R kafka:kafka /tmp/zookeeper1
sudo chown -R kafka:kafka /tmp/zookeeper2

sudo chmod -R 777 /tmp/zookeeper 
sudo chmod -R 777 /tmp/zookeeper1
sudo chmod -R 777 /tmp/zookeeper2

問題之一是 Kafka 附帶的默認 zookeeper 沒有顯示寫入錯誤的日志。 一旦我執行了這個命令（基於 GiorgosMyrianthous 評論）：

journalctl -u zookeeper.service

我可以清楚地看到錯誤並解決問題。