Zookeeper leader not the Mesos Master

Question

I am new to Mesos and I have built a mesos cluster of three CentOS 7 nodes (all three nodes acting as masters and slaves) in a local hypervisor. The nodes are named mesos1, mesos2 and mesos3

I have this running with Zookeeper marathon and chronos. I was wondering how I can check who is the acting mesos master at any given time, when I came across this post !

I also found out that I could find the leading master of mesos by adding /redirect to the endpoint.

So when I tried that, Mesos UI at 5050 pport redirected me to the node mesos2.

However, when I tried to find the zookeeper leader using this command: /opt/zookeeper/bin/zkServer.sh status I got the following response that the leader was mesos3

[root@mesos3 ~]# /opt/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Mode: leader

I am confused: Shouldn't the mesos master be the node that is indicated as leader by zookeeper?

Any help is greatly appreciated

Answer 1

Mesos and Zookeeper have distinct concepts of leadership and it sounds like you are mixing the two up. You are running three instances of the Zookeeper server process which form a quorum, and three Mesos master processes which also form a quorum that happen to be on the same set of servers. Mesos uses Zookeeper for storing state and other critical functions but Zookeeper's cluster leadership is in no way related to Mesos cluster leadership . They do use very similar terminology so it is easy to see how they can be mixed up.