Confluent Control center not showing system health (for a Multi-Cluster Configuration)

Question

I'm having trouble getting control-center to work. Setted up a 3 node kafka cluster using the following docker image = confluentinc/cp-enterprise-kafka. On a separate machine I've downloaded confluent platform v5.0.1 and I've configured (tried) control-center to monitor the docker cluster.

The kafka broker I'm using for the control-center configuration is the same from the confluent platform v5.0.1, downloaded.(I start the whole stack via bin/confluent start)

But I keep getting the rocket launching page when clicking Monitoring > System health.

My setup : --------------------------------------------------------

• 3 node kafka cluster using docker images.
• docker image used = confluentinc/cp-enterprise-kafka

kafka running on these hostnames for the 3-node cluster :

• os0 / running on tcp/29092
• os1 / running on tcp/39092
• os2 / running on tcp/49092

Control-center is running on a separate machine whose hostname = sb1

Futhermore the brokers have the following directives defined as :

• metric.reporters=io.confluent.metrics.reporter.ConfluentMetricsReporter
• confluent.metrics.reporter.bootstrap.servers=sb1:9092

For the control-center I added the 3 node cluster config :

• confluent.controlcenter.kafka.osd.bootstrap.servers=os0:29092,os1:39092,os2:49092

---

I'm expecting the kafka brokers writing to the kafka broker @ sb1 (used by control-center) topic _confluent-metrics

What I've tried/checked/debugged so far :

• dumped the the topic _confluent-metrics, and I have messages being written there
• I dont know if logs from control-center (@ /tmp/confluent.QJ2C4BmE/control-center/control-center.stdout) do show anyhting useful (at least for what I can interpret)
  • I can see HTTP/200 for the cluster I'm trying to monitor written down in the blog.
• at the log from the kafka brokers I also see written the following, which put me thinking the messages were written to the topic : [2018-12-15 07:57:59,893] ERROR Failed to submit metrics to Kafka topic __confluent.support.metrics (due to exception): java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for __confluent.support.metrics-0: 30083 ms has passed since batch creation plus linger time (io.confluent.support.metrics.submitters.KafkaSubmitter) [2018-12-15 07:58:01,088] INFO Successfully submitted metrics to Confluent via secure endpoint (io.confluent.support.metrics.submitters.ConfluentSubmitter)

I run out of viable solutions to debug this, any help would be appreciated.

thanks in advance.

Answer 1

I was accessing control center via an ssh tunnel. (This was a testing environment I was using to set up CC (control center)).

When I accessed directly to the ip:port of CC everything run smoothly.