I'm trying to deploy Prometheus on GKE to monitor an existing Strimzi Kafka GKE cluster, and am facing issues. (ref - https://strimzi.io/docs/operators/latest/deploying.html#proc-metrics-deploying-prometheus-operator-str )
Here is what is done:
here are the changes:
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: prom-graf #1
spec:
kafka:
version: 3.0.0
........
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: kafka-metrics-config.yml
zookeeper:
......
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: zookeeper-metrics-config.yml
entityOperator: #11
topicOperator: {}
userOperator: {}
kafkaExporter:
topicRegex: ".*"
groupRegex: ".*"
Command to apply changes to Kafka cluster:
kubectl apply -f kafka-deployment.yaml -n kafka
command :
curl -s https://raw.githubusercontent.com/coreos/prometheus-operator/master/bundle.yaml > bundle.yaml
changed the namespace to 'monitoring' in bundle.yaml and created the Prometheous operator
kubectl create -f bundle.yaml -n monitoring
Ran the following commands to create the Prometheus & Grafana on GKE: files are the same as in github - https://github.com/strimzi/strimzi-kafka-operator/tree/0.26.0/examples/metrics , although i've mad the namespace changes
kubectl apply -f strimzi-pod-monitor.yaml
(changed namespace to kafka, since kafka is deployed in namespace kafka)
kubectl apply -f prometheus-rules.yaml -n monitoring
kubectl apply -f prometheus.yaml -n monitoring
kubectl apply -f grafana.yaml - monitoring
- created secret from file - prometheus-additional.yaml
kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml -n monitoring
When i inspect the object created on namespace - monitoring, here is what is listed:
Karans-MacBook-Pro:prometheus-yamls karanalang$ kc get all,prometheus -n monitoring
NAME READY STATUS RESTARTS AGE
pod/grafana-5ccfb68647-55mzk 0/1 Running 0 21s
pod/prometheus-operator-85d84bb848-57lrn 1/1 Running 0 46m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana ClusterIP 10.100.3.10 <none> 3000/TCP 21s
service/prometheus-operator ClusterIP None <none> 8080/TCP 46m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/grafana 0/1 1 0 24s
deployment.apps/prometheus-operator 1/1 1 1 46m
NAME DESIRED CURRENT READY AGE
replicaset.apps/grafana-5ccfb68647 1 1 0 23s
replicaset.apps/prometheus-operator-85d84bb848 1 1 1 46m
NAME VERSION REPLICAS AGE
prometheus.monitoring.coreos.com/prometheus 3 44m
Question is - how do i get the hostname for the Prometheus install (since i need this to add as source in Grafana)? Somehow, i don't see the prometheus pod, should that not be created?
any ideas how to debug/fix this? tia !
Assuming that the Prometheus pods did start, their collective hostnames would be found via service discovery like prometheus.monitoring.svc.cluster.local
https://cloud.google.com/kubernetes-engine/docs/concepts/service-discovery
You might also be interesting in exposing Prometheus itself
this issue is resolved, file prometheus-additional.yaml is already a secret yaml file, and the secret should be created using command
kubectl create -f prometheus-additional.yaml
instead of :
kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml
ref - https://issueexplorer.com/issue/strimzi/strimzi-kafka-operator/5873
seems the format of the file is changed in the recent strimzi releases
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.