Kubernetes Pod和服务未在Prometheus目标中显示

Question

我已经在由kubeadm制成的多节点kubernetes集群上部署了prometheus 2.0。 访问prometheus仪表板时，即使在prometheus配置yaml文件中配置了pod和服务作业，我也无法查看pod和服务作业。 普罗米修斯的目标如下https://i.stack.imgur.com/jiQPG.png 。 这个问题与普罗米修斯版本有关吗？ 我认为配置的语法部分出了问题。

 global: scrape_interval: 5s scrape_timeout: 5s evaluation_interval: 5s scrape_configs: - job_name: kubernetes-apiservers scrape_interval: 5s scrape_timeout: 5s metrics_path: /metrics scheme: https kubernetes_sd_configs: - api_server: null role: endpoints namespaces: names: [] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: false relabel_configs: - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name] separator: ; regex: default;kubernetes;https replacement: $1 action: keep - job_name: kubernetes-nodes scrape_interval: 5s scrape_timeout: 5s metrics_path: /metrics scheme: https kubernetes_sd_configs: - api_server: null role: node namespaces: names: [] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: false relabel_configs: - separator: ; regex: __meta_kubernetes_node_label_(.+) replacement: $1 action: labelmap - separator: ; regex: (.*) target_label: __address__ replacement: kubernetes.default.svc:443 action: replace - source_labels: [__meta_kubernetes_node_name] separator: ; regex: (.+) target_label: __metrics_path__ replacement: /api/v1/nodes/${1}/proxy/metrics action: replace - job_name: kubernetes-pods scrape_interval: 5s scrape_timeout: 5s metrics_path: /metrics scheme: https kubernetes_sd_configs: - api_server: null role: pod namespaces: names: [] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: false relabel_configs: - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] separator: ; regex: "true" replacement: $1 action: keep - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] separator: ; regex: (.+) target_label: __metrics_path__ replacement: $1 action: replace - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port] separator: ; regex: ([^:]+)(?::\\d+)?;(\\d+) target_label: __address__ replacement: $1:$2 action: replace - separator: ; regex: __meta_kubernetes_pod_label_(.+) replacement: $1 action: labelmap - source_labels: [__meta_kubernetes_namespace] separator: ; regex: (.*) target_label: kubernetes_namespace replacement: $1 action: replace - source_labels: [__meta_kubernetes_pod_name] separator: ; regex: (.*) target_label: kubernetes_pod_name replacement: $1 action: replace - job_name: kubernetes-cadvisor scrape_interval: 5s scrape_timeout: 5s metrics_path: /metrics scheme: https kubernetes_sd_configs: - api_server: null role: node namespaces: names: [] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: false relabel_configs: - separator: ; regex: __meta_kubernetes_node_label_(.+) replacement: $1 action: labelmap - separator: ; regex: (.*) target_label: __address__ replacement: kubernetes.default.svc:443 action: replace - source_labels: [__meta_kubernetes_node_name] separator: ; regex: (.+) target_label: __metrics_path__ replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor action: replace - job_name: kubernetes-service-endpoints scrape_interval: 5s scrape_timeout: 5s metrics_path: /metrics scheme: https kubernetes_sd_configs: - api_server: null role: endpoints namespaces: names: [] relabel_configs: - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] separator: ; regex: "true" replacement: $1 action: keep - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] separator: ; regex: (https?) target_label: __scheme__ replacement: $1 action: replace - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] separator: ; regex: (.+) target_label: __metrics_path__ replacement: $1 action: replace - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] separator: ; regex: ([^:]+)(?::\\d+)?;(\\d+) target_label: __address__ replacement: $1:$2 action: replace - separator: ; regex: __meta_kubernetes_service_label_(.+) replacement: $1 action: labelmap - source_labels: [__meta_kubernetes_namespace] separator: ; regex: (.*) target_label: kubernetes_namespace replacement: $1 action: replace - source_labels: [__meta_kubernetes_service_name] separator: ; regex: (.*) target_label: kubernetes_name replacement: $1 action: replace 

谢谢

Answer 1

我假设您正在执行您在评论中发布的查询：

container_memory_usage_bytes{job="kubernetes-pods"}

这不起作用，因为您按作业名称kubernetes-pods进行过滤，但是container_memory_usage_bytes来自cAdvisor。 因此，根据您的配置，该作业名为kubernetes-cadvisor 。

因此，这应该工作：

container_memory_usage_bytes{job="kubernetes-cadvisor"}

由于系列名称非常独特，因此您可以省略作业名称：

container_memory_usage_bytes