[英]OpenTelemetry doesn't show metrics in Prometheus
I've setup OpenTelemetry in Kubernetes.我已经在 Kubernetes 中设置了 OpenTelemetry。 Below is my config.下面是我的配置。
exporters:
logging: {}
extensions:
health_check: {}
memory_ballast: {}
processors:
batch: {}
memory_limiter:
check_interval: 5s
limit_mib: 819
spike_limit_mib: 256
receivers:
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_compact:
endpoint: 0.0.0.0:6831
thrift_http:
endpoint: 0.0.0.0:14268
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
prometheus:
config:
scrape_configs:
- job_name: opentelemetry-collector
scrape_interval: 10s
static_configs:
- targets:
- ${MY_POD_IP}:8888
zipkin:
endpoint: 0.0.0.0:9411
service:
extensions:
- health_check
- memory_ballast
pipelines:
logs:
exporters:
- logging
processors:
- memory_limiter
- batch
receivers:
- otlp
metrics:
exporters:
- logging
processors:
- memory_limiter
- batch
receivers:
- otlp
- prometheus
traces:
exporters:
- logging
processors:
- memory_limiter
- batch
receivers:
- otlp
- jaeger
- zipkin
telemetry:
metrics:
address: 0.0.0.0:8888
The endpoint is showing as up in Prometheus.端点在 Prometheus 中显示为 up。 But it doesn't show any data.但它没有显示任何数据。 When I check the OTEL collector logs, it shows as below当我检查 OTEL 收集器日志时,它显示如下
I have manually added the scrape config in Prometheus.我已经在 Prometheus 中手动添加了抓取配置。
scrape_configs: scrape_configs:
- job_name: 'otel-collector'
scrape_interval: 10s
static_configs:
- targets: ['opentelemetry-collector.opentelemetry:8888']
So in OTEL collectore configmap I also see Prometheus scrape config.所以在 OTEL collectore configmap 中,我还看到了 Prometheus scrape 配置。
prometheus:
config:
scrape_configs:
- job_name: opentelemetry-collector
scrape_interval: 10s
static_configs:
- targets:
- ${MY_POD_IP}:8888
--Added--New-- --添加--新--
kubectl get all -n thanos
NAME READY STATUS RESTARTS AGE
pod/thanos-query-776688f499-pvm24 1/1 Running 0 14h
pod/thanos-query-frontend-5b55d44cc-b6qx5 1/1 Running 0 14h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/thanos-query ClusterIP 10.0.112.105 <none> 9090/TCP,10901/TCP 14h
service/thanos-query-frontend ClusterIP 10.0.223.246 <none> 9090/TCP 14h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/thanos-query 1/1 1 1 14h
deployment.apps/thanos-query-frontend 1/1 1 1 14h
NAME DESIRED CURRENT READY AGE
replicaset.apps/thanos-query-776688f499 1 1 1 14h
replicaset.apps/thanos-query-frontend-5b55d44cc 1 1 1 14h
--Logs-- --日志--
2022-06-07T07:20:49.852Z error exporterhelper/queued_retry.go:183 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "name": "prometheusremotewrite", "error": "Permanent error: Permanent error: remote write returned HTTP status 404 Not Found; err = <nil>: 404 page not found\n", "dropped_items": 18}
go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry.go:183
go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send
go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/metrics.go:132
go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1
go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry_inmemory.go:118
go.opentelemetry.io/collector/exporter/exporterhelper/internal.consumerFunc.consume
go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:82
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func2
go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:69
I configured the below URL in OpenTelemetry我在 OpenTelemetry 中配置了以下 URL
exporters:
prometheusremotewrite:
endpoint: "http://thanos-query-frontend.thanos:9090/api/v1/write"
You have configured only logging
exporter, which exports data to the console via zap.Logger only = it doesn't write data to the prometheus.您只配置了logging
导出器,它仅通过 zap.Logger 将数据导出到控制台 = 它不会将数据写入普罗米修斯。
Configure also prometheusremotewrite
exporter and add it to the metric pipeline.还配置prometheusremotewrite
导出器并将其添加到度量管道。 Minimalistic example:简约示例:
receivers:
prometheus:
config:
scrape_configs:
- job_name: opentelemetry-collector
scrape_interval: 10s
static_configs:
- targets: ['localhost:8888']
exporters:
prometheusremotewrite:
endpoint: <example: my-prometheus/api/v1/write>
service:
pipelines:
metrics:
receivers:
- prometheus
exporters:
- prometheusremotewrite
telemetry:
metrics:
address: 0.0.0.0:8888
level: basic
See doc for prometheusremotewrite
exporter: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/prometheusremotewriteexporter有关prometheusremotewrite
导出器,请参阅文档: https ://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/prometheusremotewriteexporter
See https://grafana.com/grafana/dashboards/15983 if you want to have a Grafana dashboard for OpenTelemetry collector telemetry metrics.如果您想要一个用于 OpenTelemetry 收集器遥测指标的 Grafana 仪表板,请参阅https://grafana.com/grafana/dashboards/15983 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.