如何编写 prometheus 服务监视器以从 k8s 节点上的服务中抓取指标

Question

Team, I am new to prometheus. Please assist.

I have a k8s node and on that node locally i have a service running (lustre-exporter) which is exposing metrics as shown below.

node:~$ curl http://localhost:9169/metrics | head -n 10
go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 8.4924e-05
go_gc_duration_seconds{quantile="0.25"} 9.0081e-05
go_gc_duration_seconds{quantile="0.5"} 9.2247e-05

I want to know how would i write a service monitor such that above is scraped by prometheus.

I tried below but getting syntax error from k8s.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: lustre-exporter
  namespace: team-monitoring
spec:
  endpoints:
    - path: /metrics
      port: "9169"

The ServiceMonitor "lustre-exporter" is invalid: []: Invalid value: map[string]interface {}{"metadata":map[string]interface {}{"name":"lustre-exporter", "namespace":"team-monitoring", "creationTimestamp":"2022-07-01T02:14:16Z", "generation":1, "uid":"824d31a3-f8e3-11ec-b65c-ac1f6b4ea082"}, "spec":map[string]interface {}{"endpoints":[]interface {}{map[string]interface {}{"path":"/metrics", "port":"9169"}}}, "apiVersion":"monitoring.coreos.com/v1", "kind":"ServiceMonitor"}: validation failure list:
spec.selector in body is required

Answer 1

figured out and thanks for hint in comment but label selector was missing too.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: lustre-exporter
  namespace: team-monitoring
spec:
  endpoints:
    - path: /metrics
      port: "9169"
  selector:
   nodeGroup: gpu