[英]Unable to retrieve custom metrics from prometheus-adapter
i am trying to experiment with scaling one of my application pods running on my raspberry pi kubernetes cluster using HPA + custom metrics but ran into several issues which despite reading the documentations on https://github.com/DirectXMan12/k8s-prometheus-adapter and troubleshooting for the past 2 days, i am still having difficulties grasping why some problems are happening.我正在尝试使用 HPA + 自定义指标扩展在我的树莓派 kubernetes 集群上运行的应用程序 pod 之一,但遇到了几个问题,尽管阅读了https://github.com/DirectXMan12/k8s-prometheus-adapter上的文档和过去两天的故障排除,我仍然难以理解为什么会发生一些问题。
Firstly, i built an ARM-compatible image of k8s-prometheus-adapter and install it using helm.首先,我构建了一个与 ARM 兼容的 k8s-prometheus-adapter 映像并使用 helm 安装它。 I can confirm its running properly by checking the pod logs.我可以通过检查 pod 日志来确认它是否正常运行。
I have also set up a script which sends raspberry pis temperature to pushgateway and i can query via this Prometheus query node_temp
, which will return the following series我还设置了一个脚本,将树莓派温度发送到 pushgateway,我可以通过这个 Prometheus 查询node_temp
进行查询,它将返回以下系列
node_temp{job="kube4"} 42
node_temp{job="kube1"} 44
node_temp{job="kube2"} 39
node_temp{job="kube3"} 40
Now i want to be able to scale one of my application pods using the above temperature values as an experiment to understand better how it works.现在我希望能够使用上述温度值作为实验来扩展我的一个应用程序吊舱,以更好地了解它是如何工作的。
Below is my k8s-prometheus-adapter helm values.yml
file下面是我的 k8s-prometheus-adapter helm values.yml
文件
image:
repository: jaanhio/k8s-prometheus-adapter-arm
tag: latest
logLevel: 7
prometheus:
url: http://10.17.0.12
rules:
default: false
custom:
- seriesQuery: 'etcd_object_counts'
resources:
template: <<.Resource>>
name:
as: "etcd_object"
metricsQuery: count(etcd_object_counts)
- seriesQuery: 'node_temp'
resources:
template: <<.Resource>>
name:
as: "node_temp"
metricsQuery: count(node_temp)
After installing via helm, i ran kubectl get apiservices
and can see v1beta1.custom.metrics.k8s.io
listed.通过 helm 安装后,我运行kubectl get apiservices
并可以看到列出的v1beta1.custom.metrics.k8s.io
。
i then ran kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq
然后我运行kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq
and got the following kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq
并得到以下
{
"kind": "APIResourceList",
"apiVersion": "v1",
"groupVersion": "custom.metrics.k8s.io/v1beta1",
"resources": [
{
"name": "jobs.batch/node_temp",
"singularName": "",
"namespaced": true,
"kind": "MetricValueList",
"verbs": [
"get"
]
},
{
"name": "jobs.batch/etcd_object",
"singularName": "",
"namespaced": true,
"kind": "MetricValueList",
"verbs": [
"get"
]
},
]
i then tried to query the value of the registered node_temp metrics using kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
but got the following response Error from server (InternalError): Internal error occurred: unable to list matching resources
然后我尝试使用kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
查询已注册的 node_temp 指标的值,但得到以下响应Error from server (InternalError): Internal error occurred: unable to list matching resources
Questions:问题:
Why is the node_temp metrics associated with jobs.batch
resource type?为什么 node_temp 指标与jobs.batch
资源类型相关联?
Why am i not able to retrieve the value of metrics via kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
?为什么我无法通过kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
检索指标的值?
What is a definitive way of figuring the path of the query?确定查询路径的明确方法是什么? eg /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
i kinda trial and error until i got see somewhat of a response.例如/apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp
我有点反复试验,直到我看到一些回应。 i also see some other path with namespaces in the query eg /apis/custom.metrics.k8s.io/v1beta1/namespaces/*/metrics/foo_metrics
我还在查询中看到了一些其他带有命名空间的路径,例如/apis/custom.metrics.k8s.io/v1beta1/namespaces/*/metrics/foo_metrics
Any help and advice will be greatly appreciate!任何帮助和建议将不胜感激!
- Why is the node_temp metrics associated with jobs.batch resource type?为什么 node_temp 指标与 jobs.batch 资源类型相关联?
It picks the labels attached to the prometheus metrics and tries to interpret them, in this case u have clearely "job-kube4"它选择附加到普罗米修斯指标的标签并尝试解释它们,在这种情况下,你显然有“job-kube4”
- Why am i not able to retrieve the value of metrics via kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp?为什么我无法通过 kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/jobs/*/node_temp 检索指标的值?
Metrics are namespaced, see the "namespaced:true" so you'll need "/apis/custom.metrics.k8s.io/v1beta1/namespaces/ /jobs/ /node_temp"指标是命名空间的,请参阅“namespaced:true”,因此您需要“/apis/custom.metrics.k8s.io/v1beta1/namespaces/ /jobs/ /node_temp”
- What is a definitive way of figuring the path of the query?确定查询路径的明确方法是什么? eg /apis/custom.metrics.k8s.io/v1beta1/jobs/ /node_temp i kinda trial and error until i got see somewhat of a response.例如 /apis/custom.metrics.k8s.io/v1beta1/jobs/ /node_temp 我有点反复试验,直到我看到一些回应。 i also see some other path with namespaces in the query eg /apis/custom.metrics.k8s.io/v1beta1/namespaces/ /metrics/foo_metrics我还在查询中看到了一些其他带有命名空间的路径,例如 /apis/custom.metrics.k8s.io/v1beta1/namespaces/ /metrics/foo_metrics
Check https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md#api-paths检查https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md#api-paths
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.