HPA无法从Prometheus适配器获取自定义指标

Question

I have configured Prometheus-adapter to fetch custom metrics from Prometheus. When I execute the command: kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1

Following is the result.

 {
      "name": "namespaces/envoy_http_ingress_http_downstream_cx_http1",
      "singularName": "",
      "namespaced": false,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "namespaces/envoy_cluster_xds_cluster_upstream_cx_rx_bytes",
      "singularName": "",
      "namespaced": false,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "jobs.batch/statsd_exporter_lines",
      "singularName": "",
      "namespaced": true,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "pods/fs_writes_merged",
      "singularName": "",
      "namespaced": true,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },

My HPA configuration is as follows:

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: scale
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: billing-app
  minReplicas: 1
  maxReplicas: 10
  # targetCPUUtilizationPercentage: 50
  metrics:
    - type: External
      external:
        metricName: fs_writes_merged
        targetValue: 100

Hpa results in unknown. Not sure why it is not able to fetch metrics.

Hpa must be able to read the custom metrics.

Answer 1

Answer

Since your HPA configuration declares the metric as type: External , the HPA tries to fetch it from the External Metrics API ( /apis/custom.metrics.k8s.io ), but the Prometheus Adapter exposes it on the Custom metrics API ( /apis/custom.metrics.k8s.io ).

Since your metric comes from the Pods of the Deployment that you're trying to autoscale, you should use the Pods metric type. So, change your HPA configuration to:

  # ...
  metrics:
    - type: Pods
      pods:
        metricName: fs_writes_merged
        targetValue: 100

Background

You can see all the available HPA metric types and their usages here:

kubectl explain --api-version=autoscaling/v2beta1 hpa.spec.metrics

There are four metric types, and they map to the various metric APIs as follows:

• Resource : Resource Metrics API ( /apis/metrics.k8s.io/v1beta1 )
• Pods : Custom Metrics API ( /apis/custom.metrics.k8s.io/v1beta1 )
• Object : Custom Metrics API ( /apis/custom.metrics.k8s.io/v1beta1 )
• External : External Metrics API ( /apis/external.metrics.k8s.io/v1beta1 )

See HPA docs , and design documents for Resource Metrics API , Custom Metrics API , and External Metrics API .