Minikube horizontal pod autoscaling error: unable to get metrics for resource cpu: no metrics returned

Question

I'v enabled heapster on minikube

minikube addons start heapster

And custom metrics with

minikube start --extra-config kubelet.EnableCustomMetrics=true

My deployment looks like

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: kubia
spec:
  replicas: 1
  template:
    metadata:
      name: kubia
      labels:
        app: kubia
      annotations:
        pod.beta.kubernetes.io/init-containers: '[
            {
                "name": "setup",
                "image": "busybox",
                "imagePullPolicy": "IfNotPresent",
                "command": ["sh", "-c", "echo \"{\\\"endpoint\\\": \\\"http://$POD_IP:8080/metrics\\\"}\" > /etc/custom-metrics/definition.json"],
                "env": [{
                  "name": "POD_IP",
                  "valueFrom": {
                    "fieldRef": {
                      "apiVersion": "v1",
                      "fieldPath": "status.podIP"
                    }
                  }
                }],
                "volumeMounts": [
                    {
                        "name": "config",
                        "mountPath": "/etc/custom-metrics"
                    }
                ]
            }
        ]'
    spec:
      containers:
      - image: luksa/kubia:qps
        name: nodejs
        ports:
          - containerPort: 8080
        volumeMounts:
        - name: config
          mountPath: /etc/custom-metrics
        resources:
          requests:
            cpu: 100m
      volumes:
      - name: config
        emptyDir:

My hpa looks like

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: kubia
  annotations:
    alpha/target.custom-metrics.podautoscaler.kubernetes.io: '{"items":[{"name":"qps", "value": "20"}]}'
spec:
  maxReplicas: 5
  minReplicas: 1
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: kubia
  targetCPUUtilizationPercentage: 1000000

However I get target unknown

jonathan@ubuntu ~> kubectl get hpa
NAME      REFERENCE          TARGETS                MINPODS   MAXPODS   REPLICAS   AGE
kubia     Deployment/kubia   <unknown> / 1000000%   1         5         1          31m

And the following warnings from the hpa

  Warning  FailedGetResourceMetric       27m (x12 over 33m)  horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from heapster
  Warning  FailedComputeMetricsReplicas  27m (x12 over 33m)  horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from heapster

Answer 1

Ensure the metrics-server addons is enabled on minikube.

When I start minikube () I have the following addons enabled by default:

$ minikube addons list
- addon-manager: enabled
- coredns: disabled
- dashboard: enabled
- default-storageclass: enabled
- efk: disabled
- freshpod: disabled
- heapster: enabled
- ingress: disabled
- kube-dns: enabled
- metrics-server: disabled
- registry: disabled
- registry-creds: disabled
- storage-provisioner: enabled

Enable the metrics server and HPAs appear to work great.

$ minikube addons enable metrics-server
metrics-server was successfully enabled

Answer 2

metrics-server monitoring needs to be deployed in the cluster to provide metrics via the resource metrics API, as Horizontal Pod Autoscaler uses this API to collect metrics.

Thus enable metrics-server addon via;

$ minikube addons enable metrics-server

Refer - Horizontal Pod Autoscaler Walkthrough - Before you begin