Horizontal pod autoscaling using a logging custom metric in GKE

Question

I would like to scale my deployment based on a custom logging metric, but I'm not able to make that work, I created already the custom metric and I'm also able to see it in the metric explorer but for some reason the stackdriver adapter is not able to get the metric values.

This is my hpa.yaml

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-hpa
spec:
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - external:
      metricName: logging.googleapis.com|user|http_request_custom
      targetValue: "20"
    type: External
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx

But i'm always getting the following error:

"unable to get external metric default/logging.googleapis.com|user|http_request_custom/nil: unable to fetch metrics from external metrics API: the server could not find the requested resource (get logging.googleapis.com|user|http_request_custom.external.metrics.k8s.io)"

Should i do something different?? any idea?

Answer 1

Not sure you have created the service account and granted access to the adapter however there is two models of custom metrics adapter. Legacy adapter and new resource version.

If adapter is up and running did you check the logs of POD?

New resource model to install:

kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/k8s-stackdriver/master/custom-metrics-stackdriver-adapter/deploy/production/adapter_new_resource_model.yaml

ref yaml you can use this way further metrics

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: pubsub
spec:
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - external:
      metric:
       name: pubsub.googleapis.com|subscription|num_undelivered_messages
       selector:
         matchLabels:
           resource.labels.subscription_id: echo-read
      target:
        type: AverageValue
        averageValue: 2
    type: External
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: pubsub

Ref: https://cloud.google.com/kube.netes-engine/docs/tutorials/autoscaling-metrics#pubsub_4