kubernetes 在 queue_size 上自动缩放与运行的 pod 数量

Question

Is there a way I can set up autoscaling with a complete customized algorithm?
1. I would like to autoscale my pods based on the ratio of (size of the azure storage queue) to (number of pods currently running) which means the average number of messages every pod needs to handle.
2. Can I create metrics and scale based on the metrics freely? Like

if 2 > metrics > 1:
     scale pods to 2
if 5 > metrics > 2:
     scale pods to 3
if 10 > metrics > 5:
     scale pods to 4

if so, where can I set up the autoscale algorithm?

Answer 1

You can use keda with the corresponding trigger for that:

https://keda.sh/docs/1.4/scalers/azure-storage-queue/

Here is an example from the docs for azure storage queue.

 apiVersion: keda.k8s.io/v1alpha1
kind: TriggerAuthentication
metadata:
  name: azure-queue-auth
spec:
  podIdentity:
    provider: azure
---
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: azure-queue-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    deploymentName: azurequeue-function
  triggers:
  - type: azure-queue
    metadata:
      # Required
      queueName: functionsqueue
      # Required: connection OR authenticationRef that defines connection
      connection: STORAGE_CONNECTIONSTRING_ENV_NAME # Default: AzureWebJobsStorage. Reference to a connection string in deployment
      # or authenticationRef as defined below
      #
      # Optional
      queueLength: "5" # default 5
    authenticationRef:
        name: azure-queue-auth # 

All kinds of other triggers are supported as well.