Autoscale Docker Swarm Services

Question

We are using Ansible along with AWS Autoscaling Groups to scale our docker swarm cluster. Right now this is working because we run the services in global mode so we automatically get one of each container type per docker node. However this is not very efficient because we end up with more containers of some services than we need.

Is anyone doing something similar but having docker swarm scale out specific services based on their resource utilization? It looks like it would be possible by leveraging docker stats to figure out which containers need to scale but don't want to re-invent the wheel if there's already a good solution for this.

EDIT: not looking for how to scale the swarm cluster itself but individual services running on it.

Answer 1

Take a look at kubernetes.

Kubernetes (commonly referred to as "K8s") is an open source container cluster manager originally designed by Google. It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts". It usually works with the Docker container tool and coordinates between a wide cluster of hosts running Docker.

Now lets get to the point:

Kubernetes Horizontal Autoscaling:

With Horizontal Pod Autoscaling, Kubernetes automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization (or, with alpha support, on some other, application-provided metrics).

More info