Scaling microservices using Docker

Question

I've created a Node.js (Meteor) application and I'm looking at strategies to handle scaling in the future. I've designed my application as a set of microservices, and I'm now considering implementing this in production.

What I'd like to do however is have many microservices running on one server instance to maximise resource usage whilst they are using a small number of resources. I know containers are useful for this, but I'm curious if there's a way to create a dynamically scaling set of containers where I can:

• Write commands such as "provision another app container on this server if the containers running this app reach > 80% CPU/other limiting metrics",
• Provision and prepare other servers if needed for extra containers,
• Load balance connections between these containers (and does this affect server load balancing, eg, send less connections to servers with fewer containers?)

I've looked into AWS EC2, Docker Compose and nginx, but I'm uncertain if I'm going in the right direction.

Answer 1

Investigate Kubernetes and/or Mesos, and you'll never look back. They're tailor-made for what you're looking to do. The two components you should focus on are:

1. Service Discovery: This allows inter-dependent services (micro-service "A" calls "B") to "find" each other. It's typically done using DNS, but with registration features on top of it that handle what happens as instances are scaled.

2. Scheduling: In Docker-land, scheduling isn't about CRON jobs, it means how containers are scaled and "packed" into servers in various ways to maximize efficient usage of available resources.

There are actually dozens of options here: Docker Swarm, Rancher, etc. are also competing alternatives. Many cloud vendors like Amazon also offer dedicated services (such as ECS) with these features. But Kubernetes and Mesos are emerging as standard choices, so you'd be in good company if you at least start there.

Answer 2

Metrics could be collected via Docker API ( and cool blog post ) and it's often used for that. Tinkering with DAPI and docker stack tools (compose/swarm/machine) could provide alot of tools to scale microservice architecture efficiently.

I could advise in favor of Consul to manage discovery in such resource-aware system.

Answer 3

We are using AWS to host our miroservices application, and using ECS (AWS docker service) to containerize the different API.

And in this context, we use AWS auto scaling feature to manage the scale in and scale out. Check this .

Hope it helps.