Change Instance type of a cluster registered ec2 instance

Question

I have an Amazon EC2 instance which is registered to a cluster of Amazon ECS.

And I want to change this instance's type from c4.large to c4.8xlarge .

I'm able to change its type from c4.large to c4.8xlarge in AWS console. But after the change, I found

[ERROR] Could not register module="api client" err="ClientException: Container instance type changes are not supported. Container instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX was previously registered as c4.large.

being printed in /var/log/ecs/ecs-agent.log.20XX-XX-XX-XX file.

Is it possible to change ec2 instance type and re-register it to a cluster?

I think maybe deregister it first, then register it again should work. But I'm afraid this may cause something irreversible in my AWS working environment. So I haven't tried this method yet.

Answer 1

To solve this connection problem between the agent and cluster, just delete the file /var/lib/ecs/data/ecs_agent_data.json and restart docker and ECS. After that, a new container instance will be created in your cluster with the new size.

sudo rm /var/lib/ecs/data/ecs_agent_data.json
sudo service docker restart
sudo start ecs

Then you can go to the ECS cluster console and deregister the old container instance

UPDATE: According to @florins and @MBear commented below, AWS updated the data file on ECS instances.

sudo rm /var/lib/ecs/data/agent.db
sudo service docker restart
sudo start ecs

Answer 2

As of March 2021 / AMI image ami-0db98e57137013b2d, /var/lib/ecs/data/ecs_agent_data.json mentioned in the last useful answer does not exist. For me, the commands to execute on the changed instance were:

sudo rm /var/lib/ecs/data/agent.db
sudo service docker restart

After that, it was possible to deploy containers to the instance, without fresh registration (AWS automatically registered a second ECS container instance of the new type). I did have a leftover container instance with the resources of the old instance type to remove.

Answer 3

You can't do this. Per their docs :

The type of EC2 instance that you choose for your container instances determines the resources available in your cluster. Amazon EC2 provides different instance types, each with different CPU, memory, storage, and networking capacity that you can use to run your tasks. For more information, see Amazon EC2 Instances.

This means that when you launch a container on an instance, the agent gathers a bunch of metadata about the instance to run it. If you change it, all of that metadata (or a lot) has changed in a bad way. CPU units, memory, etc. The agent is aware of this and will report it as an error .

You should spin up a new instance of the new type and register it to the cluster and let the task run on it. If it's a service, just terminate the old instance and let it run it against the new one.

I can't think of any real reason why terminating your old instance would cause something irreversible unless it is misconfigured or fragile via user specific settings, by default this would not cause anything destructive.

Answer 4

As alternative approach if the EC2 instance does not store any valuable a new instance using the old instance as template could be started. This takes all existing values and can be achieved just with a few clicks in minutes.

1. Select the EC2 instance and then "Actions -> Images and templates -> Start more like this". Just change the instance type.
2. When the instance is running got the the ECS cluster to the tab "ECS instances" and activate the new created instance.
3. Shutdown the old instance
4. Update your task maybe taking more cpu and memory and update the service to take the new task revision