AWS autoscale group scale in event

Question

I am using autoscale group for adding and removing additional instances for my application. I am using CPU Utilization as my scaling parameter and wondering what happens when an instance is running a program and the CPU Utilization comes below 65% (ie threshold value).

Does it wait for the instance to finish the program or terminate the instance at that moment? If it terminates the instance at that moment then it might lead to data loss/data inconsistency.

Any help would be appreciated.

Answer 1

If you're looking to prevent or delay an instance during a scale in event you could take a look at lifecycle hooks .

By enabling this autoscaling can send a notification that a specific instance action is about to occur (scale out or scale in). Using a combination of services (such as SNS, Lambda, SSM etc) you would be able to programmatically notify the instance that is is about to be terminated which you can then take any necessary actions.

The instance termination will wait until there is a confirmation to the autoscaling group that it has been completed which will lead to it being terminated. Additionally a lifecycle hook will have a timeout, if no confirmation is received by the time the timeout has been exceeded then the termination will still occur.

Answer 2

I think that you are looking for termination policy

Look at this link:

https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.html#default-termination-policy

And in my experience, the instance will be terminated no matter what it is running

Answer 3

Does it wait for the instance to finish the program or terminate the instance at that moment.

Sadly, I does not wait . ASG works outside of your instances and is not concerned with any programs running on your instance.

Having said that, there are few things you can do, some of which are described in:

• Controlling which Auto Scaling instances terminate during scale in

Generally speaking you should develop your applications to be stateless . This means the applications should be "aware" that they can be terminated at any time. One way to achieve is by using external storage systems , such as S3 or EFS, which will persist data between terminations.

Other way, is to use termination protection . In this case, the application will put its instance in this state at the beginning of processing, and then whey the calcuation finishes, the termination protection will be removed.