How to detect ECS fargate task autoscaling events like LifeCycleHook

Question

I have ECS container running some tasks. The server running inside the task may take 1~10 minutes to complete one request.

I am using SQS for task queuing. When certain amount tasks exceeds it scale-up the ECS tasks. And it scale-down when task in queue go below certain numbers.

However, as there is no LifeCycleHook feature for ECS task, during the time of scale-down the ECS tasks are shut down while the processing is still running. And it's not possible to delay the task termination due to the lack of LifeCycleHook.

According to our specification, we can't use the timeout feature, as we don't know earlier how much time it will take to finish the job.

Please suggest how to solve the problem.

Answer 1

There is no general solution to this problem, especially if you don't want to use timeout. In fact there is long lasting, still open, github issue dedicated to this:

• [ECS] [request]: Control which containers are terminated on scale in

You could somehow control this through running your services on EC2 (EC2 scale-in protection), not Fargate. So either you have to re-architect your solution, or manually scale-out and in your service.