AWS ECS fargate task stopping and restarting somewhat randomnly

Question

One of my ECS fargate tasks is stopping and restarting in what seems to be a somewhat random fashion. I started the task in Dec 2019 and it has stopped/restarted three times since then. I've found that the task stopped and restarted from its 'Events' log (image below) but there's no info provided as to why it stopped..

So what I've tried to do to date to debug this is

1. Checked the 'Stopped' tasks inside the cluster for info as to why it might have stopped. No luck here as it appears 'Stopped' tasks are only held there for a short period of time.
2. Checked CloudWatch logs for any log messages that could be pertinent to this issue, nothing found
3. Checked CloudTrail event logs for any event pertinent to this issue, nothing found
4. Confirmed the memory and CPU utilisation is sufficient for the task, in fact the task never reaches 30% of it's limits
5. Read multiple AWS threads about similar issues where solutions mainly seem to be connected to using an ELB which I'm not..

Any have any further debugging device or ideas what might be going on here?

Answer 1

I ran into the same issue and found this from aws

https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-maintenance.html

When AWS determines that a security or infrastructure update is needed for an Amazon ECS task hosted on AWS Fargate, the tasks need to be stopped and new tasks launched to replace them.

Also a github post on storing stopped tasks info in cloudwatch logs:

https://github.com/aws/amazon-ecs-agent/issues/368