安排 AWS ECS Fargate 启动和停止任务

Question

Per AWS documentation they say: " Fargate supports scheduling tasks in response to CloudWatch Events. You can easily launch and stop Fargate tasks that you only need to run at certain times "

Start of Fargate Tasks can easily be enabled from CloudWatch Events or ECS Task Scheduler.

But STOP of Fargate Tasks I cannot find. Is it possible that need to use Lambda and not native Fargate feature to stop tasks?

My goal is to run ECS Container between 8ap-5pm Only!

Answer 1

answer by @Francesco Grotta is right. By we can create the following resources, to trigger this action on schedule:

• A lambda function, to start or stop by update the ECS service DesiredCount .
• A scheduled CloudWatch Events, to start the ECS tasks.
• A scheduled CloudWatch Events, to stop the ECS tasks.

Lambda Function that will start or stop the ECS service based on the input from CloudWatch Events:

    if(event.status == 'stop'){
        var params = {
            cluster: process.env.ECS_CLUSTER,
            service: process.env.ECS_SERVICE_NAME,
            desiredCount: 0
        };
    }
    else{
        var params = {
            cluster: process.env.ECS_CLUSTER,
            service: process.env.ECS_SERVICE_NAME,
            desiredCount: 1
        };
    }

    var ecs = new AWS.ECS();
    ecs.updateService(params, function (err, data) {
        if (err) console.log(err, err.stack); // an error occurred
        else console.log(data);           // successful response
    });

In Cloudformation template, create resources that will invoke the Lambda Function on a schedule:

  StartEcsLambdaSchedule:
    Type: AWS::Events::Rule
    Properties:
      Description: >
        A schedule for the Lambda function to start ECS service during office hours.
      ScheduleExpression: !Ref StartEcsLambdaScheduleExpression
      State: ENABLED
      Targets:
        - Arn: !Sub ${EcsTaskScheduleLambdaFunction.Arn}
          Id: StartEcsLambdaScheduleV1
          Input: '{"status": "start"}'

  StopEcsLambdaSchedule:
    Type: AWS::Events::Rule
    Properties:
      Description: >
        A schedule for the Lambda function to stop ECS service after office hours.
      ScheduleExpression: !Ref StopEcsLambdaScheduleExpression
      State: ENABLED
      Targets:
        - Arn: !Sub ${EcsTaskScheduleLambdaFunction.Arn}
          Id: StopEcsLambdaScheduleV1
          Input: '{"status": "stop"}'

Answer 2

I've taken the code @Mr3381 proposed and made some improvements:

interface EventLambda {
  status: string,
  services: string[]
}

interface UpdateServiceParams {
  cluster: string
  service: string
  desiredCount: number
}

// Load AWS SDK for Node.js
import AWS from 'aws-sdk';

export const handler = async (event: EventLambda): Promise<string[]> => {
  const ecs = new AWS.ECS({region: 'sa-east-1'});
  const promises = new Array<Promise<any>>();
  const desiredCount = event.status == 'start' ? 1 : 0

  event.services.forEach(service => {
      var params: UpdateServiceParams = {
        cluster: process.env.ECS_CLUSTER!,
        service,
        desiredCount
      };
      promises.push(updateService(ecs, params, desiredCount))
    }
  )
  return Promise.all(promises)
};

function updateService(ecs: AWS.ECS, params: UpdateServiceParams, desiredCount: number): Promise<string> {
  return new Promise((resolve, reject) => {
    ecs.updateService(params, function(err, data) {
      if (err) {
        console.log(err, err.stack); // An error occurred
        resolve(`${params.service} not updated`);
      }
      else {
        console.log(data); // Successful response
        resolve(`${params.service} updated => Desired count: ${desiredCount}`)
      }
    });
  })
}

It's now in TypeScript and supports an array of services as an input.

There's only the need to transpile it to JavaScript to be able to run on AWS.

Make sure to have the required permissions to execute the function and update your services. This can be achieved by attaching the policies describeServices and updateServices from ECS to your Lambda function IAM Role.

Answer 3

I believe this is the best solution for your office hours problem.

Just schedule a local cron job in the EC2 instance with the below command to update desired count according the timestamp and you are good to go.

$ aws ecs update-service --cluster <cluster-name> --service <service-name> --desired-count x

Do let me know in case this helped you.

Answer 4

It's the same to Start, but you need specify "Desired task" to 0. My issue is that the Scheduler need the Task Definition, so if I upgrade it, I need change also the Scheduler. Exists some service, lambda or project to handle it? My need is to stop a Cluster of Fargate container.

Answer 5

The following will find out the all the qa, uat or dev clusters (Provided they have the string qa, uat or dev in their name) and then set the number of tasks for all the services under them to 0. Just cron this script and create another cron for --desired-count 1 if you want to start all non-prod environments. Create /tmp/ecs-stop.log file to use the script as is. Make sure that the file is writable and the script is executable for cron to be able to run the script and write to the log file.

#!/bin/bash
export AWS_PAGER="" # Doesnt ask user to press q for paginated outputs
export AWS_PROFILE=default
d=$(date +%Y-%m-%d)
for cluster in $(/usr/local/bin/aws ecs list-clusters --output text | awk '{print $2}' | egrep -i 'qa|uat|dev'  )
do 
    echo -e "Cluster ==> $cluster"
    for service in $(/usr/local/bin/aws ecs list-services --cluster $cluster --output text | awk '{print $2}')
    do
        echo "Service ==> Stopping $service..."
        /usr/local/bin/aws ecs update-service --cluster $cluster --service $service --desired-count 0 > /dev/null 2>&1
    done
echo
done
echo "Stopped all non-prod ECS services."
echo "The script ran on $d" >> /tmp/ecs-stop.log

Answer 6

This is my solution which is highly based on a great AWS employee (Alfredo J).

First, you need to create a Lambda function and add this Python script:

import json
import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

client = boto3.client('ecs')

def lambda_handler(event, context):
    cluster = event["cluster"]
    service_names = event["service_names"]
    service_desired_count = int(event["service_desired_count"])

    for service_name in service_names.split(","):
        response = client.update_service(
            cluster=cluster,
            service=service_name,
            desiredCount=service_desired_count
            )

        logger.info("Updated {0} service in {1} cluster with desire count set to {2} tasks".format(service_name, cluster, service_desired_count))
        
    return {
        'statusCode': 200,
        'new_desired_count': service_desired_count
    }

The script expects the following variables in a JSON format:

{
  "cluster": "clusterName",
  "service_names": "service1,service2",
  "service_desired_count": "0"
}

Where:

• cluster is the name of the cluster you want to modify.
• service_names is an array for the collection of services.
• service_desired_count is the number of desired services. 0 is to stop the service/s, any other number is to start the service/s.

After everything is created you need to create some rules in Amazon EventBridge (formerly, CloudWatch Events). Here, you define the event you want to trigger based on the schedule that you expect.

If something fails, you need to double-check that the created IAM role has the required policies like: ecs:UpdateService . You can check this from the logs.