How do wait for AWS ECS Task to be in RUNNING state before moving to next step in AWS Step Function?

Question

I have an array of objects which I want to pass to a Lambda function iteratively. However, I also need to have an ECS Task running for each Lambda function I start.

I found that I would need to have an AWS Step Function where I iterate through an array of JSON inputs. For each input, I would have to start an ECS Task, wait for it to be in a RUNNING state, then move to next step where I invoke a Lambda function. In my case, the ECS Task itself does not return anything. It is supposed to stay running because the Lambda function uses it.

Currently, I have it so that the ECS Task starts but it stays stuck in the starting the ECS Task step because it does not return anything. How would I be able to wait for it be in a RUNNING state before moving to the next step?

Current Step Function definition:

{
  "StartAt": "Iterate",
  "States": {
    "Iterate": {
      "Type": "Map",
      "Iterator": {
        "StartAt": "Start ECS Task",
        "States": {
          "Start ECS Task": {
            "Type": "Task",
            "Resource": "arn:aws:states:::ecs:runTask",
            "Parameters": {
              "LaunchType": "FARGATE",
              "Cluster": "<cluster-arn>",
              "TaskDefinition": "<task-definition-arn>",
              "NetworkConfiguration": {
                "AwsvpcConfiguration": {
                  "Subnets": [
                    "<subnet-id>"
                  ],
                  "AssignPublicIp": "ENABLED"
                }
              }
            },
            "Next": "Invoke Lambda function"
          },
          "Invoke Lambda function": {
            "Type": "Task",
            "Resource": "arn:aws:states:::lambda:invoke",
            "Parameters": {
              "FunctionName": "<lambda-function-arn>",
              "Payload": {
                "Input.$": "$"
              }
            },
            "End": true
          }
        }
      },
      "End": true
    }
  }
}

Answer 1

ecs:runTask can be triggered in two different ways.

• arn:aws:states:::ecs:runTask.sync > runs and waits until task is completed
• arn:aws:states:::ecs:runTask.waitForTaskToken - triggers and wait until it receives a SendTaskSucess or sendTaskFailure

We need to use second method in this case,

Pass TASK_TOKEN as environment variable to ECS task and within ECS task first few lines of code, we need to send a SendTaskSucess.

Here is an example:

{
   "StartAt":"Run",
   "States":{
      "Run":{
         "End":true,
         "Type":"Task",
         "Resource":"arn:aws:states:::ecs:runTask.waitForTaskToken",
         "Parameters":{
            "Cluster":"arn:aws:ecs:us-east-1:620018741331:cluster/HelloCdkStack-ecstaskEc2ClusterB0EAAA1E-yPWjSYf8d03O",
            "TaskDefinition":"HelloCdkStackecstaskTD1950FF01",
            "Overrides":{
               "ContainerOverrides":[
                  {
                     "Name":"TheContainer",
                     "Environment":[
                        {
                           "Name":"TASK_TOKEN",
                           "Value.$":"$$.Task.Token"
                        }
                     ]
                  }
               ]
            },
            "LaunchType":"EC2"
         }
      }
   }
}

Answer 2

Instead of using an integrated AWS service (ie AWS Step Function), I created my own script for starting ECS Tasks using aws-sdk v3 for JavaScript. I specifically used the waitUntilTasksRunning function from the @aws-sdk/client-ecs package to wait for the ECS Task to be in a RUNNING state.

Read more about it from the documentation

• AWS SDK for JavaScript v3: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html
• @aws-sdk/client-ecs: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/index.html
• @aws-sdk/client-ecs waitUntilTasksRunning: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/globals.html#waituntiltasksrunning

Sample function for running and waiting for the ECS Task to be in a RUNNING state (TypeScript) note: not all parameters in the functions are required, check the documentation for that:

import { ECSClient, RunTaskCommand, waitUntilTasksRunning } from '@aws-sdk/client-ecs'

const startAndWaitUntilECSTaskRunning = async (region: string, clusterARN: string, launchType: string, subnets: Array<string>, taskDefinition: string, securityGroups: Array<string>, assignPublicIp: string) => {
    var ecsClient = new ECSClient({ "region": region })
    var runTaskCommand = new RunTaskCommand({
        cluster: clusterARN,
        taskDefinition: taskDefinition,
        launchType: launchType,
        networkConfiguration: {
            awsvpcConfiguration: {
                assignPublicIp: assignPublicIp,
                subnets: subnets,
                securityGroups: securityGroups
            }
        }
    })
    var ecsTask = await ecsClient.send(runTaskCommand)
    var taskArn: string | undefined = ecsTask.tasks?.[0].taskArn
    if (typeof taskArn !== "string") {
        throw Error("Task ARN is not defined.")
    }
    var waitECSTask = await waitUntilTasksRunning({"client": ecsClient, "maxWaitTime": 600, "maxDelay": 1, "minDelay": 1}, {"cluster": clusterARN, "tasks": [taskArn]})
    // note: there are multiple waitECSTask states, check the documentation for more about that
    if (waitECSTask.state !== 'SUCCESS') {
        // your code to handle this
    } else {
        // your code to handle this
    }
}