Get data from AWS ECS run task back to AWS Lambda

Question

I am running my container in ECS Fargate using Lambda. But I want output of my container back to my Lambda which seems not possible directly as per How to pass & return the values from lambda to ECS task

I also read its possible here https://nuvalence.io/insights/aws-step-function-integration-with-ecs-or-fargate-tasks-data-in-and-out/ but unable to work it out. My docker image returns few lines and want to capture in s3 and read back in Lambda again. My Lambda is as follows which is not working

def lambda_handler(event, context):
    client = boto3.client('ecs')
    run_task = client.run_task(
        cluster='arn:aws:ecs:us-east-1:12345:cluster/listmodels',
        enableExecuteCommand=True,
        group='family:listmodels',
        launchType='FARGATE',
        networkConfiguration={
            'awsvpcConfiguration': {
                'subnets': [
                    'subnet-12345',
                ],
                'securityGroups': [
                    'sg-12345',
                ],
                'assignPublicIp': 'ENABLED'
            }
        },
        overrides={
            'containerOverrides': [
                {
                    'name': 'listmodels',
                    'command': [
                        "ls",
                        "--target",
                        "dev"
                    ],
                },
            {'Name':'OUTPUT_LOCATION','value':'s3://sm1-retail/1.txt'}
            ],
            'executionRoleArn': 'arn:aws:iam::509094867785:role/ecsTaskExecutionRole',
            'taskRoleArn': 'arn:aws:iam::509094867785:role/ecsTaskExecutionRole',
        },
        propagateTags='TASK_DEFINITION',
        taskDefinition='listmodels-task'
    )

Getting below error and Iam sure this error is because I added
{'Name':'OUTPUT_LOCATION','value':'s3://sm1-retail/1.txt'}

  "errorMessage": "Parameter validation failed:\nUnknown parameter in overrides.containerOverrides[1]: \"Name\", must be one of: name, command, environment, environmentFiles, cpu, memory, memoryReservation, resourceRequirements\nUnknown parameter in overrides.containerOverrides[1]: \"value\", must be one of: name, command, environment, environmentFiles, cpu, memory, memoryReservation, resourceRequirements",
  "errorType": "ParamValidationError",

I would like to know what is better approach to get output of my run_task? Since run_task does not return anything to calling function, I thought of writing logs to s3 and read it from there and that is also failing. Any help is appreciated.

USECASE: I need to run docker image and the ouput of docker image is few file names and based on that my program decides which all files got changed and take necessary action

Answer 1

Your are missing environment argument . It should be:

        overrides={
            'containerOverrides': [
                {
                    'name': 'listmodels',
                    'command': [
                        "ls",
                        "--target",
                        "dev"
                    ],
                    'environment': [
                            {
                                'name': 'OUTPUT_LOCATION',
                                'value': 's3://sm1-retail/1.txt'
                            },
                        ]  
                },
            ],