Terraform aws secrets - nested object variable

Question

I have a terraform template that create aws ecs task. I filled a variable with a list of object like this:

` variables.tf

variable "microservices" {
  description = "the microservices to implement"
  type = list(object({
    name = string,
    port = number,
    secrets = optional(list(object({
      key = string,
      arn = string
    })))
  }))

`

Then in my main.tf I have the following: ` main.tf

resource "aws_ecs_task_definition" "task_definition" {
  count = length("${var.microservices}")
  family = "${var.microservices[count.index].name}-${var.environment}"
  requires_compatibilities = ["FARGATE"]
  network_mode = "awsvpc"
  cpu = 1024
  memory= 2048
  execution_role_arn = "arn:aws:iam::xxxxx:role/service-role/xxxx-test-service-role"
  container_definitions = jsonencode([
    {
      name      = "${var.microservices[count.index].name}"
      image     = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
      cpu       = 1
      essential = true
      Ulimits = [{
       Name = "nofile"
       SoftLimit = 65535
       HardLimit = 65535

      }]
      //length("${var.microservices[count.index].secrets}") > 0 ? 
      Secrets = [{ 
        Name = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].key}" : 0
        ValueFrom = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].arn}" : 0
        //Name = "${var.microservices[count.index].secrets[0].key}" 
        //ValueFrom = "${var.microservices[count.index].secrets[0].arn}" 

`

I don't understand how can I create Secrets parsing the variables. The secrets can be optional (it could exist or not). I should need a sort of for_each only in Secrets section in order to check if secret exist in input and then fill this filed.

An example of inputs is the following: `

  microservices = [
    { 
    "name" = "api", 
    "port" = 3000, 
    "secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
    { 
    "name" = "web", 
    "port" = 3000 
    "secrets" = [{ "key" = "test456", "arn" = "4567"}]
    }]

`

Anyone approach this kind of issue/configuration? What I would like to achieve is to create a task definition in aws ecs with secrets field (or empty secrets section) based on microservices input.

I tested a different data structure like here: flatten object made of nested list in terraform But in this scenario I was able to create a new data structure but when I create the resource (eg) aws_ecs_task_definition with a For_each it replicate some configuration like ecs tasks with the same name:

`

    locals {
   microservices_and_secrets = merge([
            for ecs_taks, group in var.microservices:
               {
                 for secrets_key, secret in group["secrets"]:                     
                       "${ecs_taks}-${secrets_key}" => {
                       name = group["name"]
                       port = group["port"]
                       secret = secret
                   }
               }
          
     ]...)
}

`

`

 

       resource "aws_ecs_task_definition" "task_definition" {
      for_each = local.microservices_and_secrets
      family = "${each.value.name}-${var.environment}" <-- ISSUE with creation because it replicates the ecs task microservice name due to foreach
      requires_compatibilities = ["FARGATE"]
      network_mode = "awsvpc"
      cpu = 1024
      memory= 2048

`

The problem is also that with this solution I can't have a microservice without any secret. eg the issue is the following:

`

 microservices = [
{ 
"name" = "api", 
"port" = 3000, 
"secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
{ 
"name" = "web", 
"port" = 3000 
"secrets" = [{ "key" = "test456", "arn" = "4567"}]
},
{ 
"name" = "ciaotask", 
"port" = 3000
}
]

`

`

Error: Iteration over null value
│
│   on main-aws-ecs.tf line 153, in locals:
│  152:                {
│  153:                  for secrets_key, secret in group["secrets"]:
│  154:                        "${ecs_taks}-${secrets_key}" => {
│  155:                        name = group["name"]
│  156:                        port = group["port"]
│  157:                        secret = secret
│  158:                    }
│  159:                }
│     ├────────────────
│     │ group["secrets"] is null
│
│ A null value cannot be used as the collection in a 'for' expression.

`

Anyone could help how can I manage the ecs task creation based on microservice input posted above? The question is, how can I create one aws_ecs_task_definition for each microservice present into microservices variable and it can have zero to n Secrets, starting from microservices variable list of objects.

Answer 1

I solved the issue. I started from this guide https://codeburst.io/how-to-securely-use-aws-secrets-manager-to-inject-secrets-into-ecs-using-infrastructure-as-code-ff2b39b420b6 then I created a template file like this:

`container_definitions.json.tpl

[{
    "name" : "${name}",
    "image": "${image}", 
    "cpu" : 1,
    "essential" : true,
    "Ulimits" : [{
        "Name" : "nofile",
        "SoftLimit" : 65535,
        "HardLimit" : 65535

    }],

    "Secrets" : ${secrets},
    "Environment" : ${environment},

    "LogConfiguration" : {
      "LogDriver" : "awslogs",
      "Options" : {
          "awslogs-group" : "${awslogs-group}",
          "awslogs-region" : "${aws_region}",
          "awslogs-stream-prefix" : "ecs"
      }
      },
    
    "portMappings" : [
      {
        "containerPort" : 3000,
        "hostPort"      : 3000
      }
    ]
  }]

`

in my main.tf instead I created the resources in this way: `

*/
data "template_file" "container_definitions" {
  count = length("${var.microservices}")
  template = file("${path.module}/template_dir/container_definitions.json.tpl")
   vars = {
     aws_region     = "${var.aws_region}"
     cpu            = 1
     image          = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
     name           = "${var.microservices[count.index].name}"
     awslogs-group  = "${aws_cloudwatch_log_group.cloudwatch_log_groups[count.index].id}"
     environment    = jsonencode("${var.microservices[count.index].environment}")
     secrets       = jsonencode("${var.microservices[count.index].secrets}")
   }
 }


/*
    AWS ECS Task definition
*/
resource "aws_ecs_task_definition" "task_definition" {
  count = length("${var.microservices}")
  family = "${var.microservices[count.index].name}-${var.environment}"
  requires_compatibilities = ["FARGATE"]
  network_mode = "awsvpc"
  cpu = "${var.microservices[count.index].cpu}"
  memory= "${var.microservices[count.index].memory}"
  execution_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}" 
  task_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}" 
  container_definitions = "${data.template_file.container_definitions[count.index].rendered}" //file("./containers_file/api.json")
}

`

In this way I was able to create a task definition in aws ecs with 0..n secrets and 0..n environment variables based on this (eg) input. `

  microservices = [
    { 
    "name" = "api", 
    "port" = 3000, 
    "cpu" = 1024,
    "memory" = 2048,
    "secrets" = [{ "name" = "test123", "valuefrom" = "0123"},{ "name" = "testXXX", "valuefrom" = "1010"}] },
    { 
    "name" = "web", 
    "port" = 3000,
    "cpu" = 1024,
    "memory" = 2048,
    "secrets" = [{ "name" = "test456", "valuefrom" = "4567"}],
    "environment" = [{ "name" = "weenv", "value" = "emi_is_ok" },{ "name" = "weenv123", "value" = "emi_is_ok123" } ]
    },
    { 
    "name" = "ciaotask", 
    "port" = 3000
    "cpu" = 1024,
    "memory" = 2048
    }
    ]

`

I hope this could help someone else that ran in the same issue.