Terraform For expression to create a single map to use in for_each, using one tuple and two list of strings

Question

How to create a single map of object to use in for_each, using one tuple and two list of strings

Let's say I have a tuple/list of instance IDs or Instance IPs coming from an ec2 module and a tuple/list of target group arns from ALB module as shown below.

instance_ids      = ["10.1.1.1", "10.2.2.2"]
target_gropup_arn = ["arn1", "arn2"]

In main.tf

output "out" {
  value = local.aws_lb_target_group_attachment
}
variable "target_groups" {
  type = any
  default = [
    {
      protocol = "TCP"
      port     = "22"
    },
    {
    protocol = "TCP"
    port     = "443"
    }
  ]
}
locals {
  instance_ips      = toset(["10.1.1.1", "10.2.2.2"])
  target_gropup_arn = toset(["arn1", "arn2"])

  aws_lb_target_group_attachment = { for idx in flatten([for instance_ids in local.instance_ids :
  [for tg in var.target_groups :
  [for arn in local.target_gropup_arn :
  {
    target_id = instance_ids
    target_group_arn = arn
    port = tg.port
  }]
  ]]) : "${idx.target_id}:${idx.port}:${idx.target_group_arn}" => idx }
}

When I run terraform apply I get the below output, but this is not the expected map I need.

Outputs:

out = {
  "10.1.1.1:22:arn1" = {
    "port" = "22"
    "target_group_arn" = "arn1"
    "target_id" = "10.1.1.1"
  }
  "10.1.1.1:22:arn2" = {
    "port" = "22"
    "target_group_arn" = "arn2"
    "target_id" = "10.1.1.1"
  }
  "10.1.1.1:443:arn1" = {
    "port" = "443"
    "target_group_arn" = "arn1"
    "target_id" = "10.1.1.1"
  }
  "10.1.1.1:443:arn2" = {
    "port" = "443"
    "target_group_arn" = "arn2"
    "target_id" = "10.1.1.1"
  }
  "10.2.2.2:22:arn1" = {
    "port" = "22"
    "target_group_arn" = "arn1"
    "target_id" = "10.2.2.2"
  }
  "10.2.2.2:22:arn2" = {
    "port" = "22"
    "target_group_arn" = "arn2"
    "target_id" = "10.2.2.2"
  }
  "10.2.2.2:443:arn1" = {
    "port" = "443"
    "target_group_arn" = "arn1"
    "target_id" = "10.2.2.2"
  }
  "10.2.2.2:443:arn2" = {
    "port" = "443"
    "target_group_arn" = "arn2"
    "target_id" = "10.2.2.2"
  }
}

The expected output is (pseudo output)

out = {
  "10.1.1.1:22" = {
    "port" = "22"
    "target_id" = "10.1.1.1"
    "target_group_arn" = "arn1"
  }
  "10.1.1.1:443" = {
    "port" = "443"
    "target_id" = "10.1.1.1"
    "target_group_arn" = "arn1"
  }
  "10.2.2.2:22" = {
    "port" = "22"
    "target_id" = "10.2.2.2"
    "target_group_arn" = "arn2"
  }
  "10.2.2.2:443" = {
    "port" = "443"
    "target_id" = "10.2.2.2"
    "target_group_arn" = "arn2"
  }
}

This is required so that later I can use it to attach multiple instances using "aws_lb_target_group_attachment" resource.

Answer 1

Your technique isn't working as you can tell, because you are creating a cartesian product of ips and arns, but according to your example output, you only want the first ip to be grouped with the first arn, and the second ip to be grouped with the second arn.

Here is a working solution:

output "out" {
  value = local.aws_lb_target_group_attachment
}

variable "target_groups" {
  type = any
  default = [
    {
      protocol = "TCP"
      port     = "22"
    },
    {
      protocol = "TCP"
      port     = "443"
    }
  ]
}

locals {
  instance_ids = ["10.1.1.1", "10.2.2.2"]
  target_arns  = ["arn1", "arn2"]

  arn_id_map = [for i, id in local.instance_ids : { "target_id" : id, "target_group_arn" : local.target_arns[i] }]
  aws_lb_target_group_attachment = merge([for tg in var.target_groups :
    { for pair in local.arn_id_map :
      "${pair.target_id}:${tg.port}" => merge(pair, { "port" : tg.port })
    }
  ]...)
}

Breakdown:

  arn_id_map = [for i, id in local.instance_ids : { "target_id" : id, "target_group_arn" : local.target_arns[i] }]

Here we loop through the list of ips and construct a new map containing two of the three keys you want in your final output. We take advantage of the fact that there is a 1-1 mapping between the ips and arns. That is, the first ip corresponds to the first arn, the second ip the second arn, and so on.

  aws_lb_target_group_attachment = merge([for tg in var.target_groups :
    { for pair in local.arn_id_map :
      "${pair.target_id}:${tg.port}" => merge(pair, { "port" : tg.port })
    }
  ]...)

Starting with the first loop, we iterate of the target groups creating a list comprehension, this means the value at this layer will be a list. Then inside we do a map comprehension, iterating over the ip+arn pairs we created in the previous step.

On the inside we create a map whose structure is your desired final output. The inner merge is a clean way to take our ip+arn pair and create a new map with the port.

But as the layers unwind we end up with a list containing two maps, each of which contains 2 submaps (one for each ip+port combination). We need to "flatten" the structure, but flatten() doesn't work on lists of maps.

To "flatten" maps we usually reach to merge() , but in this case it doesn't work directly, because merge() expects each argument to be a map. But we have a list of maps. So we use the expansion symbol ... to pass the list into merge and expand it into arguments that merge can accept directly.

The result is as you desired:

out = {
  "10.1.1.1:22" = {
    "port" = "22"
    "target_group_arn" = "arn1"
    "target_id" = "10.1.1.1"
  }
  "10.1.1.1:443" = {
    "port" = "443"
    "target_group_arn" = "arn1"
    "target_id" = "10.1.1.1"
  }
  "10.2.2.2:22" = {
    "port" = "22"
    "target_group_arn" = "arn2"
    "target_id" = "10.2.2.2"
  }
  "10.2.2.2:443" = {
    "port" = "443"
    "target_group_arn" = "arn2"
    "target_id" = "10.2.2.2"
  }
}