Terraform issues when using for_each local variable created based on another local

Question

I am trying to create azure keyvault secrets using locals which reference data resources. I am iterating over an array containing my environments and creating a list of maps where each item is the set of secrets for a given environment. Using another local, I then proceed to merge these maps into a single one by creating two lists, one with keys and another with values and then zipping them.

I finally use for_each on the second local to create the resource.

If I run my root module without creating the actual secret resources ("azurerm_key_vault_secret) and a second time with it, it all works fine.

If I try to do it all in one go, as I want to implement on my CI/CD I get the error message:

|Error: Invalid for_each argument

|on variables.tf line 239, in resource “azurerm_key_vault_secret” “example”:

│239: for_each = nonsensitive(local.example_map)

│ local.example_map will be known only after apply

| The “for_each” value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.

If anybody has any idea how i could make work. It seems to me that this data transformation within locals doesn't quite work. Maybe I am going about the whole thing wrong. Any pointers would be highly appreciated.

Here is the code I am trying to make work:

variable "environment" {
    default = [ "dev", "prod"]
}

locals {
  example = distinct(flatten([
    for namespace in var.environment : {
        "${environment}-password1" = "${environment}-password",
        "${environment}-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["${environment}"].primary_connection_string}\"}",
        "${environment}-password3" = "{\"client_id\" : \"${jsondecode("${data.azurerm_key_vault_secret.other_credentials["${environment}"].value}").clients["example"].client_id}\"}",
        "${environment}-password4" = "{\"password\" : \"${data.azurerm_key_vault_secret.k_password.value}\"}",
        "${environment}-password5" = "{\"azurestorageaccountname\" : \"${data.azurerm_storage_account.example.name}\", \"azurestorageaccountkey\" : \"${data.azurerm_storage_account.example.primary_access_key}\"}",
        "${environment}-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}",
  }]))

  example_map = zipmap(
    flatten(
      [for item in local.example : keys(item)]
    ),
    flatten(
      [for item in local.example : values(item)]
    )
  )
}

resource "azurerm_key_vault_secret" "example" {
  for_each     = nonsensitive(local.example_map)
  name         = each.key
  value        = each.value
  key_vault_id = module.keyvault.id
  content_type = "password"
}

Here is the data structures created by local.example and local.example_map


"example": {
  "value": [
    {
      "dev-password1" = "dev-password",
      "dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "dev-password3" = "{\"client_id\" : \"myclientID\"}",
      "dev-password4" = "{\"password\" : \"password123\"}",
      "dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "dev-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
    },
    {
      "prod-password1" = "prod-password",
      "prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "prod-password3" = "{\"client_id\" : \"myclientID\"}",
      "prod-password4" = "{\"password\" : \"password123\"}",
      "prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
    }
  ]
}

"example_map": {
    "value": {
      "dev-password1" = "dev-password",
      "dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "dev-password3" = "{\"client_id\" : \"myclientID\"}",
      "dev-password4" = "{\"password\" : \"password123\"}",
      "dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "dev-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
      "prod-password1" = "prod-password",
      "prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "prod-password3" = "{\"client_id\" : \"myclientID\"}",
      "prod-password4" = "{\"password\" : \"password123\"}",
      "prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
    },
    "type": [
    "object",
    {
        "dev-password1": "string",
        "dev-password2": "string",
        "dev-password3": "string",
        "dev-password4": "string",
        "dev-password5": "string",
        "dev-password6": "string",
        "prod-password1": "string",
        "prod-password2": "string",
        "prod-password3": "string",
        "prod-password4": "string",
        "prod-password5": "string",
        "prod-password6": "string",
    }
    ]
}

Also what confuses me the most is that if I work with the following data structure, which is hard coding instead of doing the first transformation based on namespaces. The entry getting information from another module doesn't cause any problems and it all works wonderfully.

locals {
  hardcoding_namespaces = {
    "dev-password1" = "dev-password"
    "dev-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["dev"].primary_connection_string}\"}"
    "dev-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
    "prod-password1" = "prod-password"
    "prod-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["prod"].primary_connection_string}\"}"
    "prod-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"

  }
}

resource "azurerm_key_vault_secret" "another_example" {
  for_each     = local.hardcoding_namespaces
  name         = each.key
  value        = each.value
  key_vault_id = module.keyvault.id
  content_type = "password"

} 

if the resulting data structure is the same, why for_each works for one and not for the other? [1]: https://i.stack.imgur.com/cTq5f.png

Answer 1

from the doc

Sensitive values, such as sensitive input variables, sensitive outputs, or sensitive resource attributes, cannot be used as arguments to for_each. The value used in for_each is used to identify the resource instance and will always be disclosed in UI output, which is why sensitive values are not allowed. Attempts to use sensitive values as for_each arguments will result in an error.(visite https://www.terraform.io/language/meta-arguments/for_each#limitations-on-values-used-in-for_each )

Keys () will always return a sensitive value if the input is sensitive, so instead try the following:

  example_map = zipmap(
flatten(
  [for item,value in local.example : item]
),
flatten(
  [for item, value in local.example : value]
)

)

Answer 2

This is probably because of module.some_module.connection_string . You can't use dynamic values in for_each . As the error message says, you have to use target to first create those dynamic resources, and then your for_each will work.