Terraform Cloud apply lambda function fails with ValidationException, AWS CLI lambda create-function with same parameters succeeds

Question

When trying to run a Terraform apply in Terraform Cloud which attempts to create an AWS Lambda function resource, the apply fails with a nondescript ValidationException. No other error is returned. There is an issue in terraform-provider-aws addressing this problem.

This is the Terraform code describing the function:

module "lambda" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 4.7"

  function_name = "${module.this.s3_bucket_id}-to-cloudwatch"

  handler = "index.handler"
  runtime = "nodejs12.x"
  timeout = 60

  create_package         = false
  local_existing_package = "${path.module}/assets/code.zip"

  environment_variables = {
    LOG_GROUP_NAME     = aws_cloudwatch_log_group.log_group.name
    LOAD_BALANCER_TYPE = var.load_balancer_type
  }

  allowed_triggers = {
    S3EventPermission = {
      principal  = "s3.amazonaws.com"
      source_arn = module.this.s3_bucket_arn
    }
  }

  role_path   = "/tf-managed/"
  policy_path = "/tf-managed/"

  attach_cloudwatch_logs_policy = true
  attach_tracing_policy         = true
  tracing_mode                  = "active"

  attach_policy_statements = true
  policy_statements = {
    describe_log_groups = {
      effect    = "Allow"
      actions   = ["logs:DescribeLogGroups"]
      resources = ["*"]
    }

    create_logs = {
      effect = "Allow"
      actions = [
        "logs:DescribeLogStreams",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
      ]
      resources = [aws_cloudwatch_log_group.log_group.arn]
    }

    get_logs = {
      effect    = "Allow"
      actions   = ["s3:GetObject"]
      resources = ["${module.this.s3_bucket_arn}/*"]
    }
  }
}

This is the output of terraform plan for the function:

  # module.cluster_nlb.module.log_bucket.module.lambda.aws_lambda_function.this[0] will be created
  + resource "aws_lambda_function" "this" {
      + architectures                  = (known after apply)
      + arn                            = (known after apply)
      + filename                       = "../../../modules/lb-log-bucket-with-cloudwatch-export/assets/code.zip"
      + function_name                  = "nlb-access-logs-04916534-to-cloudwatch"
      + handler                        = "index.handler"
      + id                             = (known after apply)
      + invoke_arn                     = (known after apply)
      + last_modified                  = (known after apply)
      + memory_size                    = 128
      + package_type                   = "Zip"
      + publish                        = false
      + qualified_arn                  = (known after apply)
      + reserved_concurrent_executions = -1
      + role                           = "arn:aws:iam::585685634436:role/tf-managed/nlb-access-logs-04916534-to-cloudwatch"
      + runtime                        = "nodejs12.x"
      + signing_job_arn                = (known after apply)
      + signing_profile_version_arn    = (known after apply)
      + source_code_hash               = "/pwL7Szm/wc/8dP8/Relzc8vy7nkAUQm9jtvgfWJa5c="
      + source_code_size               = (known after apply)
      + tags_all                       = (known after apply)
      + timeout                        = 60
      + version                        = (known after apply)

      + environment {
          + variables = {
              + "LOAD_BALANCER_TYPE" = "network"
              + "LOG_GROUP_NAME"     = "/aws/elb/network"
            }
        }

      + ephemeral_storage {
          + size = 512
        }

      + tracing_config {
          + mode = "active"
        }
    }

The error as displayed in Terraform Cloud:

Error: error creating Lambda Function (1): ValidationException: status code: 400, request id: [...]
with module.cluster_alb.module.log_bucket.module.lambda.aws_lambda_function.this[0]
on .terraform/modules/cluster_alb.log_bucket.lambda/main.tf line 24, in resource "aws_lambda_function" "this":

resource "aws_lambda_function" "this" {

I've been trying to get a more detailed error by replicating the planned apply in an AWS CLI lambda create-function command. The command completes and successfully creates the Lambda function however.

This is the AWS CLI command:

aws lambda create-function \
  --zip-file fileb://../../../modules/lb-log-bucket-with-cloudwatch-export/assets/code.zip \
  --function-name 'nlb-access-logs-04916534-to-cloudwatch' \
  --handler 'index.handler' \
  --memory-size '128' \
  --package-type 'Zip' \
  --no-publish \
  --role 'arn:aws:iam::585685634436:role/tf-managed/nlb-access-logs-04916534-to-cloudwatch' \
  --runtime 'nodejs12.x' \
  --timeout '60' \
  --environment 'Variables={LOG_GROUP_NAME=/aws/elb/network,LOAD_BALANCER_TYPE=network}' \
  --tracing-config 'Mode=Active' \
  --description '' \
  --debug

I have not been able to identify any discrepancies between the AWS CLI command, or why the validation would fail in Terraform.

Answer 1

I had set tracing_mode = "active" in the Terraform configuration, but passed --tracing-config 'Mode=Active' to the AWS CLI.

Valid values for tracing_mode are "PassThrough" and "Active". Note that the word "Active" must be capitalized.