导入 aws 资源后 Terraform 计划显示差异

Question

This is a follow-up question to my original question: Terraform (0.12.29) import not working as expected; import succeeded but plan shows destroy & recreate

Some Background: We have terraform code to create various AWS resources. Some of these resources are created per AWS account and hence are structured to be stored in a account-scope folder in our project. This was when we were only having one AWS region. Now our application is made multi-region and hence these resources are to be created per region for each AWS account.

In order to do that we have now moved these TF scripts to region-scope folder which will be run per region. Since these resources are no longer part of 'account scope' we have removed them from the account scope Terraform state. Now when I try to import these resources to region scope

My Imports (running from xyz-region-scope directory) and terraform plan:

terraform import module.buckets.random_id.cloudtrail_bucket_suffix cqLFzQ
terraform import module.buckets.aws_s3_bucket.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.module.access_logging_bucket.aws_s3_bucket.default "ab-xyz-stage-access-logging-9d8e94ff"
terraform import module.buckets.module.access_logging_bucket.random_id.bucket_suffix  nY6U_w
terraform import module.encryption.module.data_key.aws_iam_policy.decrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt"
terraform import module.encryption.module.data_key.aws_iam_policy.encrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt"


mycompanymachine:xyz-region-scope kuldeepjain$ ../scripts/terraform.sh xyz-stage plan -no-color
+ set -o posix
+ IFS='
    '
++ blhome
+ BASH_LIB_HOME=/usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT
+ source /usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT/s3/bucket.sh
+ main xyz-stage plan -no-color
+ '[' 3 -lt 2 ']'
+ local env=xyz-stage
+ shift
+ local command=plan
+ shift
++ get_region xyz-stage
++ local env=xyz-stage
++ shift
+++ aws --profile xyz-stage configure get region
++ local region=us-west-2
++ '[' -z us-west-2 ']'
++ echo us-west-2
+ local region=us-west-2
++ _get_bucket xyz-stage xyz-stage-tfstate
++ local env=xyz-stage
++ shift
++ local name=xyz-stage-tfstate
++ shift
+++ _get_bucket_list xyz-stage xyz-stage-tfstate
+++ local env=xyz-stage
+++ shift
+++ local name=xyz-stage-tfstate
+++ shift
+++ aws --profile xyz-stage --output json s3api list-buckets --query 'Buckets[?contains(Name, `xyz-stage-tfstate`) == `true`].Name'
++ local 'bucket_list=[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ _count_buckets_in_json '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ local 'json=[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ shift
+++ echo '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq '. | length'
++ local number_of_buckets=1
++ '[' 1 == 0 ']'
++ '[' 1 -gt 1 ']'
+++ echo '[
    "ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq -r '.[0]'
++ local bucket_name=ab-xyz-stage-tfstate-5b8873b8
++ echo ab-xyz-stage-tfstate-5b8873b8
+ local tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8
++ get_config_file xyz-stage us-west-2
++ local env=xyz-stage
++ shift
++ local region=us-west-2
++ shift
++ local config_file=config/us-west-2/xyz-stage.tfvars
++ '[' '!' -f config/us-west-2/xyz-stage.tfvars ']'
++ config_file=config/us-west-2/default.tfvars
++ echo config/us-west-2/default.tfvars
+ local config_file=config/us-west-2/default.tfvars
+ export TF_DATA_DIR=state/xyz-stage/
+ TF_DATA_DIR=state/xyz-stage/
+ terraform get
+ terraform plan -var-file=config/us-west-2/default.tfvars -var-file=variables.tfvars -var-file=../globals.tfvars -var profile=xyz-stage -var region=us-west-2 -var tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8 -no-color
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

module.encryption.module.data_key.data.null_data_source.key: Refreshing state...
module.buckets.module.access_logging_bucket.data.template_file.dependencies: Refreshing state...
module.buckets.data.template_file.dependencies: Refreshing state...
data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_kms_alias.default: Refreshing state...
module.buckets.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_region.current: Refreshing state...
module.encryption.module.data_key.data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_kms_alias.encryption_key_alias: Refreshing state...
module.buckets.module.access_logging_bucket.random_id.bucket_suffix: Refreshing state... [id=nY6U_w]
module.buckets.module.access_logging_bucket.aws_s3_bucket.default: Refreshing state... [id=ab-xyz-stage-access-logging-9d8e94ff]
module.buckets.random_id.cloudtrail_bucket_suffix: Refreshing state... [id=cqLFzQ]
module.buckets.module.access_logging_bucket.data.template_file.encryption_configuration: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.encrypt: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.decrypt: Refreshing state...
module.encryption.module.data_key.aws_iam_policy.decrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt]
module.encryption.module.data_key.aws_iam_policy.encrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt]
module.buckets.aws_s3_bucket.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]
module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail: Refreshing state...
module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
 <= read (data resources)

Terraform will perform the following actions:


  # module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "restrict_access_cloudtrail"  {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:GetBucketAcl",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd",
            ]
          + sid       = "AWSCloudTrailAclCheck"

          + principals {
              + identifiers = [
                  + "cloudtrail.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
      + statement {
          + actions   = [
              + "s3:PutObject",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*",
            ]
          + sid       = "AWSCloudTrailWrite"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "bucket-owner-full-control",
                ]
              + variable = "s3:x-amz-acl"
            }

          + principals {
              + identifiers = [
                  + "cloudtrail.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
    }

  # module.buckets.aws_s3_bucket.cloudtrail_logging_bucket will be updated in-place
  ~ resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
      + acl                         = "private"
        arn                         = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        bucket                      = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        bucket_domain_name          = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.amazonaws.com"
        bucket_regional_domain_name = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.us-west-2.amazonaws.com"
      + force_destroy               = false
        hosted_zone_id              = "Z3BJ6K6RIION7M"
        id                          = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        region                      = "us-west-2"
        request_payer               = "BucketOwner"
        tags                        = {
            "mycompany:finance:accountenvironment"   = "xyz-stage"
            "mycompany:finance:application"          = "ab-platform"
            "mycompany:finance:billablebusinessunit" = "my-dev"
            "name"                                = "Cloudtrail logging bucket"
        }

        lifecycle_rule {
            abort_incomplete_multipart_upload_days = 0
            enabled                                = true
            id                                     = "intu-lifecycle-s3-int-tier"
            tags                                   = {}

            transition {
                days          = 32
                storage_class = "INTELLIGENT_TIERING"
            }
        }

        logging {
            target_bucket = "ab-xyz-stage-access-logging-9d8e94ff"
            target_prefix = "logs/cloudtrail-logging/"
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

  # module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket will be updated in-place
  ~ resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
        bucket = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
        id     = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
      ~ policy = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "s3:GetBucketAcl"
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "cloudtrail.amazonaws.com"
                        }
                      - Resource  = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
                      - Sid       = "AWSCloudTrailAclCheck"
                    },
                  - {
                      - Action    = "s3:PutObject"
                      - Condition = {
                          - StringEquals = {
                              - s3:x-amz-acl = "bucket-owner-full-control"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "cloudtrail.amazonaws.com"
                        }
                      - Resource  = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*"
                      - Sid       = "AWSCloudTrailWrite"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
    }

  # module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place
  ~ resource "aws_s3_bucket" "default" {
      + acl                         = "log-delivery-write"
        arn                         = "arn:aws:s3:::ab-xyz-stage-access-logging-9d8e94ff"
        bucket                      = "ab-xyz-stage-access-logging-9d8e94ff"
        bucket_domain_name          = "ab-xyz-stage-access-logging-9d8e94ff.s3.amazonaws.com"
        bucket_regional_domain_name = "ab-xyz-stage-access-logging-9d8e94ff.s3.us-west-2.amazonaws.com"
      + force_destroy               = false
        hosted_zone_id              = "Z3BJ6K6RIION7M"
        id                          = "ab-xyz-stage-access-logging-9d8e94ff"
        region                      = "us-west-2"
        request_payer               = "BucketOwner"
        tags                        = {
            "mycompany:finance:accountenvironment"   = "xyz-stage"
            "mycompany:finance:application"          = "ab-platform"
            "mycompany:finance:billablebusinessunit" = "my-dev"
            "name"                                = "Access logging bucket"
        }

      - grant {
          - permissions = [
              - "READ_ACP",
              - "WRITE",
            ] -> null
          - type        = "Group" -> null
          - uri         = "http://acs.amazonaws.com/groups/s3/LogDelivery" -> null
        }
      - grant {
          - id          = "0343271a8c2f184152c171b223945b22ceaf5be5c9b78cf167660600747b5ad8" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }

      - lifecycle_rule {
          - abort_incomplete_multipart_upload_days = 0 -> null
          - enabled                                = true -> null
          - id                                     = "intu-lifecycle-s3-int-tier" -> null
          - tags                                   = {} -> null

          - transition {
              - days          = 32 -> null
              - storage_class = "INTELLIGENT_TIERING" -> null
            }
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

Plan: 0 to add, 3 to change, 0 to destroy.

As you can see terraform plan output shows Plan: 0 to add, 3 to change, 0 to destroy.

My Questions are:

1. Why does it try to remove new aws_s3_bucket_policy for cloudtrail_logging_bucket even when there is no change in the policy. Pls see the screenshot and TF-code cloudtrail_bucket.tf below. Snippet of Diff of old account-scope ( LEFT ) vs my current remote TF state ( RIGHT ) for cloudtrail_bucket_suffix:
2. For the resource where it says module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply . It shows + sign, does that mean it will modify something here or will just read it as it says.
3. Why does it show this module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place ~ resource "aws_s3_bucket" "default" where it removes the grant and lifecycle_rule . See below s3_bucket.tf

TF Code:

cloudtrail_bucket.tf:

data "aws_caller_identity" "current" {}

resource "random_id" "cloudtrail_bucket_suffix" {
  keepers = {
    # Keep the suffix per account id / environment
    aws_account_id = "${data.aws_caller_identity.current.account_id}"
    env = "${var.environment}"
  }

  byte_length = "4"
}


resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
  bucket = "ab-${var.environment}-cloudtrail-logging-${random_id.cloudtrail_bucket_suffix.hex}"
  acl = "private"

  depends_on = [data.template_file.dependencies]

  tags = {
    name = "Cloudtrail logging bucket"
    "mycompany:finance:accountenvironment" = "${var.environment}"
    "mycompany:finance:application" = "${module.constants.finance_application}"
    "mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
  }

  lifecycle {
    ignore_changes = [ "server_side_encryption_configuration" ]
  }

  logging {
    target_bucket =  "${module.access_logging_bucket.name}"
    target_prefix = "logs/cloudtrail-logging/"
  }

  lifecycle_rule {
    enabled = "true"

    transition {
      days          = 32
      storage_class = "INTELLIGENT_TIERING"
    }
  }

}

resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
  bucket = "${aws_s3_bucket.cloudtrail_logging_bucket.id}"
  policy = "${data.aws_iam_policy_document.restrict_access_cloudtrail.json}"
}


data aws_iam_policy_document "restrict_access_cloudtrail" {

  statement {
    sid = "AWSCloudTrailAclCheck"
    effect = "Allow"
    actions = ["s3:GetBucketAcl"]
    resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}" ]
    principals {
      identifiers = ["cloudtrail.amazonaws.com"]
      type = "Service"
    }

  }

  statement {
    sid = "AWSCloudTrailWrite"
    effect = "Allow"
    actions = ["s3:PutObject"]
    resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}/*" ]
    principals {
      identifiers = ["cloudtrail.amazonaws.com"]
      type = "Service"
    }
    condition {
      test = "StringEquals"
      values = ["bucket-owner-full-control"]
      variable = "s3:x-amz-acl"
    }

  }
}

s3_bucket.tf

resource "random_id" "bucket_suffix" {
    keepers = {
        # Keep the suffix per account id / environment
        aws_account_id = "${data.aws_caller_identity.current.account_id}"
        env = "${var.environment}"
    }

    byte_length = "${var.byte_length}"
}

resource "aws_s3_bucket" "default" {
    bucket = "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}"
    acl = "${var.acl}"

    depends_on = [data.template_file.dependencies]

    tags = {
        name = "${var.name_tag}"
        "mycompany:finance:accountenvironment" = "${var.environment}"
        "mycompany:finance:application" = "${module.constants.finance_application}"
        "mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
    }

    lifecycle {
        ignore_changes = [ "server_side_encryption_configuration" ]
    }

    logging {
        target_bucket = "${lookup(var.logging, "target_bucket", "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}")}"
        target_prefix = "logs/${lookup(var.logging, "target_folder_name", "access-logging")}/"
    }
}

My Environment:

Local machine: macOS v10.14.6

Terraform v0.12.29
+ provider.aws v3.14.1
+ provider.null v2.1.2
+ provider.random v2.3.1
+ provider.template v2.1.2

Answer 1

Such a difference may be shown, if the terraform-code differs from the imported existing resource. eg if someone changed the resource by click-and-point in the AWS management console, without editing/applying the code. terraform import only imports the resource to the tfstate, but does not create terraform code.

In this sample, you could validate in the AWS console/cli if the S3 bucket "default" actually has logging configured, or not. Acording to the plan, the existing bucket is not configured for logging in AWS, but your TF-code contains this, so it would be changed.

Are you sure, your TF-code exactly matches all the attributes of the imported resource?

For further investigation, it would be necessary, that you post the corresponding tf-code as well.

Answer 2

Just answering my own question so that I could mentioned what I did for each question:

For the 2nd question it does not modify anything with terraform apply and simply reads it as it says in the message.

For my 3rd question asked it in a separate SO thread here: Terraform - Updating S3 Access Control: Question on replacing acl with grant and went ahead with the solution I mentioned in the answer there.

For the 1st question it's still not clear why it was showing the diff. I tried to compare the existing state with terraform state pull and check, why the update is done but it didn't help. But running terraform apply for that went fine and it did not make any change in the policy which is what I expect it to do.