This is a follow-up question to my original question: Terraform (0.12.29) import not working as expected; import succeeded but plan shows destroy & recreate
Some Background: We have terraform code to create various AWS resources. Some of these resources are created per AWS account and hence are structured to be stored in a account-scope folder in our project. This was when we were only having one AWS region. Now our application is made multi-region and hence these resources are to be created per region for each AWS account.
In order to do that we have now moved these TF scripts to region-scope folder which will be run per region. Since these resources are no longer part of 'account scope' we have removed them from the account scope Terraform state. Now when I try to import these resources to region scope
My Imports (running from xyz-region-scope directory) and terraform plan:
terraform import module.buckets.random_id.cloudtrail_bucket_suffix cqLFzQ
terraform import module.buckets.aws_s3_bucket.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
terraform import module.buckets.module.access_logging_bucket.aws_s3_bucket.default "ab-xyz-stage-access-logging-9d8e94ff"
terraform import module.buckets.module.access_logging_bucket.random_id.bucket_suffix nY6U_w
terraform import module.encryption.module.data_key.aws_iam_policy.decrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt"
terraform import module.encryption.module.data_key.aws_iam_policy.encrypt "arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt"
mycompanymachine:xyz-region-scope kuldeepjain$ ../scripts/terraform.sh xyz-stage plan -no-color
+ set -o posix
+ IFS='
'
++ blhome
+ BASH_LIB_HOME=/usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT
+ source /usr/local/lib/mycompany/ab/bash_library/0.0.1-SNAPSHOT/s3/bucket.sh
+ main xyz-stage plan -no-color
+ '[' 3 -lt 2 ']'
+ local env=xyz-stage
+ shift
+ local command=plan
+ shift
++ get_region xyz-stage
++ local env=xyz-stage
++ shift
+++ aws --profile xyz-stage configure get region
++ local region=us-west-2
++ '[' -z us-west-2 ']'
++ echo us-west-2
+ local region=us-west-2
++ _get_bucket xyz-stage xyz-stage-tfstate
++ local env=xyz-stage
++ shift
++ local name=xyz-stage-tfstate
++ shift
+++ _get_bucket_list xyz-stage xyz-stage-tfstate
+++ local env=xyz-stage
+++ shift
+++ local name=xyz-stage-tfstate
+++ shift
+++ aws --profile xyz-stage --output json s3api list-buckets --query 'Buckets[?contains(Name, `xyz-stage-tfstate`) == `true`].Name'
++ local 'bucket_list=[
"ab-xyz-stage-tfstate-5b8873b8"
]'
+++ _count_buckets_in_json '[
"ab-xyz-stage-tfstate-5b8873b8"
]'
+++ local 'json=[
"ab-xyz-stage-tfstate-5b8873b8"
]'
+++ shift
+++ echo '[
"ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq '. | length'
++ local number_of_buckets=1
++ '[' 1 == 0 ']'
++ '[' 1 -gt 1 ']'
+++ echo '[
"ab-xyz-stage-tfstate-5b8873b8"
]'
+++ jq -r '.[0]'
++ local bucket_name=ab-xyz-stage-tfstate-5b8873b8
++ echo ab-xyz-stage-tfstate-5b8873b8
+ local tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8
++ get_config_file xyz-stage us-west-2
++ local env=xyz-stage
++ shift
++ local region=us-west-2
++ shift
++ local config_file=config/us-west-2/xyz-stage.tfvars
++ '[' '!' -f config/us-west-2/xyz-stage.tfvars ']'
++ config_file=config/us-west-2/default.tfvars
++ echo config/us-west-2/default.tfvars
+ local config_file=config/us-west-2/default.tfvars
+ export TF_DATA_DIR=state/xyz-stage/
+ TF_DATA_DIR=state/xyz-stage/
+ terraform get
+ terraform plan -var-file=config/us-west-2/default.tfvars -var-file=variables.tfvars -var-file=../globals.tfvars -var profile=xyz-stage -var region=us-west-2 -var tfstate_bucket=ab-xyz-stage-tfstate-5b8873b8 -no-color
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
module.encryption.module.data_key.data.null_data_source.key: Refreshing state...
module.buckets.module.access_logging_bucket.data.template_file.dependencies: Refreshing state...
module.buckets.data.template_file.dependencies: Refreshing state...
data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_kms_alias.default: Refreshing state...
module.buckets.data.aws_caller_identity.current: Refreshing state...
module.encryption.module.data_key.data.aws_region.current: Refreshing state...
module.encryption.module.data_key.data.aws_caller_identity.current: Refreshing state...
module.buckets.module.access_logging_bucket.data.aws_kms_alias.encryption_key_alias: Refreshing state...
module.buckets.module.access_logging_bucket.random_id.bucket_suffix: Refreshing state... [id=nY6U_w]
module.buckets.module.access_logging_bucket.aws_s3_bucket.default: Refreshing state... [id=ab-xyz-stage-access-logging-9d8e94ff]
module.buckets.random_id.cloudtrail_bucket_suffix: Refreshing state... [id=cqLFzQ]
module.buckets.module.access_logging_bucket.data.template_file.encryption_configuration: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.encrypt: Refreshing state...
module.encryption.module.data_key.data.aws_iam_policy_document.decrypt: Refreshing state...
module.encryption.module.data_key.aws_iam_policy.decrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_decrypt]
module.encryption.module.data_key.aws_iam_policy.encrypt: Refreshing state... [id=arn:aws:iam::123412341234:policy/ab_data_key_xyz_stage_encrypt]
module.buckets.aws_s3_bucket.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]
module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail: Refreshing state...
module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket: Refreshing state... [id=ab-xyz-stage-cloudtrail-logging-72a2c5cd]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
~ update in-place
<= read (data resources)
Terraform will perform the following actions:
# module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply
# (config refers to values not yet known)
<= data "aws_iam_policy_document" "restrict_access_cloudtrail" {
+ id = (known after apply)
+ json = (known after apply)
+ statement {
+ actions = [
+ "s3:GetBucketAcl",
]
+ effect = "Allow"
+ resources = [
+ "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd",
]
+ sid = "AWSCloudTrailAclCheck"
+ principals {
+ identifiers = [
+ "cloudtrail.amazonaws.com",
]
+ type = "Service"
}
}
+ statement {
+ actions = [
+ "s3:PutObject",
]
+ effect = "Allow"
+ resources = [
+ "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*",
]
+ sid = "AWSCloudTrailWrite"
+ condition {
+ test = "StringEquals"
+ values = [
+ "bucket-owner-full-control",
]
+ variable = "s3:x-amz-acl"
}
+ principals {
+ identifiers = [
+ "cloudtrail.amazonaws.com",
]
+ type = "Service"
}
}
}
# module.buckets.aws_s3_bucket.cloudtrail_logging_bucket will be updated in-place
~ resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
+ acl = "private"
arn = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
bucket = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
bucket_domain_name = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.amazonaws.com"
bucket_regional_domain_name = "ab-xyz-stage-cloudtrail-logging-72a2c5cd.s3.us-west-2.amazonaws.com"
+ force_destroy = false
hosted_zone_id = "Z3BJ6K6RIION7M"
id = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
region = "us-west-2"
request_payer = "BucketOwner"
tags = {
"mycompany:finance:accountenvironment" = "xyz-stage"
"mycompany:finance:application" = "ab-platform"
"mycompany:finance:billablebusinessunit" = "my-dev"
"name" = "Cloudtrail logging bucket"
}
lifecycle_rule {
abort_incomplete_multipart_upload_days = 0
enabled = true
id = "intu-lifecycle-s3-int-tier"
tags = {}
transition {
days = 32
storage_class = "INTELLIGENT_TIERING"
}
}
logging {
target_bucket = "ab-xyz-stage-access-logging-9d8e94ff"
target_prefix = "logs/cloudtrail-logging/"
}
versioning {
enabled = false
mfa_delete = false
}
}
# module.buckets.aws_s3_bucket_policy.cloudtrail_logging_bucket will be updated in-place
~ resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
bucket = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
id = "ab-xyz-stage-cloudtrail-logging-72a2c5cd"
~ policy = jsonencode(
{
- Statement = [
- {
- Action = "s3:GetBucketAcl"
- Effect = "Allow"
- Principal = {
- Service = "cloudtrail.amazonaws.com"
}
- Resource = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd"
- Sid = "AWSCloudTrailAclCheck"
},
- {
- Action = "s3:PutObject"
- Condition = {
- StringEquals = {
- s3:x-amz-acl = "bucket-owner-full-control"
}
}
- Effect = "Allow"
- Principal = {
- Service = "cloudtrail.amazonaws.com"
}
- Resource = "arn:aws:s3:::ab-xyz-stage-cloudtrail-logging-72a2c5cd/*"
- Sid = "AWSCloudTrailWrite"
},
]
- Version = "2012-10-17"
}
) -> (known after apply)
}
# module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place
~ resource "aws_s3_bucket" "default" {
+ acl = "log-delivery-write"
arn = "arn:aws:s3:::ab-xyz-stage-access-logging-9d8e94ff"
bucket = "ab-xyz-stage-access-logging-9d8e94ff"
bucket_domain_name = "ab-xyz-stage-access-logging-9d8e94ff.s3.amazonaws.com"
bucket_regional_domain_name = "ab-xyz-stage-access-logging-9d8e94ff.s3.us-west-2.amazonaws.com"
+ force_destroy = false
hosted_zone_id = "Z3BJ6K6RIION7M"
id = "ab-xyz-stage-access-logging-9d8e94ff"
region = "us-west-2"
request_payer = "BucketOwner"
tags = {
"mycompany:finance:accountenvironment" = "xyz-stage"
"mycompany:finance:application" = "ab-platform"
"mycompany:finance:billablebusinessunit" = "my-dev"
"name" = "Access logging bucket"
}
- grant {
- permissions = [
- "READ_ACP",
- "WRITE",
] -> null
- type = "Group" -> null
- uri = "http://acs.amazonaws.com/groups/s3/LogDelivery" -> null
}
- grant {
- id = "0343271a8c2f184152c171b223945b22ceaf5be5c9b78cf167660600747b5ad8" -> null
- permissions = [
- "FULL_CONTROL",
] -> null
- type = "CanonicalUser" -> null
}
- lifecycle_rule {
- abort_incomplete_multipart_upload_days = 0 -> null
- enabled = true -> null
- id = "intu-lifecycle-s3-int-tier" -> null
- tags = {} -> null
- transition {
- days = 32 -> null
- storage_class = "INTELLIGENT_TIERING" -> null
}
}
versioning {
enabled = false
mfa_delete = false
}
}
Plan: 0 to add, 3 to change, 0 to destroy.
As you can see terraform plan output shows Plan: 0 to add, 3 to change, 0 to destroy.
My Questions are:
aws_s3_bucket_policy
for cloudtrail_logging_bucket
even when there is no change in the policy. Pls see the screenshot and TF-code cloudtrail_bucket.tf below. Snippet of Diff of old account-scope ( LEFT ) vs my current remote TF state ( RIGHT ) for cloudtrail_bucket_suffix: module.buckets.data.aws_iam_policy_document.restrict_access_cloudtrail will be read during apply
. It shows +
sign, does that mean it will modify something here or will just read it as it says.module.buckets.module.access_logging_bucket.aws_s3_bucket.default will be updated in-place ~ resource "aws_s3_bucket" "default"
where it removes the grant
and lifecycle_rule
. See below s3_bucket.tf
TF Code:
cloudtrail_bucket.tf:
data "aws_caller_identity" "current" {}
resource "random_id" "cloudtrail_bucket_suffix" {
keepers = {
# Keep the suffix per account id / environment
aws_account_id = "${data.aws_caller_identity.current.account_id}"
env = "${var.environment}"
}
byte_length = "4"
}
resource "aws_s3_bucket" "cloudtrail_logging_bucket" {
bucket = "ab-${var.environment}-cloudtrail-logging-${random_id.cloudtrail_bucket_suffix.hex}"
acl = "private"
depends_on = [data.template_file.dependencies]
tags = {
name = "Cloudtrail logging bucket"
"mycompany:finance:accountenvironment" = "${var.environment}"
"mycompany:finance:application" = "${module.constants.finance_application}"
"mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
}
lifecycle {
ignore_changes = [ "server_side_encryption_configuration" ]
}
logging {
target_bucket = "${module.access_logging_bucket.name}"
target_prefix = "logs/cloudtrail-logging/"
}
lifecycle_rule {
enabled = "true"
transition {
days = 32
storage_class = "INTELLIGENT_TIERING"
}
}
}
resource "aws_s3_bucket_policy" "cloudtrail_logging_bucket" {
bucket = "${aws_s3_bucket.cloudtrail_logging_bucket.id}"
policy = "${data.aws_iam_policy_document.restrict_access_cloudtrail.json}"
}
data aws_iam_policy_document "restrict_access_cloudtrail" {
statement {
sid = "AWSCloudTrailAclCheck"
effect = "Allow"
actions = ["s3:GetBucketAcl"]
resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}" ]
principals {
identifiers = ["cloudtrail.amazonaws.com"]
type = "Service"
}
}
statement {
sid = "AWSCloudTrailWrite"
effect = "Allow"
actions = ["s3:PutObject"]
resources = [ "${aws_s3_bucket.cloudtrail_logging_bucket.arn}/*" ]
principals {
identifiers = ["cloudtrail.amazonaws.com"]
type = "Service"
}
condition {
test = "StringEquals"
values = ["bucket-owner-full-control"]
variable = "s3:x-amz-acl"
}
}
}
s3_bucket.tf
resource "random_id" "bucket_suffix" {
keepers = {
# Keep the suffix per account id / environment
aws_account_id = "${data.aws_caller_identity.current.account_id}"
env = "${var.environment}"
}
byte_length = "${var.byte_length}"
}
resource "aws_s3_bucket" "default" {
bucket = "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}"
acl = "${var.acl}"
depends_on = [data.template_file.dependencies]
tags = {
name = "${var.name_tag}"
"mycompany:finance:accountenvironment" = "${var.environment}"
"mycompany:finance:application" = "${module.constants.finance_application}"
"mycompany:finance:billablebusinessunit" = "${module.constants.finance_billablebusinessunit}"
}
lifecycle {
ignore_changes = [ "server_side_encryption_configuration" ]
}
logging {
target_bucket = "${lookup(var.logging, "target_bucket", "ab-${var.environment}-${var.name}-${random_id.bucket_suffix.hex}")}"
target_prefix = "logs/${lookup(var.logging, "target_folder_name", "access-logging")}/"
}
}
My Environment:
Local machine: macOS v10.14.6
Terraform v0.12.29
+ provider.aws v3.14.1
+ provider.null v2.1.2
+ provider.random v2.3.1
+ provider.template v2.1.2
Such a difference may be shown, if the terraform-code differs from the imported existing resource. eg if someone changed the resource by click-and-point in the AWS management console, without editing/applying the code. terraform import only imports the resource to the tfstate, but does not create terraform code.
In this sample, you could validate in the AWS console/cli if the S3 bucket "default" actually has logging configured, or not. Acording to the plan, the existing bucket is not configured for logging in AWS, but your TF-code contains this, so it would be changed.
Are you sure, your TF-code exactly matches all the attributes of the imported resource?
For further investigation, it would be necessary, that you post the corresponding tf-code as well.
Just answering my own question so that I could mentioned what I did for each question:
For the 2nd question
it does not modify anything with terraform apply
and simply reads it as it says in the message.
For my 3rd question
asked it in a separate SO thread here: Terraform - Updating S3 Access Control: Question on replacing acl with grant and went ahead with the solution I mentioned in the answer there.
For the 1st question
it's still not clear why it was showing the diff. I tried to compare the existing state with terraform state pull
and check, why the update is done but it didn't help. But running terraform apply
for that went fine and it did not make any change in the policy which is what I expect it to do.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.