Terraform: Codepipeline not triggered on upload to s3 source bucket

Question

I am trying to trigger the codepipeline on upload to s3 using terraform.

Use case - So a terraform code for various resources will be pushed as a zip file to the source bucket which will trigger a pipeline. This pipeline will run terraform apply for the zip file. So in order to run the pipeline I am setting up a trigger

Here is what I have done.

• Create source s3 bucket
• Create code pipeline
• Created cloudwatch events rule for s3 events fro cloudtrail
• Created cloudTrail Manually, and added data event to log source bucket write events. , all previous steps were done using terraform.

After doing all this still, my pipeline is not triggered on upload of new bucket.

I was reading this docs and it had particular statement about sending trail events to eventbridge rule which I think is the cause but I can't find the option to add through console.

AWS CloudTrail is a service that logs and filters events on your Amazon S3 source bucket. The trail sends the filtered source changes to the Amazon CloudWatch Events rule. The Amazon CloudWatch Events rule detects the source change and then starts your pipeline.

https://docs.aws.amazon.com/codepipeline/latest/userguide/create-cloudtrail-S3-source.html

Here is my event ridge rule

resource "aws_cloudwatch_event_rule" "xxxx-pipeline-event" {
  name        = "xxxx-ci-cd-pipeline-event"
  description = "Cloud watch event when zip is uploaded to s3"

  event_pattern = <<EOF
{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject", "CompleteMultipartUpload", "CopyObject"],
    "requestParameters": {
      "bucketName": ["xxxxx-ci-cd-zip"],
      "key": ["app.zip"]
    }
  }
}
EOF
}

    resource "aws_cloudwatch_event_target" "code-pipeline" {
  rule      = aws_cloudwatch_event_rule.XXXX-pipeline-event.name
  target_id = "SendToCodePipeline"
  arn       = aws_codepipeline.cicd_pipeline.arn
  role_arn  = aws_iam_role.pipeline_role.arn
}

Event bridge role permissions terraform code

data "aws_iam_policy_document" "event_bridge_role" {
  statement {
    actions = ["sts:AssumeRole"]
    effect  = "Allow"
    principals {
      type        = "Service"
      identifiers = ["events.amazonaws.com"]
    }
  }

}

resource "aws_iam_role" "pipeline_event_role" {
  name               = "xxxxx-pipeline-event-bridge-role"
  assume_role_policy = data.aws_iam_policy_document.event_bridge_role.json
}

data "aws_iam_policy_document" "pipeline_event_role_policy" {
  statement {
    sid       = ""
    actions   = ["codepipeline:StartPipelineExecution"]
    resources = ["${aws_codepipeline.cicd_pipeline.arn}"]
    effect    = "Allow"
  }
}

resource "aws_iam_policy" "pipeline_event_role_policy" {
  name   = "xxxx-codepipeline-event-role-policy"
  policy = data.aws_iam_policy_document.pipeline_event_role_policy.json
}

resource "aws_iam_role_policy_attachment" "pipeline_event_role_attach_policy" {
  role       = aws_iam_role.pipeline_event_role.name
  policy_arn = aws_iam_policy.pipeline_event_role_policy.arn
}

Answer 1

The problem was with CLoudtrail filter. The filter was set for bucket and write actions.

I had to modify filter by adding prefix to it.Because my event bridge is looking for my-app.zip so it was not triggered if I used only bucket level prefix

bucket/prefix and write action

Docs: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html