How to trigger a Lambda on S3 file upload

Question

I can't seem to get my lambda to trigger as I would expect. In AWS EventBridge, I created a rule with the following custom event pattern:

{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": [
      "CopyObject",
      "CompleteMultipartUpload",
      "PutObject"
    ],
    "requestParameters": {
      "bucketName": ["my-bucket"],
      "key": [{"prefix": "folder1/folder2/"}]
    }
  }
}

but when I upload a file to the "directory" the rule does not trigger the lambda.

Can someone tell me where I've gone wrong?

Answer 1

You can use event notification in S3. It has direct integration with Lambda, SNS, SQS

• Goto properties tab in S3

• Navigate to Event notification. click on create event notification.

• Then add the event name.

• If you want to restrict the event to a specific folder or file type, you can fill in prefix or suffix fields or if you want it for entire bucket leave those blank.

• Then select a list of events. like put, post.

• Then choose the destination and click on save. You will be notified.

Answer 2

You can use both S3 events or cloud watch event rule to invoke the lambda function for any event on S3.

• Below Image shows how to configure cloud watch event rule for s3 putObject operation.
• Make sure you enabled cloud trail for data events in the respective region.
• Make sure you create a rule for the specific bucket.
• if you mention is for all the buckets. You will get unnecessary invocations, as cloud watch event rules works on cloud trail, where it will store all the logs to s3.

Answer 3

As mentioned in my comment to your question before, you might not have the proper permissions to allow EventBridge to Invoke your Lambda.

You can add the following Resource-based policy to your Lambda :

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "AllowExecutionFromEventBridge",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "<lambda-arn>",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "<eventbridge-rule-arn>"
        }
      }
    }
  ]
}

You need to replace <lambda-arn> and <eventbridge-rule-arn> with the respective values.

You can read more about resource-based policy here: Using resource-based policies for AWS Lambda .

If you use tools like Terraform, you can use the following snippet:

resource "aws_lambda_permission" "example" {
  statement_id  = "AllowExecutionFromCloudWatch"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.example.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.example.arn
}

You need to replace function_name and source_arn here as well, with your specific references.

Answer 4

Thanks to all for the help and suggestions - all good resources should someone come across this post in the future. Turns out in my case @Balu Vyamajala was correct in the comments - I had incorrectly configured CloudTrail.

Answer 5

As of 2021-11-29, Amazon Event Bridge now supports Amazon S3 Event Notifications without having to enable CloudTrial .

In the properties of the S3 bucket you want to monitor, you have to enable Amazon EventBridge.

This will allow you to send messages to EventBridge using Rules.

In your eventbridge rule, you can configure lambda as a "Target". I made a step-by-step tutorial on how to configure eventbridge in AWS if you want to follow along: https://youtu.be/k-jEuNb_KBM