How to create S3 and triggered lambda in 2 different cloudformation templates

Question

Can a s3 bucket and triggered lambda be created in separate cloudformation templates. I want to keep long running resources stack separate from the likes of lambda which get updated quite frequently

When tried to create lambda separately it says that bucket defined in lambda event should be defined in same template and cannot be referenced.

GetFileMetadata:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      FunctionName: !Sub '${targetenv}-lambdaname'
      CodeUri: target-file-0.0.1-SNAPSHOT.jar
      Handler: LambdaFunctionHandler::handleRequest
      Runtime: java8
      Timeout: 30
      MemorySize: 512
      Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        Variables:
          STAGE: !Sub '${targetenv}'

      Events:
        S3Event:
          Type: S3
          Properties:
            Bucket:
              Ref: MyS3Bucket
            Events:
              - 's3:ObjectCreated:*'

  MyS3Bucket:
      Type: 'AWS::S3::Bucket'
      DependsOn: BucketPermission
      Properties:
          BucketName: !Sub 'bucketname-${targetenv}'
         # LifecycleConfiguration: 
          #  Rules:
           # - ExpirationInDays: 14

Answer 1

The template is creating a bucket ( MyS3Bucket ).

Then, the serverless function is referencing it:

        Bucket:
          Ref: MyS3Bucket

If you want to refer to that bucket from another template, you can export the bucket name from the first stack:

Outputs:

  S3Bucket:
    Description: Bucket that was created
    Value: !Ref MyS3Bucket
    Export:
      Name: Stack1-Bucket

Then, import it into the second stack:

        Bucket:
            Fn::ImportValue:
              Stack1-Bucket

See: Exporting Stack Output Values - AWS CloudFormation

Answer 2

This is not possible in SAM version 2016-10-31. Copied from the S3 event source type in the SAM documentation:

NOTE: To specify an S3 bucket as an event source for a Lambda function, both resources have to be declared in the same template. AWS SAM does not support specifying an existing bucket as an event source.

Answer 3

On November 21 2021, AWS announced S3 Event Notifications with Amazon EventBridge . Consequently, you can deploy one stack with an S3 bucket with EventBridge integration enabled and then a second stack with a Lambda function that is triggered by EventBridge events for the specific bucket.

Persistence Stack:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: 'Stack with S3 bucket with EventBridge event notification enabled'

Parameters:
  BucketName:
    Type: String
    Description: 'Name of the bucket to be created'

Resources:

  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      NotificationConfiguration:
        EventBridgeConfiguration:
          EventBridgeEnabled: true

#       Alternatively shorthand config
#       EventBridgeConfiguration: {}

Application Stack:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Stack with Lambda for procesing S3 events via EventBridge

Parameters:
  BucketName:
    Type: String
    Description: Name of the bucket to listen events from

Resources:
  S3EventProcessor:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: S3EventListener
      Architectures:
        - arm64
      Runtime: nodejs14.x
      Handler: index.handler
      InlineCode: |
          exports.handler = (event, context) => {
            console.log('event:', JSON.stringify(event));
          }
      Events:
        S3EventBridgeRule:
          Type: EventBridgeRule
          Properties:
            Pattern:
              source:
                - aws.s3
              detail:
                bucket:
                  name:
                    - !Ref BucketName

By configuring the Pattern , you can filter the events stream for more specific events such as Object Create or Object Deleted , file names, file extension, etc. Please find more info in the EventBridge userguide