Adding multiple log groups to Cloudwatch event triggers without having to specify them individually?

Question

I would like to trigger a Lambda function on error events generated from my other Lambda functions. I see it is possible to add multiple log groups using Serverless. Example from here .

functions:
  hello1:
    handler: handler.hello1
    events:
      - cloudwatchLog: '/aws/lambda/hello1'
  hello2:
    handler: handler.hello2
    events:
      - cloudwatchLog: '/aws/lambda/hello2'

however that means defining an event + filter for each function (I have about 2 dozen) and this would need to be updated for any additional functions I add in the future. Is there any way to use a catch-all log group definition like /aws/lambda/* ?

I already set up a Cloudwatch alarm using a metric filter for the aws/lambda namespace with the Errors metric. Easy and done, however the SNS topic it publishes contains little information regarding the nature of the error. I want to create a Lambda function that publishes an SNS topic directly that includes the stack trace. I'm hoping it would be as easy to select-all log groups in the same way.

Answer 1

You would need an architectural change.

Create a Kinesis data stream and subscribe individual cloudwatch log groups to that single kinesis stream

The lambda that processes the error logs needs to have Kinesis as event source.

This would now enable triggers from the kinesis for logs from any of the lambda functions' cloudwatch log groups

To enable this subscription automatically, use serverless itself with Kinesis CloudWatch subscription and the kinesis stream is going to be same for all functions.

Serverless has a plugin to do the same (serverless-plugin-log-subscription) https://www.serverless.com/plugins/serverless-plugin-log-subscription