Sending logs from one serverless function to multiple cloudwatch groups

Question

I am developing an API for an app with python, using FastAPI, serverless and amazon web services. We are using CloudWatch for saving our logs. The thing is, I am required to send different logs to different groups in the same aplication, depending on wethever is it an error, an info, etc.

Let´s say, I have two log groups in CloudWatch: /aws/lambda/firstGroup and /aws/lambda/secondGroup.

An I have this function:

def foo(some_data):
    logger.info(f'calling the function with the data: {data}') # this goes to logGroup /aws/lambda/firstGroup
    try:
        doSomething()
    except:
        logger.error('ERROR! Something happened') # this goes to logGroup /aws/lambda/secondGroup

How can I configure the serverless.yml file so the logger.info goes to the first group and the logger.error goes to the second group?

Thanks in advance!

Answer 1

I am using this solution with ec2 instance.

• create a log group
• create log stream
• then dump your logs

import boto3
import time


# init clients
clw_client = boto3.client('logs', region_name=REGION)


# print("check else create new log group..")
try:
    clw_client.create_log_group(logGroupName=LOG_GROUP)
except clw_client.exceptions.ResourceAlreadyExistsException:
    pass

# print("check else create new log stream.....")
LOG_STREAM = '{}-{}'.format(time.strftime("%m-%d-%Y-%H-%M-%S"),'logstream')
try:
    clw_client.create_log_stream(logGroupName=LOG_GROUP, logStreamName=LOG_STREAM)
except clw_client.exceptions.ResourceAlreadyExistsException:
    pass
    
def log_update(text):
    print(text)
    response = clw_client.describe_log_streams(
        logGroupName = LOG_GROUP,
        logStreamNamePrefix = LOG_STREAM
        )
    
    try:
        event_log = {
            'logGroupName': LOG_GROUP,
            'logStreamName': LOG_STREAM,
            'logEvents': [{
                'timestamp': int(round(time.time() * 1000)),
                'message': f"{time.strftime('%Y-%m-%d %H:%M:%S')}\t {text}"
                }
            ],
        }

        if 'uploadSequenceToken' in response['logStreams'][0]:
            event_log.update({'sequenceToken': response['logStreams'][0] ['uploadSequenceToken']})

        response = clw_client.put_log_events(**event_log)
    except Exception as e:
        log_update(e)

Then use call that function inside your app whenever you like. Just don't check for groups and stream again and again for one job, these should run once.

You can update it, add more logic like change log-group name to implement what you wanted in this question with just some if else statements. Good luck!