How to log library logs to AWS CloudWatch using Serilog?

Question

Currently, I am working on ASP.NET 6 Web API and as a logger We use Serilog to log all the necessary details to cloudwatch and it's working fine. Now I need to add library logs such as AWS errors to cloudwatch. Currently there is an option for that in config file but it only saves logs as a file which results a No space left on device: '/app/Logs/serilog-aws-errors.txt' and the details in the file didn't appear on cloudwatch logs.

This is the appsettings data I use,

"Serilog": {
"Using": [ "AWS.Logger.SeriLog", "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
"MinimumLevel": "Debug",
"WriteTo": [
  { "Name": "AWSSeriLog" },
  { "Name": "Console" },
  {
    "Name": "File",
    "Args": {
      "path": "Logs/webapi-.txt",
      "rollingInterval": "Day"
    }
  }
],
"Region": "eu-west-2",
"LogGroup": "/development/serilog",
"LibraryLogFileName": "Logs/serilog-aws-errors.txt"
}

I need to know that there is a way to log the details in serilog-aws-errors.txt to AWS cloudwatch or S3 bucket.

Answer 1

This depends a lot on where in AWS you are trying to deploy your service. In ECS or Fargate you can log directly to the console. This would be a snippet of the container Definition:

    "containerDefinitions": [
      {
        "logConfiguration": {
          "logDriver": "awslogs",
          "options": {
            "awslogs-group": "/dev/ecs/my-api-logs-here",
            "awslogs-region": "us-east-1",
            "awslogs-stream-prefix": "ecs"
          }
        },

With the configuration above you only need the Serilog.Sinks.Console and everything will log without a special AWS sink. To write to the console you can just use

                loggerConfiguration.WriteTo.Async(a =>
                {
                    a.Console(new JsonFormatter());
                });

When deployed to Fargate or ECS, these console logs will appear in your CloudWatch logs. No additional sink is necessary. Lambda logs have a similar setup. See: https://docs.aws.amazon.com/lambda/latest/dg/csharp-logging.html for more details.

If you want to use the Serilog.Sinks.AwsCloudWatch , it does have some nice features, but the setup is a little different. You probably won't want to log to the console or your file sink at all. Instead, you'll just log directly to CloudWatch. You'll want to set it up according to their instructions on Github: https://github.com/Cimpress-MCP/serilog-sinks-awscloudwatch . You can get this up and running in your local environment and then set your app settings up in a way that this only runs when deployed to AWS, and locally you still use the console or file settings.

  var options = new CloudWatchSinkOptions
  {
    // the name of the CloudWatch Log group for logging
    LogGroupName = logGroupName,

    // the main formatter of the log event
    TextFormatter = formatter,
    
    // other defaults defaults
    MinimumLogEventLevel = LogEventLevel.Information,
    BatchSizeLimit = 100,
    QueueSizeLimit = 10000,
    Period = TimeSpan.FromSeconds(10),
    CreateLogGroup = true,
    LogStreamNameProvider = new DefaultLogStreamProvider(),
    RetryAttempts = 5
  };

  // setup AWS CloudWatch client
  var client = new AmazonCloudWatchLogsClient(myAwsRegion);

  // Attach the sink to the logger configuration
  Log.Logger = new LoggerConfiguration()
    .WriteTo.AmazonCloudWatch(options, client)
    .CreateLogger();