Can we view Logs stored in S3 using CloudWatch?

Question

My application is storing logs in S3 in a specific format. But currently, I'm not able to view those logs directly. Can we use AWS CloudWatch to view those logs that are stored in S3?

When I checked, I saw that we can use CloudWatch Logs to create Log group and then store logs to that using cloudwatch agent. But is there a way to import logs from S3 to CloudWatch and view them on the cloudwatch logs section?

Answer 1

Possible alternative:

Consider using AWS Athena to view and query your application logs that are stored in S3.

For example, here's a guide to using Athena to query CloudFront access logs that are stored in S3:

https://docs.aws.amazon.com/athena/latest/ug/cloudfront-logs.html

Answer 2

Yes you can but not directly,

Two options..

1. Lambda function trigger from s3:PutObject events then send to CW. Probably costly and not as effective as 2 obviously depends on the logs amount or log pulling frequency.
2. Use cwagent in your application and stream logs directly to a CW log group.

Option two is the best, good luck!

Answer 3

You can use Amazon Athena to query Application server logs stored in Amazon S3. See AWS documentation for Querying web server logs stored in Amazon S3 .