Filter log streams in aws cloudwatch

Question

Is there anyway to filter the log streams with patterns using the CloudWatch console?

For example, I have the following log streams in a log group - Log Group:

'/var/prod/logs'.

Log Streams:

/prod/[node_instance_id]/nginx_access
/prod/[node_instance_id]/nginx_error

I have a multi-node environment with auto scaling etc. So the log streams can be quite messy - here is an example of what I see in the log streams.

/prod/1a2b3c4d5e/nginx_access
/prod/1a2b3c4d5e/nginx_error
/prod/1b2b3c4d5e/nginx_access
/prod/1b2b3c4d5e/nginx_error
/prod/1c2b3c4d5e/nginx_access
/prod/1c2b3c4d5e/nginx_error

I am trying to filter the log streams to be all 'nginx_access' only. But from the console, it looks like I can only specify the prefix. Is there anyway that I could filter log streams using something like '*nginx_access'?

Answer 1

The DescribeLogStreams API only supports filtering by prefix, and the console is listing your log streams with that API. It is not possible to filter by something other than a prefix.

The best practice in your case would be to use 2 log groups, one for each type:

• /var/prod/nginx_access
• /var/prod/nginx_error

That way you can navigate to your logs by first choosing the appropriate log group, and then searching for the instance id with the prefix filter.