CloudWatch log filter count metric values are < 1

Question

I followed the instructions here: https://docs.amazonaws.cn/en_us/AmazonCloudWatch/latest/logs/CountOccurrencesExample.html

and created a log filter metric to count occurrences of a particular logged term

But when I graph the metric I get:

I don't see how a value of < 1 is possible for a count metric.

It seems like it is calculating something else, perhaps the ratio of hits for the log filter query vs total number of log entries. But that's a meaningless stat because these are application logs so it's not even the ratio of hits vs no of requests.

The shape of the graph looks right, but the units don't make sense.

How do I get a meaningful count from a log filter metric?

Answer 1

After thinking about this further I realised what maybe should have been obvious already... I was graphing the average rate of a count.

This can very easily be < 1

One option would be to instead graph the sum (per time bucket) of the count, so that is an easy way to get "occurrences per minute" or per second or whatever.

I realised eventually what I really wanted was the percentage of a specific subset of logs lines (potential matches) where the log term matched.

I achieved this by creating another metric which counted both the matching and non-matching instances of this log term, for a specific log path (eg requests to a particular endpoint, or calls to a specific function)

Then I can then hide both those metric lines from the graph and instead add a 'math expression' like m1 / m2 * 100 and show that to graph the percentage of requests which feature the log term of interest.