Create CloudWatch alarm on multiple SQS queues

Question

Suppose I have N EC2 instances in auto-scaling group, each of them is polling M SQS queues. How would I create an alarm on cumulative ApproximateNumberOfMessagesVisible across all SQS queues if possible?

Answer 1

There is no cumulative count of the visible messages metric available as of now.

Here is how you can solve it,

Create a Lambda/ cron job that can poll the queues, get ApproximateNumberOfMessages on each queue and update the total messages to a custom Cloudwatch Metric.

QueueAttributes:

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html

Custom Cloudwatch Metric:

http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html

You can either create a lambda (serverless) or if you want to server based, you can poll and update total count into a custom cloudwatch metric with cli to AWS cloudwatch.

Once you have have the cloudwatch metric, you can create an alarm based on those numbers.

Hope it helps.

Answer 2

You can define a Alarm with cloud formation. The key is define a custom
Expression where each expression has a one or multiples metrics by queue. The Dimesions allow associate metric with queue.

MyDeadLetterQueueAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: my-dlq-alarm
    AlarmDescription: 'Alarm when any Dead letter queue has items.'
    Metrics:
    - Id: summary
      Label: Dead Letter Queues Alarm
      Expression: IF(dlq1 > 1, 1, 0) OR IF(dlq2 > 1, 1, 0)
      ReturnData: true
    - Id: dlq1
      MetricStat:
        Metric:
          Namespace: AWS/SQS
          MetricName: ApproximateNumberOfMessagesVisible
          Dimensions:
          - Name: QueueName
            Value: !GetAtt
            - MyDeadLetterQueue1
            - QueueName
        Stat: Sum
        Period: 300
      ReturnData: false
    - Id: dlq2
      MetricStat:
        Metric:
          Namespace: AWS/SQS
          MetricName: ApproximateNumberOfMessagesVisible
          Dimensions:
          - Name: QueueName
            Value: !GetAtt
            - MyDeadLetterQueue2
            - QueueName
        Stat: Sum
        Period: 300
      ReturnData: false
    EvaluationPeriods: 1
    DatapointsToAlarm: 1
    Threshold: 0
    ComparisonOperator: GreaterThanThreshold
    TreatMissingData: notBreaching
    AlarmActions:
      - !Ref MyTopicArn

I think you should sum the expression to has the acumulative value.