How to monitor EC2 instances by memory?

Question

Using Cloudwatch you can monitor your EC2 instances by several criteria, such as.network usage, CPU usage, and so on…

Unfortunately, there is no metric for memory consumption. First of all, just out of curiosity, I would like to know, why? Can anybody explain why it is possible to, eg, monitor CPU usage, but not memory usage? At least to me, that's not obvious.

And then, my actual question: Okay, given that Cloudwatch doesn't allow monitoring the EC2 instances' memory usage - what is the alternative? How should I setup an alarm if, eg > 80% of the memory of an instance is being used?

Answer 1

Memory and Disk specific statistics require AWS to monitor at the OS level rather than the host level, so that is why they leave it out by default. It will probably be added at some point but since it has been on the wish list for about 7 years, we can assume it's a very low priority item.

The recommended way to monitor memory usage is to create a custom Cloudwatch metric by using your own monitoring scripts on your instances. AWS have published documentation on how to achieve this on Linux instances with a set of (unsupported) scripts.

Once your instances are publishing the custom metrics, you will be able to attach alarms to them in CloudWatch.

Answer 2

Well, now the new CloudWatch agent can collect metrics like memory and disk usage, see the docs.

With this you can monitor this kind of metrics, but you will need to install and configure the agent in the instances.

Answer 3

Well by default the AWS Ec2 does not provide memory metrics. So we have to install the cloud watch agent separately and then get the metrics

Steps:

1. Download the cloud watch agent from s3

   sudo wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

2. Install the cloud watch agent

   sudo rpm -U./amazon-cloudwatch-agent.rpm

3. We need to write a config file in which we will specify the metrics we need to put.

   sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json

you need to insert this inside the config file

{
    "metrics": {
        "metrics_collected": {
            "mem": {
                "measurement": [
                    "mem_used_percent"
                ],
                "metrics_collection_interval": 30
            }
        }
    }
}

4. Then after specifying the metric then we just need to start the cloud watch agent

   sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

5. You can go and test out if the metric is coming or not in the cloud watch page

Check:

Cloudwatch console --> All metrics --> under custom namespace --> CW Agent --> host --> ip of the instance server

Now you can view the memory usage and monitor the uage