Monitoring AWS EC2 Memory and Disk Metrics with CloudWatch

By default, Amazon CloudWatch monitors core EC2 metrics such as CPU utilization, network throughput, and disk read/write operations. However, because AWS cannot inspect inside your guest operating system, CloudWatch does not report memory utilization, active swap usage, or filesystem disk space availability.

To capture these critical guest-level metrics, you need to deploy custom CloudWatch monitoring scripts that report statistics directly to the CloudWatch API via IAM role authentication.

In this guide, I will walk you through installing required Perl dependencies on Ubuntu, downloading the CloudWatch monitoring scripts, establishing strict IAM permissions, configuring credentials, and scheduling automated reporting via cron.

Prerequisites

You will need an AWS EC2 instance running Ubuntu/Debian with root privileges, alongside IAM permissions to create custom policies and roles.

---

Step 1: Installing Dependencies and Downloading Scripts

Connect to your EC2 instance via SSH, escalate privileges, and install required Perl libraries alongside unzip utilities.

# Escalate to root
sudo su -

# Install unzip and required Perl dependencies
apt-get update -y
apt-get install unzip libwww-perl libdatetime-perl -y

Download and unpack the official CloudWatch monitoring scripts archive:

# Download monitoring scripts archive
curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O

# Unpack archive and remove zip
unzip CloudWatchMonitoringScripts-1.2.2.zip
rm CloudWatchMonitoringScripts-1.2.2.zip
cd aws-scripts-mon/

---

Step 2: Configuring IAM Role and Permissions

To allow your EC2 instance to push custom metrics to CloudWatch, assign an IAM role attached to the instance profile with the following required policy permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:PutMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "ec2:DescribeTags"
      ],
      "Resource": "*"
    }
  ]
}

Alternative Programmatic IAM User Credentials

If you are unable to attach an IAM role directly to your VM, you can create a dedicated IAM user (mon-user-devops) with programmatic access, assign the policy above, and configure a local credentials file:

# Copy credentials template
cp awscreds.template awscreds.conf

# Populate with IAM user Access Key and Secret Key
cat > awscreds.conf << "EOF"
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE
AWSSecretKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
EOF
chmod 600 awscreds.conf

---

Step 3: Testing Metrics and Scheduling Cron Reporting

Before automating the script, execute a manual test run to verify that memory, swap, and disk space metrics are successfully collected and transmitted to CloudWatch.

./mon-put-instance-data.pl --mem-util --mem-used --mem-avail --swap-util --swap-used --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --aws-credential-file=/root/aws-scripts-mon/awscreds.conf --verbose

Once verified, add a cron job to automate reporting every 5 minutes:

# Add reporting cron job
echo "*/5 * * * * /root/aws-scripts-mon/mon-put-instance-data.pl --mem-util --mem-used --mem-avail --swap-util --swap-used --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --aws-credential-file=/root/aws-scripts-mon/awscreds.conf --from-cron" >> /var/spool/cron/crontabs/root

---

Step 4: Advanced CloudWatch Logs Agent Setup

To transmit system log files (like /var/log/auth.log) to CloudWatch Logs alongside your metrics, install and configure the legacy CloudWatch Logs agent.

# Download CloudWatch Logs setup script
curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O
apt-get install python -y

# Execute interactive setup
python ./awslogs-agent-setup.py --region us-east-2

Interactive Prompts Summary

• AWS Access Key ID: Provide IAM credentials
• Default region name: us-east-2
• Path of log file to upload: /var/log/auth.log
• Destination Log Group name: auth_log
• Choose Log Stream name: Use EC2 instance id

Enable and start the logging daemon:

systemctl enable awslogs
systemctl start awslogs
systemctl status awslogs

References

• AWS CloudWatch Monitoring Scripts Documentation