Skip to main content

​
title: “CloudWatch to Slack Alerting” description: “Setup CloudWatch alarms for EC2 and ALB with enriched notifications sent to Slack via SNS and Lambda”

​
📘 CloudWatch → Slack Alerting Guide

Scope: Setup CloudWatch alarms for EC2 CPU utilization and ALB 4xx/5xx errors, centralize alarms via SNS + Lambda, and send enriched notifications to Slack with alarm name, resource details, and deep links.

​
1. Prerequisites

  • AWS Account with IAM admin or equivalent privileges.
  • Slack workspace with permission to create Apps or use Incoming Webhooks.
  • Python-based AWS Lambda runtime (3.9+ recommended).
api.slack.com hooks.slack.com

​
Slack API Documentation for Webhooks APP

https://docs.slack.dev/quickstart

​
2. Architecture Overview

CloudWatch Alarm → SNS Topic → Lambda → Slack Webhook
  • CloudWatch Alarms: Detect threshold breaches (CPU, ALB 4xx/5xx).
  • SNS Topic: Aggregates all alarms → single event pipeline.
  • Lambda Function: Parses alarm event, enriches with details, and posts to Slack.
  • Slack: Receives structured, actionable alerts.

​
3. Step 1 — Create an SNS Topic

aws sns create-topic --name cloudwatch-slack-topic
  • Note the Topic ARN (e.g., arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic).

​
4. Step 2 — Create CloudWatch Alarms

​
4.1 CPU Alarm (EC2)

  • Metric: AWS/EC2 → CPUUtilization.
  • Dimension: InstanceId=i-xxxxxxxx.
  • Threshold: > 80% for 5 minutes.
aws cloudwatch put-metric-alarm \
  --alarm-name "HighCPUUtilization" \
  --metric-name CPUUtilization \
  --namespace AWS/EC2 \
  --statistic Average \
  --period 300 \
  --threshold 80 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --dimensions Name=InstanceId,Value=i-0abcd1234efgh5678 \
  --alarm-actions arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic

​
4.2 ALB 5xx Alarm

  • Metric: AWS/ApplicationELB → HTTPCode_ELB_5XX_Count.
  • Dimension: LoadBalancer=app/my-alb/1234567890abcdef.
  • Threshold: > 10 errors in 5 minutes.
aws cloudwatch put-metric-alarm \
  --alarm-name "ALB-5XX-Errors" \
  --metric-name HTTPCode_ELB_5XX_Count \
  --namespace AWS/ApplicationELB \
  --statistic Sum \
  --period 300 \
  --threshold 10 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --dimensions Name=LoadBalancer,Value=app/my-alb/1234567890abcdef \
  --alarm-actions arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic

​
4.3 ALB 4xx Alarm

  • Same as 5xx, but HTTPCode_ELB_4XX_Count.
aws cloudwatch put-metric-alarm \
  --alarm-name "ALB-4XX-Errors" \
  --metric-name HTTPCode_ELB_4XX_Count \
  --namespace AWS/ApplicationELB \
  --statistic Sum \
  --period 300 \
  --threshold 100 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --dimensions Name=LoadBalancer,Value=app/my-alb/1234567890abcdef \
  --alarm-actions arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic

​
5. Step 3 — Setup Slack Webhook

​
Option A (Quick & Dirty)

  • Create an Incoming Webhook in Slack → get URL like: https://hooks.slack.com/services/TXXX/BXXX/XXXXXXXX
  • Create a Slack App → Enable Incoming Webhooks → Install to Workspace → Copy Webhook URL.

​
6. Step 4 — Create Lambda Function

​
6.1 Lambda IAM Role

Grant Lambda access to:
  • AWSLambdaBasicExecutionRole.
  • AmazonSNSFullAccess (or SNS Subscribe).

​
6.2 Lambda Code (Python 3.9+)

import json
import urllib3

http = urllib3.PoolManager()
slack_webhook_url = "https://hooks.slack.com/services/XXXX/XXXX/XXXX"  # Replace

def lambda_handler(event, context):
    for record in event['Records']:
        message = json.loads(record['Sns']['Message'])

        alarm_name = message.get('AlarmName')
        description = message.get('AlarmDescription', 'No description')
        state = message.get('NewStateValue')
        reason = message.get('NewStateReason')
        region = message.get('Region')
        account = message.get('AWSAccountId')

        trigger = message.get('Trigger', {})
        metric = trigger.get('MetricName', 'N/A')
        namespace = trigger.get('Namespace', 'N/A')
        dimensions = {d['name']: d['value'] for d in trigger.get('Dimensions', [])}
        resource = ", ".join([f"{k}={v}" for k, v in dimensions.items()]) or "N/A"

        # Direct link to alarm in CloudWatch
        alarm_url = f"https://console.aws.amazon.com/cloudwatch/home?region={region}#alarmsV2:alarm/{alarm_name}"

        slack_message = {
            "attachments": [
                {
                    "color": "danger" if state == "ALARM" else "good",
                    "title": f"🚨 CloudWatch Alarm: {alarm_name}",
                    "title_link": alarm_url,
                    "fields": [
                        {"title": "State", "value": state, "short": True},
                        {"title": "Metric", "value": f"{namespace}/{metric}", "short": True},
                        {"title": "Resource", "value": resource, "short": True},
                        {"title": "Region", "value": region, "short": True},
                        {"title": "Account", "value": account, "short": True},
                        {"title": "Description", "value": description, "short": False},
                        {"title": "Reason", "value": reason, "short": False}
                    ],
                    "footer": "AWS CloudWatch",
                    "footer_icon": "https://a0.awsstatic.com/libra-css/images/logos/aws_logo_smile_1200x630.png"
                }
            ]
        }

        http.request(
            'POST',
            slack_webhook_url,
            body=json.dumps(slack_message),
            headers={'Content-Type': 'application/json'}
        )

    return "Message sent to Slack"

​
7. Step 5 — Connect SNS → Lambda

aws sns subscribe \
  --topic-arn arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic \
  --protocol lambda \
  --notification-endpoint arn:aws:lambda:ap-south-1:123456789012:function:CloudWatchToSlack
Then give Lambda permission to be invoked by SNS: 
aws lambda add-permission \
  --function-name CloudWatchToSlack \
  --statement-id snsinvoke \
  --action "lambda:InvokeFunction" \
  --principal sns.amazonaws.com \
  --source-arn arn:aws:sns:ap-south-1:123456789012:cloudwatch-slack-topic

​
8. Step 6 — Testing

  • Manually set thresholds very low (e.g., CPU > 1%) to trigger alarms.
  • Verify messages in Slack include:
    • Alarm Name
    • State (ALARM / OK)
    • Metric Name
    • Resource ID
    • AWS Region + Account
    • Direct CloudWatch Console Link
    • Reason

​
9. Enhancements (Optional)

  • Extend Lambda to include top failing ALB paths using ALB Access Logs + CloudWatch Logs Insights.
  • Route different alarms to different Slack channels by mapping AlarmName → Slack Webhook.
  • Add severity levels (critical, warning, info) based on alarm type.

​
✅ Executive Summary

  • CPU alarms tell you which EC2 instance is overutilized.
  • ALB 4xx/5xx alarms tell you which load balancer is serving bad requests.
  • Slack notifications are enriched with alarm name, metric, resource ID, and direct AWS links.
  • A single Lambda function manages all alarms, making operations lean and scalable.