​

Enable GPU-level metrics

​

📦 Step 1: Enable Container Insights with GPU Support

​

✅ Automatic Setup (recommended):

1. Ensure your EKS cluster has OIDC enabled:
   Copy

   eksctl utils associate-iam-oidc-provider --cluster my-cluster --approve
   
2. Attach the necessary IAM access:
   Copy

   aws iam attach-role-policy \
     --role-name <EKSNodeInstanceRole> \
     --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
   
3. Enable the Container Insights add-on with GPU support:
   Copy

   aws eks create-addon \
     --cluster-name my-cluster \
     --addon-name amazon-cloudwatch-observability \
     --service-account-role-arn <CloudWatchAgentServiceAccountRoleArn> \
     --resolve-conflicts OVERWRITE
   

​

🧩 Step 2: Verify GPU Metrics in CloudWatch

• GPU Utilization
• GPU Memory Usage
• GPU Temperature
• GPU Power Consumption

​

📈 Step 3: Visualize in Grafana

​

If using Managed Grafana:

1. Create or use an existing Grafana workspace.
2. Add CloudWatch as a data source (supports Container Insights).
3. Import GPU Dashboard:
   • Browse dashboards in Grafana.com, or import a custom one.
   • Alternatively, build your own panel using metrics like: ContainerInsights → node_gpu_usage_total across ClusterName, NodeName ([aws.amazon.com][5], [grafana.com][6], [docs.aws.amazon.com][7]).

​

If using self-hosted Grafana:

1. Configure CloudWatch plugin (from Grafana > 6.5+).
2. Follow the same import or build dashboard steps.

​

🧭 Dashboard & Metrics to Use

• node_gpu_limit, node_gpu_usage_total, node_gpu_reserved_capacity — for node-level GPU capacity and usage.
• pod_gpu_usage_total, pod_gpu_request — for pod-level GPU consumption metrics

​

🎯 Why This Approach Works

• Fully automated: no manual device plugin, driver, or containerd tweaks
• Managed and supported by AWS
• Integrates seamlessly with Grafana
• Includes logs + metrics for end-to-end observability

​

Terraform-based solution

​

🚀 Step 1: Enable Container Insights with GPU support

data "aws_eks_cluster" "this" {
  name = var.eks_cluster_name
}

resource "aws_eks_addon" "cw_observability" {
  cluster_name             = data.aws_eks_cluster.this.name
  addon_name               = "amazon-cloudwatch-observability"
  addon_version            = "v2.1.2-eksbuild.1"  # or latest
  resolve_conflicts_on_update = "OVERWRITE"
  service_account_role_arn = aws_iam_role.cloudwatch_sa.arn

  configuration_values = jsonencode({
    agent = {
      config = {
        logs = {
          metrics_collected = {
            kubernetes = {
              enhanced_container_insights      = true
              accelerated_compute_metrics       = true
            }
          }
        }
      }
    }
  })
}

resource "aws_iam_role" "cloudwatch_sa" {
  name = "${var.eks_cluster_name}-cw-agent"

  assume_role_policy = data.aws_iam_policy_document.eks_assume_sa.json
}

resource "aws_iam_role_policy_attachment" "cw_agent_attachment" {
  role       = aws_iam_role.cloudwatch_sa.name
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
}

​

🛠 Step 2: Grant the IAM Role to the Service Account

data "aws_iam_policy_document" "eks_assume_sa" {
  statement {
    effect = "Allow"
    principals {
      type        = "Federated"
      identifiers = [data.aws_iam_openid_connect_provider.oidc.arn]
    }
    actions = ["sts:AssumeRoleWithWebIdentity"]
    condition {
      test     = "StringEquals"
      variable = "${data.aws_iam_openid_connect_provider.oidc.url}:sub"
      values   = ["system:serviceaccount:amazon-cloudwatch:cloudwatch-agent"]
    }
  }
}

data "aws_iam_openid_connect_provider" "oidc" {
  url = data.aws_eks_cluster.this.identity[0].oidc[0].issuer
}

​

🔍 Step 3: Validate in CloudWatch

• CloudWatch → Container Insights → EKS
• You should see panels for GPU usage, memory, temperature, and power ([docs.aws.amazon.com][4], [blog.devops.dev][5]).

​

📊 Step 4: Visualize GPU Metrics in Grafana

module "mgw" {
  source  = "terraform-aws-modules/grafana/aws"
  version = "x.y.z"

  workspace_name = "eks-gpu-monitoring"
  authentication = {
    providers = ["AWS_SSO"]
  }

  permissions = [{
    principal = aws_iam_role.cloudwatch_sa.arn
    role      = "ADMIN"
  }]
}

1. Add CloudWatch data source (select Container Insights).
2. Upload/import a dashboard watching:
   • ContainerInsights/node_gpu_usage_total
   • ContainerInsights/node_gpu_limit
   • ContainerInsights/pod_gpu_usage_total You can also import a premade GPU dashboard or build from scratch using these metrics ([aws-observability.github.io][6]).

​

✅ Summary of the Full Flow

​

🔧 What You Should Do Next

1. Copy the Terraform snippets above into your existing config.
2. Run terraform init && terraform apply.
3. Wait ~5 minutes for the add-on to deploy.
4. Confirm GPU metrics exist in CloudWatch.
5. Set up Grafana with CloudWatch integration and visualize your GPU dashboards.