Development
AWSSecretManager
Awssecretmanager

End-to-End Guide: Go App + AWS Secrets Manager + RDS (With Automatic Rotation Handling)

📌 Overview

In this guide, we’ll build and deploy a Go application on an EC2 instance that:

  • Uses AWS Secrets Manager to fetch RDS database credentials.

  • Caches secrets locally for performance.

  • Handles automatic credential rotation seamlessly.

  • Provides an HTTP endpoint /secret that:

    • Connects to the DB using cached creds.
    • If creds fail (rotated), it auto-fetches new ones from Secrets Manager and retries without failing the request.
  • Logs every action:

    • Whether creds came from cache or Secrets Manager.
    • DB connection attempts.
    • Secret fetch events.

We’ll also perform load testing (1 request/sec) to validate zero downtime during credential rotation.

LoadTesting


🟢 Step 1. Provision RDS (MySQL)

  1. Go to AWS Console → RDS → Create Database

    • Engine: MySQL (e.g. version 8.0)
    • Instance class: db.t3.micro (for testing)
    • Storage: 20GB default
    • Public access: Yes (for dev/test; use private for prod)
    • Username: admin
    • Password: temporary password
  2. Wait for status = Available.

    • Note the DB endpoint (e.g. mydb.abc123.us-east-1.rds.amazonaws.com).
    • Note the DB name (e.g. appdb).

🟢 Step 2. Store Secret in Secrets Manager

  1. Go to Secrets Manager → Store a new secret

    • Type: Credentials for RDS database
    • Enter admin + password + DB info
    • Name the secret: mydb-secret
  2. (Optional) Enable automatic rotation

    • AWS will deploy a Lambda for rotation. Load Testing

🟢 Step 3. IAM Role for EC2

  1. Go to IAM → Roles → Create role

    • Trusted entity: EC2

    • Attach policy:

      • For testing: SecretsManagerReadWrite
      • For prod: secretsmanager:GetSecretValue only
  2. Name the role: EC2SecretsRole

  3. Attach role to your EC2:

    • EC2 → Instance → Security → Modify IAM role → select EC2SecretsRole

🟢 Step 4. Launch EC2 Instance

  1. Launch Amazon Linux 2023 or Ubuntu 22.04.

  2. Make sure:

    • It’s in the same VPC/subnet as RDS.

    • Security group allows:

      • EC2 → RDS MySQL (3306)
      • Your laptop → EC2 (22 for SSH, 8080 for API)
  3. Install Go:

sudo yum update -y   # (Amazon Linux)
sudo yum install -y golang
go version

🟢 Step 5. Go Application Code

Save as main.go:

// main.go
package main
 
import (
	"context"
	"database/sql"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"sync"
	"time"
 
	_ "github.com/go-sql-driver/mysql"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
)
 
type DBSecret struct {
	Username string `json:"username"`
	Password string `json:"password"`
	Host     string `json:"host"`
	DBName   string `json:"dbname"`
}
 
type SecretCache struct {
	secret    DBSecret
	expiresAt time.Time
	mu        sync.RWMutex
}
 
type App struct {
	client     *secretsmanager.Client
	secretName string
	cache      *SecretCache
}
 
func NewApp(ctx context.Context, secretName string) *App {
	cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("ap-south-1")) // change region
	if err != nil {
		log.Fatalf("❌ Unable to load AWS SDK config: %v", err)
	}
 
	return &App{
		client:     secretsmanager.NewFromConfig(cfg),
		secretName: secretName,
		cache:      &SecretCache{},
	}
}
 
// fetchSecretFromAWS pulls fresh secret from Secrets Manager
func (a *App) fetchSecretFromAWS(ctx context.Context) (DBSecret, error) {
	secretValue, err := a.client.GetSecretValue(ctx, &secretsmanager.GetSecretValueInput{
		SecretId: &a.secretName,
	})
	if err != nil {
		return DBSecret{}, err
	}
 
	var secret DBSecret
	if err := json.Unmarshal([]byte(*secretValue.SecretString), &secret); err != nil {
		return DBSecret{}, err
	}
 
	log.Println("🔑 [AWS] Fetched new credentials from Secrets Manager")
	return secret, nil
}
 
// getCachedSecret returns cached secret or fetches new if expired
func (a *App) getCachedSecret(ctx context.Context) (DBSecret, error) {
	a.cache.mu.RLock()
	if time.Now().Before(a.cache.expiresAt) {
		secret := a.cache.secret
		a.cache.mu.RUnlock()
		log.Println("📦 [Cache] Using cached credentials")
		return secret, nil
	}
	a.cache.mu.RUnlock()
 
	secret, err := a.fetchSecretFromAWS(ctx)
	if err != nil {
		return DBSecret{}, err
	}
 
	a.cache.mu.Lock()
	a.cache.secret = secret
	a.cache.expiresAt = time.Now().Add(5 * time.Minute)
	a.cache.mu.Unlock()
 
	return secret, nil
}
 
// connectDB tries DB connection and retries with fresh creds if needed
func (a *App) connectDB(ctx context.Context) (*sql.DB, DBSecret, error) {
	secret, err := a.getCachedSecret(ctx)
	if err != nil {
		return nil, DBSecret{}, err
	}
 
	dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s", secret.Username, secret.Password, secret.Host, secret.DBName)
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		return nil, secret, err
	}
 
	if err := db.Ping(); err != nil {
		log.Println("❌ DB connection failed. Retrying with fresh creds...")
 
		secret, err = a.fetchSecretFromAWS(ctx)
		if err != nil {
			return nil, DBSecret{}, err
		}
 
		a.cache.mu.Lock()
		a.cache.secret = secret
		a.cache.expiresAt = time.Now().Add(5 * time.Minute)
		a.cache.mu.Unlock()
 
		dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s", secret.Username, secret.Password, secret.Host, secret.DBName)
		db, err = sql.Open("mysql", dsn)
		if err != nil {
			return nil, DBSecret{}, err
		}
		if err := db.Ping(); err != nil {
			return nil, DBSecret{}, err
		}
 
		log.Println("✅ DB reconnected with fresh creds")
		return db, secret, nil
	}
 
	log.Println("✅ DB connected successfully with cached creds")
	return db, secret, nil
}
 
func (a *App) secretHandler(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()
	db, secret, err := a.connectDB(ctx)
	if err != nil {
		http.Error(w, fmt.Sprintf("DB connection error: %v", err), http.StatusInternalServerError)
		return
	}
	defer db.Close()
 
	var now string
	if err := db.QueryRow("SELECT NOW()").Scan(&now); err != nil {
		http.Error(w, fmt.Sprintf("DB query error: %v", err), http.StatusInternalServerError)
		return
	}
 
	log.Printf("⏱️ DB Time: %s", now)
 
	resp := map[string]string{
		"db_time": now,
		"user":    secret.Username,
		"host":    secret.Host,
	}
	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(resp)
}
 
func main() {
	ctx := context.Background()
	app := NewApp(ctx, "mydb-secret") // Replace with your actual secret name
 
	// Initial warm-up
	_, err := app.fetchSecretFromAWS(ctx)
	if err != nil {
		log.Fatalf("❌ Failed to fetch initial secret: %v", err)
	}
 
	http.HandleFunc("/secret", app.secretHandler)
	log.Println("🚀 Server started at http://localhost:8080")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

🟢 Step 6. Build & Run

go mod init myapp
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/secretsmanager
go get github.com/go-sql-driver/mysql
go build -o app main.go
export AWS_REGION=ap-south-1   # match your region
./app

Expected output:

🔑 [AWS] Fetched new credentials from Secrets Manager
🚀 Server started at http://localhost:8080

🟢 Step 7. Test API

Load Testing

curl http://<EC2_PUBLIC_IP>:8080/secret

Response:

{
  "db_time": "2025-09-01 12:34:56",
  "user": "admin",
  "host": "mydb.abc123.ap-south-1.rds.amazonaws.com"
}

🟢 Step 8. Load Test (1 request/sec)

Option 1: while loop

while true; do
  curl -s http://13.126.172.85:8080/secret
  sleep 1
done

Option 2: watch

watch -n 1 curl -s http://13.126.172.85:8080/secret

🟢 Step 9. Simulate Secret Rotation

  • Go to Secrets Manager → your secret → Rotate secret immediately.
  • App logs will show:
❌ DB connection failed. Retrying with fresh creds...
🔑 [AWS] Fetched new credentials from Secrets Manager
✅ DB reconnected with fresh creds

✅ The API won’t fail even during rotation.


🟢 Step 10. Run as a Service (Production)

sudo nano /etc/systemd/system/goapp.service
[Unit]
Description=Go Secrets Manager Demo
After=network.target
 
[Service]
ExecStart=/home/ec2-user/app
Restart=always
User=ec2-user
WorkingDirectory=/home/ec2-user
 
[Install]
WantedBy=multi-user.target

Enable + start:

sudo systemctl daemon-reload
sudo systemctl enable goapp
sudo systemctl start goapp
journalctl -u goapp -f

✅ Conclusion

  • We deployed a Go web app on EC2.
  • It uses IAM Role auth (no keys in code).
  • Fetches DB creds from Secrets Manager.
  • Caches creds to avoid hitting AWS every request.
  • Retries instantly if creds fail (due to rotation).
  • Survives seamless secret rotation under load.

This is production-style architecture with minimal AWS overhead (no EventBridge, no extra Lambda).


🧙 AI Wizard - Instant Page Insights

Click the button below to analyze this page.
Get an AI-generated summary and key insights in seconds.
Powered by Perplexity AI!