AWS S3 File Upload Script Using AWS Secret Manager

This Bash script is designed to upload files to an S3 bucket using temporary AWS credentials retrieved from AWS Secrets Manager. Below is a detailed explanation of each section of the script.

Script Overview

#!/bin/bash
# Set variables
SECRET_NAME="s3-full-access-keys"
REGION="ap-south-1"
S3_BUCKET="name-uat-test"
LOCAL_FILE_PATH="s3-upload"
RCLONE_CONFIG_FILE="rclone.conf"

SECRET_NAME: The name of the secret in AWS Secrets Manager containing AWS credentials.
REGION: The AWS region where the secret is stored and the S3 bucket resides.
S3_BUCKET: The name of the S3 bucket to which files will be uploaded.
LOCAL_FILE_PATH: The path to the local file to be uploaded.
RCLONE_CONFIG_FILE: The configuration file for rclone (not used in this script but defined for reference).

Function to Get Secret Value

# Function to get secret value from AWS Secrets Manager
get_secret() {
  aws secretsmanager get-secret-value --secret-id $SECRET_NAME --region $REGION --query 'SecretString' --output text
}

This function retrieves the secret value from AWS Secrets Manager. It uses the AWS CLI command aws secretsmanager get-secret-value to fetch the secret.

Extracting AWS Credentials

# Extract AWS credentials from the secret
SECRET=$(get_secret)
ACCESS_KEY=$(echo $SECRET | jq -r '.ACCESS_KEY')
SECRET_KEY=$(echo $SECRET | jq -r '.SECRET_KEY')

SECRET: The JSON string returned from AWS Secrets Manager containing the AWS credentials.
ACCESS_KEY: The AWS access key extracted from the secret.
SECRET_KEY: The AWS secret key extracted from the secret.

Exporting AWS Credentials and Uploading File

# Only needed for temporary credentials
export AWS_ACCESS_KEY_ID=$ACCESS_KEY
export AWS_SECRET_ACCESS_KEY=$SECRET_KEY
export AWS_REGION=$REGION
 
aws s3 mv /path/to/file s3://pt-glacier/logs/

AWS_ACCESS_KEY_ID: The environment variable for the AWS access key.
AWS_SECRET_ACCESS_KEY: The environment variable for the AWS secret key.
AWS_REGION: The environment variable for the AWS region.
aws s3 mv: The AWS CLI command to move (upload) a file to the specified S3 bucket.

Replace /path/to/file with the path of the local file you want to upload.

Note

Ensure you have the AWS CLI and jq installed and properly configured on your system to run this script successfully.

Git Basic Bash Script Learn