> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ahmadraza.in/llms.txt
> Use this file to discover all available pages before exploring further.

# ISM

# Snapshot Lifecycle Management (SLM)

## 🏗️ Phase 1: AWS Infrastructure (S3 & IAM)

### 1. Create the S3 Bucket

1. Log in to the **AWS S3 Console**.
2. Click **Create bucket**.
3. **Bucket name:** `aws-s3-snap-backup` (or your preferred name).
4. **Region:** Choose the same region where your EC2/Elasticsearch nodes are running (to save on data transfer costs).
5. Leave other settings as default and click **Create bucket**.

### 2. Create the IAM Policy

1. Go to the **IAM Console** > **Policies** > **Create policy**.
2. Switch to the **JSON** tab and paste the following (replace `aws-s3-snap-backup` with your bucket name):

```json theme={null}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": ["arn:aws:s3:::aws-s3-snap-backup"]
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": ["arn:aws:s3:::aws-s3-snap-backup/*"]
        }
    ]
}

```

3. Name it `ElasticsearchS3BackupPolicy` and save.

### 3. Create & Attach the IAM Role

1. In the **IAM Console**, go to **Roles** > **Create role**.
2. Select **AWS Service** and choose **EC2**.
3. Attach the `ElasticsearchS3BackupPolicy` you just created.
4. Name the role `Elasticsearch-S3-Role`.
5. **Attach to EC2:** Go to your EC2 instance list > Select your Elasticsearch nodes > **Actions** > **Security** > **Modify IAM Role**. Select `Elasticsearch-S3-Role` and save.

***

## 🛠️ Phase 2: Elasticsearch Node Configuration

> \[!IMPORTANT]
> You must perform these steps on **EVERY** node in your cluster that has the `master` or `data` role.

### 1. Install the S3 Plugin

Run this from your terminal on each node:

```bash theme={null}
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3

```

*Note: You must restart Elasticsearch on each node after installation.*

### 2. Configure the Keystore (Optional but Recommended)

If you aren't using the IAM Role, you’d add your AWS keys here. Since we are using an **IAM Role**, Elasticsearch will automatically find the credentials. You can skip this unless your security team requires manual keys.

***

## 🚀 Phase 3: Elasticsearch API Setup

### 1. Register the Repository

Now, tell Elasticsearch where to find the bucket. Run this in **Kibana Dev Tools**:

```http theme={null}
PUT _snapshot/aws-s3-snap-backup
{
  "type": "s3",
  "settings": {
    "bucket": "aws-s3-snap-backup",
    "region": "us-east-1" 
  }
}

```

### 2. Create the SLM Policy (500-Day Retention)

This automates the daily backup at 11:30 PM.

```http theme={null}
PUT _slm/policy/dailysnap7d
{
  "name": "<daily-snap-{now{yyyy.MM.dd-HH.mm.ss}}>",
  "schedule": "0 30 23 * * ?",
  "repository": "aws-s3-snap-backup",
  "config": {
    "indices": "*",
    "include_global_state": true,
    "partial": true
  },
  "retention": {
    "expire_after": "500d",
    "min_count": 1,
    "max_count": 500
  }
}

```

***

## ⏪ Phase 4: How to Restore (The "Emergency" Guide)

If ILM deletes a log and you need it back, follow these steps:

### 1. Find the Snapshot Name

```http theme={null}
GET _snapshot/aws-s3-snap-backup/_all

```

### 2. Restore a Specific Index

If you want to restore `demo-app-logs-2026.02.10` from the snapshot:

```http theme={null}
POST _snapshot/aws-s3-snap-backup/daily-snap-2026.02.10-23.30.00/_restore
{
  "indices": "demo-app-logs-2026.02.10",
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1"
}

```

*Tip: Restoring with a `restored-` prefix prevents conflicts with existing indices.*

***

## ✅ Summary of the Whole System

* **Fluent Bit:** Sends logs daily (e.g., `demo-app-logs-2026.03.05`).
* **ILM:** Deletes indices from the **disk** after **15 days**.
* **SLM:** Copies indices to **S3** every night and keeps them for **500 days**.
* **Storage:** You save money by keeping only 15 days on fast disks while keeping the rest on cheap S3 storage.