OpenSearch Index State Management (ISM) Guide

1. Overview

This document outlines the standard procedure for managing index lifecycles within the OpenSearch cluster. The objective is to automate data retention by backing up indices to S3 and deleting them once they exceed a 90-day threshold.

2. Prerequisites

Before implementing the policy, ensure the S3 Snapshot Repository is correctly registered.

Verify Snapshot Repository

Run this command to ensure the cluster can communicate with the S3 bucket:

GET _snapshot/opensearch-index-backup-org-name-co-in

Target Bucket: opensearch-index-backup-org-name-co-in
Region: ap-south-1
Role ARN: arn:aws:iam::372360814385:role/PartnerOpenSearch-Role

3. Implementation Steps

Step 1: Create/Update the ISM Policy

This policy defines the three states: Hot (Active), Snapshot (Backup), and Delete (Cleanup).

Critical Production Rule: Snapshot names must be lowercase. Using uppercase letters (e.g., 90DEL) will trigger an illegal_argument_exception and fail the process.

PUT _plugins/_ism/policies/DEL_90D_APP_Index
{
  "policy": {
    "description": "Snapshot and delete indices after 90 days",
    "error_notification": {
      "channel": { "id": "sJtZsZsBIe41CKNEND83" },
      "message_template": {
        "source": "OpenSearch Notification : 90DEL ISM Policy Failure",
        "lang": "mustache"
      }
    },
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [],
        "transitions": [
          {
            "state_name": "snapshot_state",
            "conditions": { "min_index_age": "90d" }
          }
        ]
      },
      {
        "name": "snapshot_state",
        "actions": [
          {
            "retry": { "count": 3, "backoff": "exponential", "delay": "1m" },
            "snapshot": {
              "repository": "opensearch-index-backup-org-name-co-in",
              "snapshot": "autosnap-90del-{{ctx.index}}-{{ctx.creation_date}}"
            }
          }
        ],
        "transitions": [
          {
            "state_name": "delete_state",
            "conditions": { "min_index_age": "90d" }
          }
        ]
      },
      {
        "name": "delete_state",
        "actions": [
          { "retry": { "count": 3, "backoff": "exponential", "delay": "1m" },
            "delete": {} }
        ],
        "transitions": []
      }
    ],
    "ism_template": [
      {
        "index_patterns": ["transactions-*", "settlements-*", "fraudulents-*", "chargebacks-*"],
        "priority": 100
      }
    ]
  }
}

Step 2: Apply Policy to Existing Indices

For indices already present in the cluster, the policy must be attached manually to begin management.

POST _plugins/_ism/add/transactions-*,settlements-*,fraudulents-*,chargebacks-*
{
  "policy_id": "DEL_90D_APP_Index"
}

4. Verification & Status Commands (The “Get” Commands)

Use these commands to verify the setup is working as expected.

A. Check if Policy is Attached correctly

Verify that indices are “Enabled” and linked to the correct policy_id.

GET _plugins/_ism/explain/transactions-*,settlements-*?size=50

B. Verify Snapshot Creation in S3

Check the repository to see the actual files being created. Look for the autosnap-90del- prefix.

GET _snapshot/opensearch-index-backup-org-name-co-in/_all

C. Check Current Active Snapshot

If a backup is currently running, this will show the percentage completion.

GET _snapshot/opensearch-index-backup-org-name-co-in/_current

5. Debugging & Troubleshooting

Problem: Index is stuck in “Hot” even if > 90 days

Solution: The ISM runner may need a “kickstart” or the index may have failed metadata.

POST _plugins/_ism/retry/transactions-*,settlements-*,fraudulents-*,chargebacks-*

Problem: “Index has no metadata information”

Solution: This happens if the index was removed from ISM but not re-added. Run the add command from Step 2 again.

Problem: “Version Conflict” when updating policy

Solution: OpenSearch requires you to delete the old policy first if you are making major changes.

DELETE _plugins/_ism/policies/DEL_90D_APP_Index

Problem: Snapshot failed (Lowercase Error)

Solution: Check the explain output for a cause field. Ensure the snapshot name in the policy is strictly lowercase.

6. Maintenance Summary Table

Task	Command
Check Health	`GET _plugins/_ism/explain/<index_name>`
Force Retry	`POST _plugins/_ism/retry/<index_pattern>`
Swap Policy	`POST _plugins/_ism/change_policy/<index_pattern> { "policy_id": "..." }`
Stop ISM	`POST _plugins/_ism/remove/<index_pattern>`

EFK Stack

DataDog

Grafana Stack

Jaeger

Loki Stack

OpenSearch

OSS ISM

OpenSearch Index State Management (ISM) Guide

1. Overview

2. Prerequisites

Verify Snapshot Repository

3. Implementation Steps

Step 1: Create/Update the ISM Policy

Step 2: Apply Policy to Existing Indices

4. Verification & Status Commands (The “Get” Commands)

A. Check if Policy is Attached correctly

B. Verify Snapshot Creation in S3

C. Check Current Active Snapshot

5. Debugging & Troubleshooting

Problem: Index is stuck in “Hot” even if > 90 days

Problem: “Index has no metadata information”

Problem: “Version Conflict” when updating policy

Problem: Snapshot failed (Lowercase Error)

6. Maintenance Summary Table

​OpenSearch Index State Management (ISM) Guide

​1. Overview

​2. Prerequisites

​Verify Snapshot Repository

​3. Implementation Steps

​Step 1: Create/Update the ISM Policy

​Step 2: Apply Policy to Existing Indices

​4. Verification & Status Commands (The “Get” Commands)

​A. Check if Policy is Attached correctly

​B. Verify Snapshot Creation in S3

​C. Check Current Active Snapshot

​5. Debugging & Troubleshooting

​Problem: Index is stuck in “Hot” even if > 90 days

​Problem: “Index has no metadata information”

​Problem: “Version Conflict” when updating policy

​Problem: Snapshot failed (Lowercase Error)

​6. Maintenance Summary Table

OpenSearch Index State Management (ISM) Guide

1. Overview

2. Prerequisites

Verify Snapshot Repository

3. Implementation Steps

Step 1: Create/Update the ISM Policy

Step 2: Apply Policy to Existing Indices

4. Verification & Status Commands (The “Get” Commands)

A. Check if Policy is Attached correctly

B. Verify Snapshot Creation in S3

C. Check Current Active Snapshot

5. Debugging & Troubleshooting

Problem: Index is stuck in “Hot” even if > 90 days

Problem: “Index has no metadata information”

Problem: “Version Conflict” when updating policy

Problem: Snapshot failed (Lowercase Error)

6. Maintenance Summary Table