Skip to main content
Mapagam

Working with Indices

1. Creating Indices

Property Description Default Notes
index.name Name of the index to be created None Must be lowercase and cannot contain spaces or special characters
index.settings Settings for the index such as number of shards and replicas None Can include settings like number_of_shards and number_of_replicas
index.mappings Mapping definitions for fields in the index None Defines how documents and their fields are stored and indexed

Example: Creating an Index with Settings

{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age": {
        "type": "integer"
      }
    }
  }
}

Example: Creating an Index Using cURL

curl -X PUT "localhost:9200/my_index" -H "Content-Type: application/json" -d '{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age": {
        "type": "integer"
      }
    }
  }
}'

2. Listing All Indices

Command Description Example
_cat/indices List all indices with basic info GET /_cat/indices?v
_cat/indices?v&h=index,health,status List specific fields for all indices GET /_cat/indices?v&h=index,health,status
_cat/indices?format=json Return indices in JSON format GET /_cat/indices?format=json
Note: The _cat API is a convenient way to view cluster information in a tabular format. It's primarily used for monitoring and debugging.

3. Viewing Index Settings

Setting Description Default Value Notes
index.number_of_shards Number of primary shards for the index 1 Cannot be changed after index creation
index.number_of_replicas Number of replica shards for the index 1 Can be adjusted after creation
index.refresh_interval How often the index is refreshed 1s Controls search latency and performance
index.store.type Storage type for the index fs Options: fs, hdfs, s3
index.analysis Analysis settings for the index None Custom analyzers can be defined here

Example: Retrieve Index Settings

GET /my_index/_settings

{
  "my_index": {
    "settings": {
      "index": {
        "number_of_shards": "1",
        "number_of_replicas": "1",
        "refresh_interval": "1s"
      }
    }
  }
}

4. Checking Index Stats

Stat Description Command Example
docs.count Number of documents in the index GET /{index}/_stats 
{
  "count": 12345
}
docs.deleted Number of deleted documents in the index GET /{index}/_stats 
{
  "deleted": 123
}
store.size Total size of the index on disk GET /{index}/_stats 
{
  "size_in_bytes": 123456789
}
segments.count Number of segments in the index GET /{index}/_stats 
{
  "count": 5
}

Example: Retrieve Index Stats

GET /my-index/_stats

Example: Retrieve Index Stats with Specific Metrics

GET /my-index/_stats/docs,store

5. Opening and Closing Indices

Action Description API Endpoint Example
Open Index Reopens a closed index, making it available for read and write operations. POST /{index}/_open PUT /my_index/_open
Close Index Closes an index, freeing up resources and making it read-only. POST /{index}/_close PUT /my_index/_close
Note: Closing an index makes it read-only and releases its resources. It can be reopened later using the _open endpoint.
Warning: Closing an index is not reversible without restoring from a snapshot. Ensure backups are in place before closing.

6. Deleting Indices

Option Values Notes
indices.delete DELETE /{index} Delete a specific index by name
indices.delete DELETE /{index1},{index2} Delete multiple indices by name
indices.delete DELETE /* Delete all indices (use with caution)

Example: Delete a Single Index

curl -X DELETE "http://localhost:9200/logs-2024-01-01"

Example: Delete Multiple Indices

curl -X DELETE "http://localhost:9200/logs-2024-01-01,logs-2024-01-02"
Warning: Deleting indices is irreversible. Ensure you have backups or are certain about the deletion before proceeding.

7. Setting Index Aliases

Property Description Example
alias.name Name of the alias to be created or updated logs-2024
index.pattern Pattern to match indices for aliasing logs-*
alias.filter Query to filter documents in the alias 
{
  "term": {
    "status": "active"
  }
}

Example: Create an Index Alias

{
  "actions": [
    {
      "add": {
        "index": "logs-2024.01",
        "alias": "logs-latest"
      }
    }
  ]
}

Example: Add Filtered Alias

{
  "actions": [
    {
      "add": {
        "index": "logs-*",
        "alias": "active-logs",
        "filter": {
          "term": {
            "status": "active"
          }
        }
      }
    }
  ]
}
Note: Aliases are not stored in the index itself but managed by Elasticsearch.

8. Forcing Index Refresh

Option Values Description
/index/_refresh POST Forces a refresh on the specified index or indices
refresh true | false | wait_for Controls when the operation returns: false (immediate), true (wait for refresh), wait_for (wait for refresh and return after it completes)

Example: Refresh a Single Index

POST /my_index/_refresh

Example: Refresh Multiple Indices

POST /index1,index2/_refresh
Note: Refreshing indices makes recent changes visible to search queries, but it's an expensive operation and should be used sparingly.

9. Flushing Indices

Note: Flushing indices in the ELK Stack is typically handled automatically by Elasticsearch. Manual flushing should be used sparingly and only when necessary for specific use cases.
Property Description Default Value Notes
indices.flush Controls whether to flush indices false Manual flush operation
indices.flush.sync Whether to wait for flush completion false Sync vs async flush

Example: Flushing an Index

PUT /my_index/_flush

Example: Flushing All Indices

POST /_flush

Example: Sync Flush

POST /my_index/_flush?sync=true

Manual Flush Workflow

  1. Identify the index or indices to flush
  2. Determine if a sync or async flush is required
  3. Execute the flush API call
  4. Verify that the flush was successful

When to Flush Indices

  • After large bulk index operations
  • Before taking snapshots
  • When optimizing for search performance
  • During maintenance windows

10. Reindexing Data

Example: Basic Reindexing

{
  "source": {
    "index": "old-logs-*"
  },
  "dest": {
    "index": "new-logs-*"
  }
}
Field Description Values
source.index Source index or pattern to reindex from old-logs-*
dest.index Destination index or pattern to reindex to new-logs-*
conflicts How to handle document conflicts proceed, abort
script Script to transform documents during reindexing { "source": "ctx._source.new_field = 'value'" }
Note: Reindexing is a synchronous operation that can be resource-intensive. It's recommended to run reindexing jobs during low-traffic periods.

Example: Reindexing with Script Transformation

{
  "source": {
    "index": "logs-2023-*"
  },
  "dest": {
    "index": "logs-2023-transformed"
  },
  "script": {
    "source": "ctx._source.timestamp = ctx._source.timestamp + 'Z'; ctx._source.level = ctx._source.level.toUpperCase()"
  }
}

Example: Reindexing with Conflicts Handling

{
  "source": {
    "index": "logs-*"
  },
  "dest": {
    "index": "reindexed-logs-*"
  },
  "conflicts": "proceed"
}

11. Cloning Indices

Property Description Values
source.index Source index to clone from string
target.index Name of the new cloned index string
settings Optional settings for the new index object

Example: Clone an Index

{
  "source": {
    "index": "my-source-index"
  },
  "target": {
    "index": "my-cloned-index"
  }
}
Note: The source index must exist and be closed to perform a clone operation.
Warning: Cloning does not copy data, it creates a new index with the same settings and mappings.

Was this article helpful?

Related Cheat Sheets