Implementing Search Features

1. Paginating Results

Parameter	Description	Default	Example
from	Starting index of the results	0	`from: 10`
size	Number of results to return	10	`size: 20`

Example: Paginating Search Results

const searchParams = {
  query: {
    match_all: {}
  },
  from: 10,
  size: 20
};

const response = await client.search({
  index: 'my-index',
  body: searchParams
});

Note: The from parameter is used to skip a number of documents in the search results. It's important to note that from does not work well with deep pagination as it requires Elasticsearch to skip over documents, which can be inefficient.

2. Sorting Results

Field	Type	Description	Default
sort	`string`	Sort field to use for sorting results	`_score`
order	`string`	Sort order: `asc` or `desc`	`desc`

Example: Sorting by timestamp descending

{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

Example: Sorting by multiple fields

{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    },
    {
      "message": {
        "order": "asc"
      }
    }
  ]
}

3. Filtering Source Fields

Field	Description	Usage
_source	Contains the original JSON document ingested by Elasticsearch	Used to retrieve full document content in search results
_source.field_name	Accesses a specific field from the original document	Useful for selective field retrieval to reduce response size
_source.*	Wildcard to select multiple fields from the source	Allows flexible field selection without specifying each one

Example: Selecting Specific Fields

{
  "_source": [
    "user.name",
    "message"
  ],
  "query": {
    "match": {
      "message": "error"
    }
  }
}

Example: Using Wildcards for Field Selection

{
  "_source": {
    "includes": ["user.*", "timestamp"]
  },
  "query": {
    "match": {
      "level": "error"
    }
  }
}

Note: Filtering source fields can significantly reduce bandwidth and improve query performance by limiting the amount of data returned in search results.

4. Highlighting Matches

Field	Description	Example
highlight	Enables highlighting of matched terms in search results	`"highlight": {}`
highlight.fields	Defines fields to highlight and their formatting options	`"highlight": {"fields": {"title": {}}}`
highlight.pre_tags	HTML tags to wrap before matched terms	`"highlight": {"pre_tags": ["<mark>"]}`
highlight.post_tags	HTML tags to wrap after matched terms	`"highlight": {"post_tags": ["</mark>"]}`

Example: Basic Highlighting Configuration

{
  "query": {
    "match": {
      "content": "elasticsearch"
    }
  },
  "highlight": {
    "fields": {
      "content": {}
    },
    "pre_tags": ["<mark>"],
    "post_tags": ["</mark>"]
  }
}

Example: Highlighting with Custom Tags

{
  "query": {
    "multi_match": {
      "query": "log analysis",
      "fields": ["message", "tags"]
    }
  },
  "highlight": {
    "fields": {
      "message": {
        "pre_tags": ["<strong class='highlight'>"],
        "post_tags": ["</strong>"]
      },
      "tags": {
        "pre_tags": ["<em>"],
        "post_tags": ["</em>"]
      }
    }
  }
}

Note: Highlighting is only available for analyzed fields. Fields with type: keyword will not be highlighted.

Warning: Highlighting can increase query response time, especially for large documents or many matched terms.

5. Using Search After

The search_after parameter in Elasticsearch is used to efficiently paginate through large result sets when using sorting. It's particularly useful for deep pagination where traditional from and size parameters become inefficient.

Parameter	Description	Values
search_after	Array of values used to define the starting point for the next page of results	`[timestamp, id]`

Example: Using search_after for pagination

{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "timestamp": {
        "order": "asc"
      }
    },
    {
      "_id": {
        "order": "asc"
      }
    }
  ],
  "size": 10
}

Example: Fetching next page with search_after

{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "timestamp": {
        "order": "asc"
      }
    },
    {
      "_id": {
        "order": "asc"
      }
    }
  ],
  "search_after": ["2023-01-01T10:00:00Z", "abc123"],
  "size": 10
}

Note: The search_after array must contain the same number of values as the sort fields, and they must be in the same order.

Warning: Using search_after is more efficient than from + size for deep pagination, but it requires that the sort fields are stable and deterministic.

6. Using Scroll API

Parameter	Description	Default	Required
scroll	Time to keep the search context alive	5m	Yes
scroll_id	ID of the scroll context to retrieve results	None	Yes (for subsequent requests)
size	Number of hits to return per scroll	100	No

Example: Initial Scroll Request

GET /my_index/_search?scroll=10m
{
  "query": {
    "match_all": {}
  },
  "size": 100
}

Example: Subsequent Scroll Requests

GET /_search/scroll
{
  "scroll": "10m",
  "scroll_id": "DXF1ZXJ5QW5kV2VY..."
}

Note: The scroll context is kept alive for the duration specified in the scroll parameter. If no scroll requests are made within this time, the context is discarded.

Warning: Scrolling through large datasets can be resource-intensive. Use with caution on production systems.

7. Using Point in Time

The Point in Time (PIT) feature in Elasticsearch allows you to search across a consistent snapshot of data, ensuring that your search results remain stable even as new data is indexed or documents are deleted.

Parameter	Description	Values
pit.id	Identifier of the point in time	`string`
pit.keep_alive	Duration to keep the PIT open	`time value`
pit.index	Index or indices to create PIT for	`string`

Example: Creating a Point in Time

{
  "index": "my-index",
  "keep_alive": "1m"
}

Example: Searching with Point in Time

{
  "pit": {
    "id": "1234567890abcdef"
  },
  "query": {
    "match_all": {}
  }
}

Note: PITs must be explicitly closed using the pit.delete API to free up resources.

Warning: PITs can consume significant memory if kept open for extended periods or used with large indices.

8. Searching Multiple Indices

Option	Values	Notes
indices	`index1,index2,index3`	Comma-separated list of index names to search
q	`query_string`	Query string to search across indices
body	`{ "query": { "multi_match": { "query": "search term", "type": "best_fields" } } }`	JSON body with multi-index query

Example: Search Across Multiple Indices

GET /index1,index2,_search
{
  "query": {
    "multi_match": {
      "query": "elasticsearch",
      "type": "best_fields"
    }
  }
}

Note: Ensure indices exist and are accessible before querying.

9. Using Search Templates

Property	Description	Example
template_id	Unique identifier for the search template	`"my_template"`
source	Search template content in JSON format	`{ "query": { "match": { "message": "{{query_string}}" } } }`

Example: Creating a Search Template

{
  "id": "error_logs_template",
  "template": {
    "query": {
      "bool": {
        "must": [
          {
            "match": {
              "log_level": "ERROR"
            }
          },
          {
            "range": {
              "timestamp": {
                "gte": "{{start_time}}",
                "lte": "{{end_time}}"
              }
            }
          }
        ]
      }
    }
  }
}

Example: Using a Search Template

GET /_scripts/search/error_logs_template
{
  "params": {
    "start_time": "2023-10-01T00:00:00Z",
    "end_time": "2023-10-02T00:00:00Z"
  }
}

Note: Search templates are stored in Elasticsearch's _scripts endpoint and can be reused across queries.

10. Implementing Autocomplete

Feature	Description	Implementation	Notes
field	Field to enable autocomplete on	`field: 'user.name'`	Must be a keyword field
size	Number of suggestions to return	`size: 10`	Default is 5
fuzzy	Enable fuzzy matching	`fuzzy: true`	Improves matching for typos

Example: Autocomplete Query

{
  "suggest": {
    "user_suggest": {
      "text": "john",
      "completion": {
        "field": "user.name",
        "size": 10,
        "fuzzy": true
      }
    }
  }
}

Note: The user.name field must be mapped as a completion type in the index mapping.

Warning: Autocomplete performance degrades with large datasets unless properly indexed.

11. Using Suggesters

Suggesters in the ELK Stack provide auto-complete and search assistance features, enhancing user experience by suggesting terms based on indexed data.

Property	Description	Values	Notes
type	Defines the suggester type	`suggest`, `completion`, `phrase`	Most common is `suggest` for basic term suggestions
field	Field to suggest terms from	`string`	Mandatory for `suggest` type
size	Number of suggestions to return	`integer`	Default is 5
analyzer	Analyzer to use for tokenization	`string`	Optional, uses field's analyzer if not specified

Example: Basic Suggester Query

{
  "suggest": {
    "my-suggestion": {
      "text": "elasti",
      "suggester": {
        "field": "name",
        "size": 3
      }
    }
  }
}

Note: Suggesters are particularly useful in search UIs where users benefit from real-time term suggestions.

12. Configuring Request Cache

Property	Description	Default Value	Notes
indices.cache.request.enable	Enables or disables the request cache for indices	`true`	Set to `false` to disable caching
indices.cache.request.size	Size of the request cache in memory	`10%`	Can be specified in bytes (e.g., `1gb`) or percentage of heap
indices.cache.request.expire	Expiration time for cached requests	`1h`	Set to `-1` for no expiration

Example: Setting Request Cache Size

{
  "indices": {
    "cache": {
      "request": {
        "size": "2gb",
        "expire": "30m"
      }
    }
  }
}

Note: The request cache is particularly useful for queries that are frequently repeated, such as dashboard views or common search terms.