Implementing Advanced Queries

1. Using Nested Query

Property	Description	Example
nested	Defines a nested object or array for querying	`{ "nested": { "path": "comments", "query": { "match": { "comments.text": "great" } } } }`
query	Inner query to execute on the nested objects	`{ "nested": { "path": "tags", "query": { "terms": { "tags.name": ["urgent", "important"] } } } }`

Example: Nested Query for Comments

{
  "query": {
    "nested": {
      "path": "comments",
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "comments.author": "john"
              }
            },
            {
              "range": {
                "comments.timestamp": {
                  "gte": "2023-01-01"
                }
              }
            }
          ]
        }
      }
    }
  }
}

2. Using Has Child Query

The has_child query in the ELK Stack is used to find documents that have child documents matching a specified query. It's particularly useful in nested or parent-child relationships within Elasticsearch.

Field	Description	Example
has_child	Query to find parent documents based on child document matches	`{ "has_child": { "type": "comment", "query": { "match": { "text": "great" } } } }`

Example: Find parent documents with matching child documents

{
  "query": {
    "has_child": {
      "type": "comment",
      "query": {
        "match": {
          "text": "great"
        }
      },
      "score_mode": "max"
    }
  }
}

Note: The has_child query requires that the parent-child relationship is properly configured in the index mapping.

Key Points

type: Specifies the child document type to match against.
query: The query to run on child documents.
score_mode: Determines how scores from child documents are aggregated to the parent document.

3. Using Has Parent Query

The has_parent query in the ELK Stack is used to find documents that have a parent document matching a specified query. It's particularly useful in nested or joined document structures where you want to filter based on parent document properties.

Property	Description	Example
parent	The type of the parent document to match against.	`"parent": "user"`
query	The query to run on the parent documents.	`"query": { "match": { "name": "John" } }`
score_mode	How to score the results. Options: `none`, `max`, `sum`, `avg`.	`"score_mode": "max"`

Example: Find child documents with matching parent

{
  "query": {
    "has_parent": {
      "parent": "user",
      "query": {
        "match": {
          "name": "John"
        }
      },
      "score_mode": "max"
    }
  }
}

Note: The has_parent query requires that the parent-child relationship is properly configured in the index mapping.

4. Using Query String

Property	Description	Example
query_string	Used to parse and execute query strings in Elasticsearch	`"query_string": { "query": "field:value" }`
default_field	Specifies the default field to search if no field is specified in the query	`"default_field": "message"`
analyze_wildcard	Allows wildcard queries to be analyzed using the analyzer of the field	`"analyze_wildcard": true`
allow_leading_wildcard	Controls whether leading wildcards are allowed in query strings	`"allow_leading_wildcard": false`

Example: Basic Query String Usage

{
  "query": {
    "query_string": {
      "query": "elasticsearch AND logstash",
      "default_field": "message"
    }
  }
}

Example: Query String with Wildcards

{
  "query": {
    "query_string": {
      "query": "field:*value*",
      "analyze_wildcard": true
    }
  }
}

Note: Query strings support a wide range of operators like AND, OR, NOT, and parentheses for grouping.

Warning: Using wildcards at the beginning of a term can significantly impact query performance.

5. Using Simple Query String

The simple_query_string query is a simplified version of the standard query string that provides a more user-friendly way to construct queries. It's particularly useful for search interfaces where users might not be familiar with the full query syntax.

Field	Description	Example
query	The query string to parse and execute.	`"+elasticsearch +logstash"`
default_field	Field to search if no field is specified in the query.	`"message"`
fields	Specific fields to search in, can include wildcards.	`"message", "tags.*"`
analyzer	Analyzer to use for parsing the query string.	`"standard"`
operator	Operator to use when joining clauses (AND or OR).	`"OR"`
minimum_should_match	Minimum number of clauses that must match.	`"2<75%"`

Example: Basic Simple Query String Usage

{
  "query": {
    "simple_query_string": {
      "query": "elasticsearch logstash",
      "default_field": "message"
    }
  }
}

Example: Advanced Simple Query String with Fields

{
  "query": {
    "simple_query_string": {
      "query": "+elasticsearch +logstash",
      "fields": ["message", "tags.*"],
      "analyzer": "standard"
    }
  }
}

Note: The simple_query_string query does not support complex features like phrase queries or fuzzy matching. It is designed for simplicity and ease of use, especially in UIs where users might not understand full query syntax.

6. Using More Like This Query

The more_like_this query in Elasticsearch allows you to find documents that are similar to a given document based on the content of specified fields. It's particularly useful for recommendation systems or finding related content.

Field	Description	Example
more_like_this	Object containing configuration for the query	`{ "like": "This is a sample document.", "fields": ["title", "content"] }`

Example: Basic More Like This Query

{
  "query": {
    "more_like_this": {
      "like": "This is a sample document.",
      "fields": ["title", "content"],
      "min_term_freq": 1,
      "max_query_terms": 12
    }
  }
}

Example: More Like This with Document ID

{
  "query": {
    "more_like_this": {
      "like": {
        "_id": "12345"
      },
      "fields": ["title", "content"]
    }
  }
}

Note: The more_like_this query works by analyzing the input text and comparing it to indexed content in the specified fields. It uses term frequency and inverse document frequency (TF-IDF) to calculate similarity.

7. Using Percolate Query

Property	Description	Example
percolate	Used to match documents against a set of registered queries	`{ "query": { "percolate": { "field": "query", "document": { "message": "this is a test" } } } }`

Example: Registering a Query

{
  "query": {
    "match": {
      "message": "test"
    }
  }
}

Example: Percolating a Document

{
  "query": {
    "percolate": {
      "field": "query",
      "document": {
        "message": "this is a test"
      }
    }
  }
}

Note: The percolate query is useful for matching documents against a set of registered queries without storing the full document content in the index.

8. Using Script Query

Property	Description	Example
script	Defines the script to be executed. Can use Painless scripting language.	`{ "script": { "source": "doc['field'].value * 2" } }`
lang	Specifies the scripting language. Default is 'painless'.	`"lang": "painless"`
params	Parameters to pass into the script.	`"params": { "multiplier": 2 }`

Example: Using Script Query to Multiply Field Values

{
  "query": {
    "script": {
      "source": "doc['price'].value * params.multiplier",
      "params": {
        "multiplier": 1.1
      }
    }
  }
}

Note: Script queries can be resource-intensive. Use with caution in production environments.

9. Using Span Queries

Span queries in the ELK Stack are used to match documents based on the proximity of terms within a field. They are particularly useful for finding phrases or terms that appear near each other.

Query Type	Description	Use Case	Example
span_term	Matches documents containing a specific term within a span field.	Searching for a specific keyword in a document field.	`{ "span_term": { "field_name": "keyword" } }`
span_multi	Matches documents using a multi-term query inside a span context.	Using wildcard or regexp queries within a span field.	`{ "span_multi": { "match": { "field_name": { "wildcard": "keyword" } } } }`
span_near	Matches documents where terms appear within a specified distance.	Finding phrases or terms that are near each other in the document.	`{ "span_near": { "clauses": [ { "span_term": { "field_name": "term1" } }, { "span_term": { "field_name": "term2" } } ], "slop": 5, "in_order": true } }`
span_or	Matches documents containing any of the specified span queries.	Searching for multiple terms in a field with OR logic.	`{ "span_or": { "clauses": [ { "span_term": { "field_name": "term1" } }, { "span_term": { "field_name": "term2" } } ] } }`
span_not	Matches documents containing a span query but not another.	Excluding specific terms from search results.	`{ "span_not": { "include": { "span_term": { "field_name": "term1" } }, "exclude": { "span_term": { "field_name": "term2" } } } }`

Example: Using span_near to find phrases

{
  "query": {
    "span_near": {
      "clauses": [
        { "span_term": { "content": "quick" } },
        { "span_term": { "content": "brown" } }
      ],
      "slop": 3,
      "in_order": true
    }
  }
}

Note: The slop parameter defines how many positions the terms can be apart. A value of 0 means they must be adjacent.

10. Using Geo Queries

Property	Value	Description
geo_point	`{"lat": 40.7128, "lon": -74.0060}`	Defines a geographical point with latitude and longitude
geo_distance	`{"distance": "10km", "lat": 40.7128, "lon": -74.0060}`	Calculates distance between two geographical points
geo_bounding_box	`{"top_left": {"lat": 41.0, "lon": -75.0}, "bottom_right": {"lat": 40.0, "lon": -73.0}}`	Defines a bounding box for geographical search
geo_polygon	`{"points": [{"lat": 40.0, "lon": -75.0}, {"lat": 41.0, "lon": -75.0}, {"lat": 41.0, "lon": -73.0}]}`	Defines a polygon for geographical search

Example: Geo Distance Query

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "status": "active"
          }
        }
      ],
      "filter": {
        "geo_distance": {
          "distance": "50km",
          "location": {
            "lat": 40.7128,
            "lon": -74.0060
          }
        }
      }
    }
  }
}

Example: Bounding Box Query

{
  "query": {
    "geo_bounding_box": {
      "location": {
        "top_left": {
          "lat": 41.0,
          "lon": -75.0
        },
        "bottom_right": {
          "lat": 40.0,
          "lon": -73.0
        }
      }
    }
  }
}

Example: Polygon Query

{
  "query": {
    "geo_polygon": {
      "location": {
        "points": [
          {
            "lat": 40.0,
            "lon": -75.0
          },
          {
            "lat": 41.0,
            "lon": -75.0
          },
          {
            "lat": 41.0,
            "lon": -73.0
          },
          {
            "lat": 40.0,
            "lon": -73.0
          }
        ]
      }
    }
  }
}

11. Using Shape Queries

Shape queries in the ELK Stack are used to define and query geospatial data, particularly when working with geographic features like polygons, lines, and points. These queries are often used in conjunction with the geo_shape data type in Elasticsearch.

Query Type Description Usage Example

geo_shape

Query Type	Description	Usage Example
geo_shape	Matches documents containing a `geo_shape` field that intersects with the specified shape.	`{ "query": { "geo_shape": { "location": { "shape": { "type": "polygon", "coordinates": [ [ [10, 10], [20, 20], [30, 10], [10, 10] ] ] }, "relation": "within" } } } }`
geo_bounding_box	Matches documents where the `geo_shape` field is within a bounding box.	`{ "query": { "geo_bounding_box": { "location": { "top_left": { "lat": 40.73, "lon": -74.1 }, "bottom_right": { "lat": 40.01, "lon": -71.12 } } } } }`
geo_distance	Matches documents where the `geo_shape` field is within a specified distance from a point.	`{ "query": { "geo_distance": { "distance": "10km", "location": { "lat": 40.73, "lon": -74.1 } } } }`

Matches documents containing a geo_shape field that intersects with the specified shape.

{
  "query": {
    "geo_shape": {
      "location": {
        "shape": {
          "type": "polygon",
          "coordinates": [
            [
              [10, 10],
              [20, 20],
              [30, 10],
              [10, 10]
            ]
          ]
        },
        "relation": "within"
      }
    }
  }
}

geo_bounding_box

Matches documents where the geo_shape field is within a bounding box.

{
  "query": {
    "geo_bounding_box": {
      "location": {
        "top_left": {
          "lat": 40.73,
          "lon": -74.1
        },
        "bottom_right": {
          "lat": 40.01,
          "lon": -71.12
        }
      }
    }
  }
}

geo_distance

Matches documents where the geo_shape field is within a specified distance from a point.

{
  "query": {
    "geo_distance": {
      "distance": "10km",
      "location": {
        "lat": 40.73,
        "lon": -74.1
      }
    }
  }
}

Example: Using geo_shape with relation

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "type": "restaurant"
          }
        }
      ],
      "filter": [
        {
          "geo_shape": {
            "location": {
              "shape": {
                "type": "polygon",
                "coordinates": [
                  [
                    [10, 10],
                    [20, 20],
                    [30, 10],
                    [10, 10]
                  ]
                ]
              },
              "relation": "within"
            }
          }
        }
      ]
    }
  }
}

Note: The geo_shape query supports various shape types including point, line_string, polygon, and multi_polygon. The relation parameter defines how the shapes interact (e.g., within, intersects, disjoint).