Optimizing Kibana Performance

1. Configuring Query Time Range

Option	Values	Notes
time.from	`now-15m`	Start time for the query range
time.to	`now`	End time for the query range
time.mode	`quick`	Time range mode: quick, absolute, or relative

Example: Setting Time Range in Kibana Dashboard

{
  "time": {
    "from": "now-1h",
    "to": "now"
  }
}

Note: Using now-1h sets the query to look at the last hour of data.

Warning: Setting a large time range can impact query performance and memory usage.

2. Using Filters vs Queries

Aspect	Filters	Queries
Execution	Applied to the `_all` field and uses the filter cache	Executed on all fields and does not use filter cache
Caching	Uses the filter cache for performance improvement	Does not use filter cache; results are not cached
Use Case	Best for exact matches, ranges, and boolean operations	Best for full-text search and complex field analysis
Performance	Highly efficient for repeated searches	Slower due to lack of caching and full-text analysis
Field Usage	Works on fields with `doc_values` enabled	Works on all fields including analyzed text fields

Example: Using Filters in Kibana

{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "status": "active"
          }
        },
        {
          "range": {
            "timestamp": {
              "gte": "2023-01-01T00:00:00Z"
            }
          }
        }
      ]
    }
  }
}

Example: Using Queries in Kibana

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "message": "error"
          }
        }
      ]
    }
  }
}

3. Optimizing Visualizations

Property	Description	Recommendation
visualization.type	Defines the type of visualization (e.g., line chart, bar chart, heatmap)	Use simple visualizations for better performance. Complex visualizations like heatmaps or 3D charts can slow down rendering.
timeframe	Time range used in the visualization	Limit the date range to reduce data load. Use filters or aggregations to narrow down the scope.
aggregation	How data is grouped or aggregated in the visualization	Use efficient aggregations like terms, date_histogram, and filters. Avoid complex scripted aggregations.
data.size	Amount of data being visualized	Limit the number of data points. Use sampling or downsampling techniques for large datasets.

Example: Simple Line Chart Visualization

{
  "title": "CPU Usage Over Time",
  "type": "line",
  "params": {
    "timeFieldName": "@timestamp",
    "interval": "1m"
  },
  "aggs": [
    {
      "id": "cpu_usage",
      "enabled": true,
      "type": "avg",
      "schema": "metric",
      "params": {
        "field": "cpu.percent"
      }
    }
  ]
}

Note: For better performance, avoid using date_histogram with a very small interval (e.g., less than 1 minute) on large datasets.

Warning: Complex visualizations with many data points or real-time aggregations can cause Kibana to become unresponsive.

4. Caching Dashboard Data

Setting	Description	Default Value	Notes
kibana.index	Index pattern used for Kibana data	`.kibana`	Used to store Kibana dashboards, visualizations, and other configurations
kibana.caching.enabled	Enables or disables caching of dashboard data	`false`	When enabled, improves performance by caching frequently accessed dashboard data
kibana.caching.ttl	Time-to-live for cached dashboard data (in milliseconds)	`300000`	Defaults to 5 minutes; adjust based on data refresh frequency

Example: Enabling Caching in Kibana Configuration

kibana:
  caching:
    enabled: true
    ttl: 600000

Note: Caching is particularly effective for dashboards with static or slowly changing data. For highly dynamic dashboards, caching might lead to outdated visualizations.

5. Limiting Panel Count

Setting	Value	Description
kibana.yml	`panel.maxPerPanel`	Maximum number of panels allowed per visualization
kibana.yml	`panel.maxPerDashboard`	Maximum number of panels allowed per dashboard

Example: Configuring Panel Limits

kibana:
  panel:
    maxPerPanel: 50
    maxPerDashboard: 200

Note: Setting these limits helps prevent performance degradation due to excessive panel rendering.

6. Using Lens Over TSVB

Feature	Description	Comparison
Lens	Modern visualization engine in Kibana	Replaces TSVB in newer versions
TSVB	Time Series Visual Builder	Legacy visualization tool

Note: Lens provides more flexibility and better performance for time-series data visualization compared to TSVB.

Example: Creating a Lens Visualization

{
  "title": "CPU Usage Over Time",
  "visState": {
    "type": "lnsXY",
    "params": {
      "addTimeMarker": true,
      "addLegend": true
    },
    "data": {
      "attributes": {
        "index": "logs-*"
      }
    }
  }
}

TSVB vs Lens: Key Differences

Performance: Lens is faster and more efficient
Flexibility: Lens supports more chart types
Maintenance: TSVB is deprecated, Lens is actively developed

7. Configuring Browser Settings

Setting	Description	Default Value	Notes
kibana.browser.loadThumbnails	Controls whether Kibana loads thumbnails for visualizations	`true`	Set to `false` to improve performance on low-end browsers
kibana.browser.maxConcurrentRequests	Maximum number of concurrent HTTP requests from the browser	`10`	Reduce to `5` for better performance on older browsers
kibana.browser.maxRequestSize	Maximum size (in bytes) of a single HTTP request	`1048576`	Reduce to `524288` for better memory usage

Example: Browser Settings Configuration

kibana:
  browser:
    loadThumbnails: false
    maxConcurrentRequests: 5
    maxRequestSize: 524288

8. Optimizing Date Histograms

Property	Description	Default	Notes
date_histogram.interval	Interval for grouping data points	`auto`	Can be `1m`, `1h`, `1d`, etc.
date_histogram.calendar_interval	Calendar-aware interval (e.g., `1M`)	`auto`	Useful for month/year grouping
date_histogram.min_doc_count	Minimum number of documents to include in bucket	`1`	Set to `0` to show empty buckets
date_histogram.format	Date format for display	`yyyy-MM-dd`	Can use custom formats like `MM/dd/yyyy`

Example: Date Histogram Aggregation

{
  "aggs": {
    "sales_over_time": {
      "date_histogram": {
        "field": "timestamp",
        "interval": "1d",
        "format": "yyyy-MM-dd"
      }
    }
  }
}

Note: Using auto interval can lead to performance issues with large datasets. Explicitly setting the interval improves query speed.

Warning: Setting min_doc_count to 0 may cause performance degradation on large date ranges.

Best Practices for Date Histograms

Use fixed intervals instead of auto for better performance
Limit date range in queries to reduce data processing
Pre-aggregate data if frequent time-based queries are expected
Use format to optimize display performance

9. Using Sampling

Sampling is a powerful technique to reduce the amount of data processed and displayed in Kibana, improving performance for large datasets.

Option	Values	Description
sample	`true` / `false`	Enables or disables sampling of data in visualizations and dashboards.
sampleSize	`1000`, `5000`, etc.	Defines the number of documents to sample from the index.

Example: Enabling Sampling in Kibana

{
  "index": "logs-*",
  "sample": true,
  "sampleSize": 5000
}

Note: Sampling is particularly useful for dashboards with high-cardinality fields or large time ranges where full data rendering may cause performance issues.

10. Disabling Auto-Refresh

Setting	Description	Default Value	Notes
autoRefresh	Controls whether Kibana automatically refreshes the dashboard	`true`	Set to `false` to disable auto-refresh

Example: Disabling Auto-Refresh in Kibana

{
  "autoRefresh": false
}

Note: Disabling auto-refresh can improve performance in dashboards with heavy data loads or complex visualizations.

11. Configuring Kibana Memory

Property	Default Value	Description	Notes
kibana.memoryLimit	`1gb`	Maximum memory Kibana can use	Set to a value that does not exceed available system memory
kibana.memoryAllocation	`1gb`	Amount of memory allocated to Kibana process	Should be less than or equal to `kibana.memoryLimit`
kibana.maxOldSpaceSize	`1024`	Maximum size of the old generation heap in MB	Used to configure V8's memory allocation for older objects

Example: Setting Kibana Memory Limits

kibana:
  memoryLimit: 2gb
  maxOldSpaceSize: 1536

Note: Adjust kibana.memoryLimit and kibana.maxOldSpaceSize based on your system's available RAM to prevent out-of-memory errors.

12. Monitoring Kibana Performance

Metric	Description	Recommended Threshold	Monitoring Tool
Response Time	Time taken for Kibana to respond to user requests	< 2 seconds	Kibana Logs, APM
CPU Usage	Percentage of CPU utilized by Kibana process	< 80%	System monitoring tools (e.g., Prometheus, Grafana)
Memory Usage	Amount of RAM used by Kibana process	< 80% of available memory	System monitoring tools (e.g., Prometheus, Grafana)
Search Latency	Time to execute queries in Kibana	< 5 seconds	Kibana Logs, APM
Index Pattern Load Time	Time to load index patterns in Kibana UI	< 1 second	Kibana Logs, APM

Example: Kibana Performance Metrics Dashboard

{
  "title": "Kibana Performance Dashboard",
  "panels": [
    {
      "id": "response-time",
      "type": "metric",
      "query": "avg(kibana.response.time)"
    },
    {
      "id": "cpu-usage",
      "type": "metric",
      "query": "avg(kibana.cpu.usage)"
    },
    {
      "id": "memory-usage",
      "type": "metric",
      "query": "avg(kibana.memory.usage)"
    }
  ]
}

Note: Use Kibana's built-in monitoring features or external APM tools like Elastic APM to track these metrics in real-time.