Optimizing Dashboard Performance

1. Reducing Query Load (limit time range)

Option	Values	Notes
time range	`last 1 hour`	Reduce query load by limiting the time range
time range	`last 6 hours`	Optimal for real-time monitoring dashboards
time range	`last 24 hours`	Useful for daily performance analysis

Example: Limiting Time Range in Query

SELECT * FROM metrics 
WHERE timestamp >= now() - 1h 
AND timestamp <= now()

Note: Limiting the time range reduces database load and improves dashboard response times.

2. Using Query Caching (data source cache)

Option	Values	Notes
queryTimeout	`30s`	Maximum time to wait for a query result
cacheTimeout	`5m`	Duration to cache query results before re-fetching
maxCacheSize	`100MB`	Maximum size of the query cache

Example: Enabling Query Caching in a Data Source

datasources:
  - name: "Prometheus"
    type: "prometheus"
    access: "proxy"
    url: "http://localhost:9090"
    cacheTimeout: "5m"
    queryTimeout: "30s"

Note: Query caching can significantly reduce load on data sources but may lead to stale data if not configured properly.

3. Optimizing Panel Count (limit per dashboard)

Dashboard performance can be significantly impacted by the number of panels displayed. Grafana provides a way to limit the number of panels per dashboard to help maintain optimal performance.

Configuration	Value	Description
dashboard.panelLimit	`50`	Maximum number of panels allowed per dashboard. Default is 50.

Note: Setting a panel limit helps prevent performance degradation due to excessive panel rendering. This is particularly important in dashboards with many data sources or complex queries.

Example: Setting Panel Limit in Dashboard JSON

{
  "dashboard": {
    "panelLimit": 50
  }
}

Property	Description	Default
shared	Indicates if the query is shared across dashboards	`false`
sharedQueryId	Unique identifier for the shared query	`null`

Example: Creating a Shared Query

{
  "name": "Shared Database Query",
  "shared": true,
  "query": "SELECT * FROM logs WHERE timestamp > now() - 1h"
}

Note: Shared queries can significantly reduce query load on the backend by reusing results.

Warning: Ensure that shared queries do not expose sensitive data to unauthorized users.

Query Type	Shared Query Support	Notes
`SQL`	NEW	Support added in Grafana v10.0
`Logs`	DEPRECATED	Use `SQL` instead

5. Setting Appropriate Refresh (avoid frequent)

Setting	Description	Recommendation
dashboard.refresh	Controls how often the dashboard updates all panels	Set to `30s` or higher for most use cases. Avoid `5s` or less unless real-time monitoring is required.
panel.refresh	Individual panel refresh rate (if different from dashboard)	Use `off` for static data or `30s`+

Example: Dashboard Refresh Configuration

{
  "dashboard": {
    "refresh": "30s"
  }
}

Note: Frequent refreshes can overload the backend and cause dashboard lag or timeouts.

Warning: Setting refresh to less than 5s may lead to performance degradation and increased resource usage.

6. Using Data Source Proxy Cache (TTL)

Property	Value	Description
proxy	`true`	Enables proxying of data source requests through Grafana
cacheTTL	`30s`	TTL (Time To Live) for cached responses in seconds

Example: Configuring Data Source Proxy Cache

{
  "name": "Prometheus",
  "type": "prometheus",
  "access": "proxy",
  "cacheTTL": "30s"
}

Note: Setting cacheTTL to 0s disables caching entirely.

7. Minimizing Transformations (processing cost)

Transformation	Description	Impact	Best Practice
math	Applies mathematical operations to data series	High CPU usage if applied to large datasets	Use `math` only when necessary; consider pre-calculating values
alias	Renames series or applies labels to data	Low processing cost	Safe to use in large datasets
rate	Calculates rate per second for counters	High processing cost due to time-series analysis	Precompute rates in backend if possible
groupBy	Groups data by specified fields	Medium to high depending on group size	Limit grouping to essential fields; use filters before grouping
reduce	Reduces multiple series into one using functions like sum, avg	High CPU usage for large datasets	Use `reduce` with appropriate time ranges and filters

Example: Reducing Transformation Overhead

// Instead of applying multiple transformations
// Apply only necessary ones and filter early
const dashboard = {
  panels: [
    {
      targets: [
        {
          expr: 'rate(http_requests_total[5m])',
          refId: 'A'
        }
      ],
      transformations: [
        {
          id: 'reduce',
          options: {
            reducer: 'sum'
          }
        }
      ]
    }
  ]
}

Note: Grafana applies transformations in the order they appear in the panel configuration. Reordering or removing unnecessary steps can significantly reduce dashboard load times.

8. Limiting Dashboard Variables (complexity)

Variable Type	Description	Impact on Performance	Best Practices
Template Variables	Dynamic placeholders that can be used in queries and panels	High - each variable can trigger multiple queries or data fetches	Limit the number of variables and use caching where possible
Multi-value Variables	Variables that allow selecting multiple values at once	High - increases query complexity and data volume	Use default values or limit the number of selections
Query Variables	Variables populated by querying data sources directly	Very High - can cause significant load on data sources	Use static or cached variables when possible; limit query scope
Constant Variables	Fixed values defined in the dashboard JSON	Low - minimal performance impact	Use for fixed configuration or metadata

Example: Reducing Variable Complexity

{
  "templating": {
    "list": [
      {
        "name": "env",
        "type": "constant",
        "value": "production"
      },
      {
        "name": "service",
        "type": "query",
        "datasource": "Prometheus",
        "label": "Service",
        "query": "label_values(up, service)",
        "refresh": 2
      }
    ]
  }
}

Note: Query variables can significantly increase dashboard load times if not properly optimized. Consider using multi with a limited set of options or caching results.

Warning: Avoid using query variables with large datasets or complex queries. This can lead to dashboard rendering delays and timeouts.

9. Using Server-Side Rendering (images)

Option	Values	Notes
rendering.mode	`server`	Enables server-side rendering for dashboard images
rendering.timeout	`30s`	Maximum time to wait for rendering to complete
rendering.cacheTTL	`5m`	Time to cache rendered images before regenerating

Example: Configure server-side rendering

[rendering]
mode = server
timeout = 30s
cacheTTL = 5m

Note: Server-side rendering is useful for reducing client load and improving dashboard image generation speed in high-concurrency environments.

10. Profiling Dashboard Load (performance tab)

Profile Type	Description	Usage
Dashboard Load Time	Measures total time to load a dashboard including all panels and data sources	`1500ms` for a dashboard with 20 panels
Panel Rendering Time	Time taken to render each panel individually	`50ms` per panel on average
Data Fetch Time	Time spent fetching data from data sources	`300ms` per query

Example: Dashboard Performance Profiling

// Example dashboard JSON with performance metrics
{
  "title": "Performance Dashboard",
  "panels": [
    {
      "id": 1,
      "type": "graph",
      "targets": [
        {
          "expr": "rate(http_requests_total[5m])",
          "interval": "1m"
        }
      ],
      "maxDataPoints": 1000
    }
  ],
  "time": {
    "from": "now-1h",
    "to": "now"
  }
}

Note: Use maxDataPoints to limit the number of data points returned by queries.

Warning: Avoid using maxDataPoints too low, as it can cause data loss or inaccurate visualizations.

Key Metrics to Monitor

Load Time - Total dashboard load time
Panel Render Time - Individual panel rendering times
Query Duration - Time taken for each data source query