Managing Kibana Data Views
1. Understanding Data Views
| Property | Description | Example |
|---|---|---|
| name | Name of the data view | my-data-view |
| indexPattern | Index pattern to use for the data view | logs-* |
| timeField | Time field used for time-based queries | @timestamp |
| fieldFormats | Formatting rules for specific fields | { "user_agent": { "type": "string" } } |
Example: Creating a Data View
{
"name": "application-logs",
"indexPattern": "app-logs-*",
"timeField": "@timestamp",
"fieldFormats": {
"user_agent": {
"type": "string"
}
}
}
Note: Data views are used to define how Kibana should interpret and display data from Elasticsearch indices.
2. Creating Data Views
Example: Create a Data View via Kibana UI
# Navigate to Kibana > Stack Management > Data Views
# Click "Create data view"
# Enter name: "my-data-view"
# Select index pattern: "logs-*"
# Set timestamp field: "@timestamp"| Field | Description | Required | Default |
|---|---|---|---|
| name | Name of the data view | Yes | |
| indexPattern | Index pattern to use for data view | Yes | |
| timeField | Timestamp field for time-based queries | No | @timestamp |
| fieldFormatMap | Custom formatting for fields | No |
Note: Data views are stored in Kibana's internal Elasticsearch index
.kibana.
3. Configuring Time Fields
| Field | Description | Default | Notes |
|---|---|---|---|
| timeField | Name of the field containing timestamp data | None | Required for time-series visualizations |
| timeFormat | Format of the timestamp field (ISO8601, epoch, etc.) | ISO8601 | Used for parsing and displaying timestamps |
| timeZone | Time zone to apply when displaying timestamps | UTC | Affects how time is displayed in Kibana UI |
Example: Setting up a time field in Kibana
{
"timeField": "@timestamp",
"timeFormat": "ISO8601",
"timeZone": "America/New_York"
}
Note: The
timeField must be present in the index pattern to enable time-series features.
4. Refreshing Field List
Refreshing the field list in Kibana ensures that newly indexed data or schema changes are reflected in the data view.
| Field | Description | Notes |
|---|---|---|
| field.refresh | Triggers a refresh of field mappings | Can be done via UI or API |
Example: Refresh Field List via Kibana UI
1. Navigate to Kibana > Stack Management
2. Go to "Data Views"
3. Select the target data view
4. Click "Refresh field list"
Note: Field refreshes may take a few seconds depending on the size of your index.
Warning: Refreshing fields does not reindex data; it only updates field metadata.
5. Setting Field Formats
| Field | Format Type | Description | Notes |
|---|---|---|---|
| field.name | string |
Name of the field in the document | Used to identify the field in Kibana |
| format.type | string |
Format type for the field | Can be string, number, date, etc. |
| format.params | object |
Parameters for the format | Used to customize how the field is displayed |
Example: Configure field format in Kibana
{
"field": "timestamp",
"format": {
"type": "date",
"params": {
"pattern": "YYYY-MM-DD HH:mm:ss"
}
}
}
Note: Field formats in Kibana can be customized to display data in a more readable way, such as formatting timestamps or numbers with specific patterns.
6. Creating Scripted Fields
Scripted fields in Kibana allow you to create calculated fields using Painless scripting language. These fields are computed on-the-fly during query execution and can be used in visualizations, dashboards, and searches.
| Field | Description | Example |
|---|---|---|
| name | Name of the scripted field | "total_sales" |
| language | Scripting language (always painless) |
"painless" |
| source | Painless script to compute the field value | params._source.price * params._source.quantity |
| type | Data type of the computed field | "number" |
Example: Creating a Scripted Field for Total Sales
{
"name": "total_sales",
"language": "painless",
"source": "params._source.price * params._source.quantity",
"type": "number"
}
Note: Scripted fields are computed at query time and may impact performance if used excessively or with large datasets.
Warning: Avoid complex scripts or loops in Painless as they can cause performance degradation.
7. Creating Runtime Fields
Runtime fields are dynamic fields that are computed at query time, allowing you to derive new data from existing fields without storing additional data.
| Field | Description | Example |
|---|---|---|
| name | Name of the runtime field | "user_agent" |
| type | Data type of the runtime field | "keyword" |
| script | Painless script to compute the field value | "params._source.user_agent" |
Example: Creating a Runtime Field
{
"name": "user_agent",
"type": "keyword",
"script": {
"source": "params._source.user_agent"
}
}
Note: Runtime fields are computed at query time, which can impact performance for large datasets.
8. Managing Field Popularity
| Field | Description | Popularity Status | Actions |
|---|---|---|---|
| field.name | Name of the field in the document | NEW | View usage statistics |
| field.type | Data type of the field (string, number, date) | Auto-detected | Update field mapping |
| field.usage.count | Number of times field appears in documents | Calculated dynamically | Filter by usage |
Example: Field Popularity Query
GET /_search
{
"aggs": {
"field_popularity": {
"terms": {
"field": "field.name",
"size": 10
}
}
}
}
Note: Popularity is calculated based on field presence in documents over a rolling window of 30 days.
9. Setting Default Data View
| Property | Description | Values |
|---|---|---|
| name | Name of the data view | string |
| indexPattern | Index pattern to use for the data view | string |
| timeField | Time field used for time-based queries | string |
| isDefault | Flag to set as default data view | boolean |
Example: Setting Default Data View via API
{
"name": "my-default-data-view",
"indexPattern": "logs-*",
"timeField": "@timestamp",
"isDefault": true
}
Note: Setting
isDefault to true will make this data view the default for new Kibana sessions.
10. Using Wildcards
| Wildcard | Description | Usage Example |
|---|---|---|
| * | Matches any number of characters (including none) | logs-* matches logs-app-1, logs-db-2 |
| ? | Matches exactly one character | log??.txt matches log12.txt, logab.txt |
| [abc] | Matches any one character from the set | file[123].log matches file1.log, file2.log |
| [a-z] | Matches any one character in the range | log[a-z].txt matches loga.txt, logz.txt |
Example: Applying Wildcards in Kibana Data Views
// Data view pattern using wildcards
{
"name": "Application Logs",
"indexPattern": "app-logs-*"
}
Note: Wildcards in Kibana data views are processed at index time and support basic glob patterns.
11. Viewing Field Capabilities
| Field | Type | Description | Capabilities |
|---|---|---|---|
| field.name | string |
Name of the field in the index pattern | read, aggregate |
| field.type | string |
Data type of the field (e.g., text, keyword) |
read |
| field.aggregatable | boolean |
Whether the field can be used in aggregations | aggregate |
| field.searchable | boolean |
Whether the field can be used in search queries | read |
Example: Fetching field capabilities via API
const response = await fetch(
'http://localhost:5601/api/index_patterns/_field_capabilities',
{
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer <token>'
}
}
);
const fieldCapabilities = await response.json();
console.log(fieldCapabilities);
Note: Field capabilities are cached and updated when index patterns change.