Working with Fleet and Elastic Agent

1. Understanding Fleet Architecture

Note: Fleet is a component of the Elastic Stack that provides centralized management and configuration for Elastic Agents.

Component	Description	Role
Elastic Agent	Lightweight agent that collects and sends data to Elasticsearch	Collects logs, metrics, and other data from hosts
Fleet Server	Manages and orchestrates Elastic Agents	Acts as the central control plane for agent management
Elasticsearch	Search and analytics engine	Stores and indexes data collected by agents
Kibana	Visualization and management interface	Provides UI for Fleet and agent configuration

Example: Fleet Server Configuration

fleet:
  server.host: "0.0.0.0"
  server.port: 8220
  agent.policy.default: "default-policy"

Deployment Workflow

Deploy Fleet Server to a host in your infrastructure
Configure Elastic Agents to connect to the Fleet Server
Assign policies to agents for data collection and management
Monitor agent status through Kibana or Fleet UI

+------------------+     +------------------+
|   Elastic Agent  |<---|   Fleet Server   |
+------------------+     +------------------+
       |                         |
       |                         |
+------------------+     +------------------+
|   Elasticsearch  |     |    Kibana        |
+------------------+     +------------------+

2. Installing Fleet Server

Example: Install Fleet Server on Linux

# Download and install Fleet Server
curl -L -O https://artifacts.elastic.co/downloads/fleet-server/fleet-server-8.12.0-linux-x86_64.tar.gz
tar xzvf fleet-server-8.12.0-linux-x86_64.tar.gz
cd fleet-server-8.12.0-linux-x86_64

# Run Fleet Server
./fleet-server run --elastic-agent.path.config /etc/fleet-server --elastic-agent.path.data /var/lib/fleet-server --elastic-agent.path.logs /var/log/fleet-server

Configuration Option	Description	Default Value
elastic-agent.path.config	Path to the configuration directory	`/etc/fleet-server`
elastic-agent.path.data	Path to the data directory	`/var/lib/fleet-server`
elastic-agent.path.logs	Path to the logs directory	`/var/log/fleet-server`
server.host	Host address to bind the server to	`0.0.0.0`
server.port	Port to listen on for incoming connections	`8220`

Note: Ensure the elastic-agent.path.config directory has proper permissions for the Fleet Server process to read and write configuration files.

Warning: Fleet Server must be configured to communicate securely with the Elastic Stack using TLS certificates.

Installation Steps

Download the Fleet Server package for your OS
Extract and navigate to the directory
Run Fleet Server with appropriate configuration paths

3. Enrolling Elastic Agents

Property	Description	Example
fleet.agent.id	Unique identifier for the agent	`"agent-12345"`
fleet.agent.name	Name of the agent as it appears in Fleet UI	`"web-server-agent"`
fleet.agent.version	Version of the Elastic Agent	`"8.12.0"`
fleet.agent.enrollment_token	Token used to enroll the agent with Fleet	`"abc123xyz"`

Example: Enroll an Elastic Agent

elastic-agent enroll --url https://fleet.elastic.co:443 --enrollment-token abc123xyz

Note: Ensure the enrollment_token is valid and has not expired.

Warning: The elastic-agent must have network access to the Fleet server.

4. Creating Agent Policies

Example: Basic Fleet Policy Creation

{
  "name": "my-agent-policy",
  "description": "Policy for monitoring web servers",
  "namespace": "default",
  "monitoring_enabled": ["logs", "metrics"],
  "tags": ["web-server", "production"],
  "agent_features": [
    {
      "name": "fleet-server",
      "enabled": true
    }
  ]
}

Field	Description	Values	Notes
name	Policy name	`string`	Must be unique within the space
description	Policy description	`string`	Optional, for documentation
namespace	Namespace for policy	`string`	Defaults to `default`
monitoring_enabled	Monitoring capabilities	`["logs", "metrics"]`	Enable logs and metrics collection
tags	Tags for policy categorization	`["string"]`	Used for filtering and grouping
agent_features	Features to enable on agents	`[{"name": "string", "enabled": boolean}]`	Configure agent capabilities

Note: Agent policies define how Elastic Agents behave and what they monitor. Policies can be applied to groups of agents or individual agents.

Warning: Modifying a policy after agents are assigned may require agent restart to take effect.

Policy Creation Steps

Create a new policy in Fleet UI or via API
Define name, namespace, and monitoring capabilities
Assign tags for organization
Enable required agent features (e.g., Fleet Server)
Apply policy to target agents

5. Managing Integrations

Example: Install a new integration

# Using Fleet UI to install an integration
# Navigate to "Integrations" in the Elastic UI
# Search for "Apache" integration
# Click "Install" and configure as needed

# Or via API
curl -X POST "http://localhost:5601/api/fleet/epm/packages" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "apache",
    "version": "1.0.0"
  }'

Integration	Description	Version	Status
apache	Collects logs and metrics from Apache web servers	`1.0.0`	NEW
nginx	Collects logs and metrics from Nginx web servers	`2.0.0`	Active
mysql	Collects performance and error logs from MySQL databases	`1.5.0`	Active

Managing Integrations Overview

Integrations are pre-built solutions to collect data from various sources
They can be installed via Fleet UI or API
Each integration includes a set of pre-defined rules, dashboards, and visualizations
Integrations can be updated or uninstalled as needed

Note: Ensure that the Elastic Agent has appropriate permissions to access the data sources before installing integrations.

6. Configuring Agent Output

Property	Values	Description
output.elasticsearch.enabled	`true` \| `false`	Enables or disables sending data to Elasticsearch.
output.elasticsearch.hosts	`["http://localhost:9200"]`	List of Elasticsearch hosts to send data to.
output.elasticsearch.index	`elastic-agent-default`	Index name pattern for logs and metrics.
output.elasticsearch.username	`elastic`	Username for Elasticsearch authentication.
output.elasticsearch.password	`password`	Password for Elasticsearch authentication.
output.elasticsearch.api_key	`base64-encoded-key`	API key for Elasticsearch authentication (alternative to username/password).
output.elasticsearch.ssl.enabled	`true` \| `false`	Enables or disables SSL/TLS for communication with Elasticsearch.
output.elasticsearch.ssl.certificate	`/path/to/cert.pem`	Path to the SSL certificate file.
output.elasticsearch.ssl.key	`/path/to/key.pem`	Path to the SSL key file.
output.elasticsearch.ssl.ca	`/path/to/ca.pem`	Path to the CA certificate file for verifying Elasticsearch.

Example: Basic Elasticsearch Output Configuration

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "elastic-agent-default"
  username: "elastic"
  password: "password"

Example: Secure Elasticsearch Output with SSL

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  index: "elastic-agent-default"
  username: "elastic"
  password: "password"
  ssl:
    enabled: true
    certificate: "/etc/elastic-agent/certs/cert.pem"
    key: "/etc/elastic-agent/certs/key.pem"
    ca: "/etc/elastic-agent/certs/ca.pem"

Note: Ensure that the elastic user has appropriate permissions to write to the specified index in Elasticsearch.

7. Updating Agents Remotely

Update Method	Description	Notes
Fleet Server	Manages and updates Elastic Agents via Fleet UI or API	Requires Fleet Server to be configured and running
Elastic Agent CLI	Use `elastic-agent update` command to update agents	Can be run on the host where agent is installed
Elastic Cloud UI	Update agents through the Elastic Cloud console	Applies to agents managed by Elastic Cloud

Example: Update Agent via CLI

elastic-agent update --version 8.12.0

Note: Ensure the target version is compatible with your current setup to avoid breaking changes.

Warning: Remote updates may fail if the agent is offline or has network issues.

8. Monitoring Agent Status

Property	Description	Values
agent.status	Current operational status of the Elastic Agent	`online`, `offline`, `error`, `updating`
agent.version	Version of the Elastic Agent currently running	`8.10.0`
agent.id	Unique identifier for the agent instance	`abc123def456`

Example: Check Agent Status via API

GET /api/fleet/agents/<agent_id>/status

Response:
{
  "id": "abc123def456",
  "status": "online",
  "version": "8.10.0"
}

Note: The agent status can be monitored via Fleet UI or the Elastic Agent API endpoint.

Warning: If the agent status is error, check logs for more details on the failure.

9. Using Standalone Mode

In standalone mode, the Elastic Agent operates independently without connecting to a Fleet server. This mode is useful for environments where centralized management isn't required or when deploying agents in isolated networks.

Configuration	Description	Default Value
agent.id	Unique identifier for the agent instance	Generated automatically
agent.name	Name of the agent instance	Hostname
agent.insecure	Disables TLS verification (not recommended for production)	false

Example: Standalone Mode Configuration

agent:
  id: standalone-agent-1
  name: "Standalone Agent"
  insecure: false
  logs:
    level: info

Note: When using standalone mode, all configuration and policy management must be handled locally on each agent instance.

10. Configuring Proxy Settings

Example: Fleet Server Proxy Configuration

# fleet-server.yml
proxy:
  url: "http://proxy.example.com:8080"
  username: "proxy-user"
  password: "proxy-password"

Property	Description	Default	Notes
proxy.url	Proxy server URL	None	Required if proxy is used
proxy.username	Proxy authentication username	None	Required if proxy requires auth
proxy.password	Proxy authentication password	None	Required if proxy requires auth

Note: Ensure proxy credentials are securely managed using Elastic's secret management features.

Warning: Misconfigured proxy settings can prevent Fleet Server from communicating with the Elastic Stack.

11. Managing Multiple Agent Policies

Policy Name	Description	Scope	Actions
default-policy	Default policy applied to all agents unless overridden	Global	`edit`, `clone`
monitoring-policy	Policy for collecting logs and metrics from agents	Monitoring	`edit`, `disable`
security-policy	Policy for security-related agent configurations	Security	`edit`, `delete`

Example: Creating a New Agent Policy

curl -X POST "http://localhost:5601/api/fleet/agent_policies" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "custom-policy",
    "description": "Custom policy for specific use case",
    "namespace": "default"
  }'

Note: Agent policies can be scoped to specific namespaces or environments for better organization.

Warning: Modifying a policy that is already assigned to agents will affect all agents using that policy.

12. Troubleshooting Agent Issues

Issue	Possible Cause	Solution
Agent not starting	Missing dependencies or configuration errors	Check logs in `/var/log/elastic-agent/`, verify config file syntax with `elastic-agent validate`
High CPU usage	Overloaded or misconfigured integrations	Review running integrations, disable unnecessary ones, check for memory leaks in logs
Agent not reporting data	Network issues or incorrect endpoint configuration	Verify connectivity to Elasticsearch, check `agent.host` and `agent.port` in config, test with `curl`
Agent logs flooded with errors	Malformed events or integration bugs	Use `elastic-agent logs --level debug`, filter logs by integration name, report issues to Elastic or community forums

Example: Check Agent Status

elastic-agent status

Example: Validate Configuration

elastic-agent validate /etc/elastic-agent/elastic-agent.yml

Note: Use elastic-agent logs --level debug to get detailed logs for troubleshooting.

Warning: Do not modify /etc/elastic-agent/elastic-agent.yml directly in production without a backup.