Installing and Setting Up Elasticsearch

1. Installing Elasticsearch

Platform	Installation Method	Notes
Linux (Debian/Ubuntu)	`apt install elasticsearch`	Requires Java 17 or higher. Add repository for latest version.
Linux (RHEL/CentOS)	`yum install elasticsearch`	Use `rpm` for manual installation. Requires Java 17+.
macOS	`brew install elasticsearch`	Homebrew formula includes Java 17. Default config in `/usr/local/etc/elasticsearch`.
Windows	MSI installer or ZIP archive	Download from Elastic.co. Requires Java 17+ installed.
Docker	`docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.15.0`	Use `discovery.type=single-node` for single-node setup. Default credentials: `elastic`/`changeme`.

Example: Docker Run Command

docker run -d \
  --name elasticsearch \
  --net elastic-network \
  -p 9200:9200 \
  -p 9300:9300 \
  -e "discovery.type=single-node" \
  -e "xpack.security.enabled=true" \
  docker.elastic.co/elasticsearch/elasticsearch:8.15.0

Note: Elasticsearch requires at least 2GB of RAM for optimal performance. On systems with less memory, configure ES_JAVA_OPTS to limit heap size.

Warning: Do not run Elasticsearch as root user. Set up a dedicated elasticsearch system user with appropriate permissions.

2. Configuring JVM Heap Size

Note: The JVM heap size configuration is critical for Elasticsearch performance. It should be set to no more than 50% of your system's total RAM and must not exceed 32GB.

Configuration File	Property	Default Value	Description
jvm.options	`-Xms` and `-Xmx`	`1g`	Initial and maximum heap size for the JVM

Example: Setting Heap Size in jvm.options

-Xms2g
-Xmx2g

Best Practices for Heap Size

Set -Xms and -Xmx to the same value to avoid heap resizing overhead.
Do not set heap size above 32GB due to GC limitations in older JVM versions.
Monitor heap usage with elasticsearch logs or GET /_nodes/stats.

3. Setting Up Network Bindings

Property	Default Value	Description
network.bind_host	`localhost`	Host to bind the network interface to. Can be a hostname or IP address.
network.publish_host	Derived from `network.bind_host`	Host to publish for other nodes in the cluster to connect to.
network.host	`localhost`	Shortcut to set both `bind_host` and `publish_host`.
http.port	`9200`	Port for the HTTP interface.
transport.port	`9300`	Port for the transport layer used by nodes to communicate.

Example: Binding to All Interfaces

network:
  host: 0.0.0.0
  port: 9200

transport:
  port: 9300

Note: Binding to 0.0.0.0 allows external connections, which may pose security risks in production environments.

Warning: Ensure that firewall rules are configured to allow traffic on the specified ports if binding to external interfaces.

4. Configuring Cluster Name

Property	Default Value	Description
cluster.name	`elasticsearch`	The name of the Elasticsearch cluster. All nodes that join the cluster must have the same cluster name.

Example: Setting Cluster Name

cluster.name: my-cluster

Note: Changing the cluster name will cause nodes to form a new cluster, and existing data will not be accessible.

5. Setting Up Node Roles

Role	Description	Configuration Key	Default
node.role	Defines the primary role of the node	`node.roles`	data
node.master	Enables or disables master node functionality	`node.master`	false
node.data	Enables or disables data node functionality	`node.data`	true
node.ingest	Enables or disables ingest node functionality	`node.ingest`	true
node.ml	Enables or disables machine learning node functionality	`node.ml`	false
node.transform	Enables or disables transform node functionality	`node.transform`	false

Example: Configure Node Roles

node:
  roles:
    - master
    - data
    - ingest

Note: A node can have multiple roles assigned, but it must have at least one role to function properly.

6. Configuring Path Settings

Property	Default Value	Description
path.data	`/var/lib/elasticsearch`	Directory where Elasticsearch stores its data files
path.logs	`/var/log/elasticsearch`	Directory where Elasticsearch writes its log files
path.conf	`/etc/elasticsearch`	Directory where Elasticsearch reads its configuration files
path.plugins	`/usr/share/elasticsearch/plugins`	Directory where Elasticsearch installs plugins

Example: Setting custom paths in elasticsearch.yml

path:
  data: /custom/data/path
  logs: /custom/logs/path
  conf: /custom/conf/path
  plugins: /custom/plugins/path

Note: Ensure that the Elasticsearch user has appropriate read/write permissions for the configured paths.

7. Setting Up Environment Variables

Example: Elasticsearch Environment Configuration

# Set environment variables for Elasticsearch
export ES_HOME=/opt/elasticsearch
export ES_PATH_CONF=$ES_HOME/config
export ES_JAVA_OPTS="-Xms2g -Xmx2g"
export ES_JAVA_HOME=/usr/lib/jvm/java-17-openjdk

Variable	Description	Default Value	Notes
ES_HOME	Root directory of Elasticsearch installation	`/usr/share/elasticsearch`	Must be set for proper operation
ES_PATH_CONF	Configuration directory path	`$ES_HOME/config`	Used to locate elasticsearch.yml and jvm.options
ES_JAVA_OPTS	Java options for Elasticsearch process	`-Xms1g -Xmx1g`	Recommended to set heap size to 50% of available RAM, max 32GB
ES_JAVA_HOME	Path to Java installation	`/usr/lib/jvm/java-17-openjdk`	Must point to valid JDK installation

Note: Environment variables can be set in /etc/default/elasticsearch on Debian/Ubuntu systems or /etc/sysconfig/elasticsearch on Red Hat/CentOS systems.

Warning: Setting ES_JAVA_OPTS incorrectly can cause Elasticsearch to fail to start or run inefficiently.

Best Practices for Environment Variables

Set ES_JAVA_OPTS to appropriate heap size based on available system memory
Ensure ES_HOME and ES_PATH_CONF are correctly set to avoid configuration issues
Use ES_JAVA_HOME to specify the correct Java version (JDK 17 or later required)
Apply environment variable changes via system configuration files for persistence

8. Configuring Cluster Discovery

Property	Default Value	Description
discovery.seed_hosts	`[]`	List of host addresses to bootstrap the cluster from
cluster.initial_master_nodes	`[]`	Initial list of master-eligible nodes in the cluster
discovery.type	`single-node`	Discovery mechanism to use (e.g., `single-node`, `zen`)

Example: Basic Cluster Discovery Configuration

discovery:
  seed_hosts: [&#34;192.168.1.10:9300&#34;, &#34;192.168.1.11:9300&#34;]
  type: single-node

cluster:
  initial_master_nodes: [&#34;node-1&#34;, &#34;node-2&#34;]

Note: For multi-node clusters, discovery.seed_hosts must include at least one reachable node address. The cluster.initial_master_nodes list should contain all nodes that can become master nodes.

Warning: Misconfiguring discovery.seed_hosts can lead to split-brain scenarios or nodes failing to join the cluster.

9. Starting and Stopping Services

Starting Elasticsearch Service

Start the service using systemd:

sudo systemctl start elasticsearch

Verify the service status:

sudo systemctl status elasticsearch

Enable the service to start at boot:

sudo systemctl enable elasticsearch

Stopping Elasticsearch Service

Stop the service using systemd:

sudo systemctl stop elasticsearch

Disable the service from starting at boot:

sudo systemctl disable elasticsearch

Note: On systems without systemd, use the init script or service command directly.

Command	Description	Notes
systemctl start elasticsearch	Starts the Elasticsearch service	Service must be installed and configured
systemctl stop elasticsearch	Stops the Elasticsearch service	Gracefully shuts down the node
systemctl status elasticsearch	Displays current service status	Shows if the service is running or failed
systemctl enable elasticsearch	Enables the service to start at boot	Persists across reboots
systemctl disable elasticsearch	Disables the service from starting at boot	Service remains stopped but won't auto-start

10. Verifying Installation

Example: Check Elasticsearch Status

curl -X GET "localhost:9200/_cluster/health?pretty"

Field	Description	Sample Value
status	Cluster health status (green, yellow, red)	`green`
number_of_nodes	Total number of nodes in the cluster	`1`
active_primary_shards	Number of active primary shards	`1`

Note: A green status indicates all shards are allocated and the cluster is healthy.

Example: List All Indices

curl -X GET "localhost:9200/_cat/indices?v"

Index Name	Status	Docs Count	Store Size
`.kibana`	`open`	`1`	`3.2kb`
`.kibana_task_manager`	`open`	`1`	`3.2kb`

Warning: If no indices are listed, Elasticsearch may not be running or configured correctly.

11. Configuring Bootstrap Checks

Check	Description	Default Value	Notes
bootstrap.memory_lock	Prevents Elasticsearch from swapping memory pages to disk	`false`	Set to `true` for production environments to avoid performance issues
bootstrap.system_call_filter	Enables system call filtering to restrict potentially dangerous operations	`true`	Recommended for security; can be disabled in some containerized environments
discovery.seed_hosts	List of host addresses to use for initial discovery in a cluster	`[]`	Must be set for multi-node clusters to form a cluster
cluster.initial_master_nodes	Initial list of master-eligible nodes in a new cluster	`[]`	Required when starting a new cluster with v7.x+

Example: Enabling Memory Lock

bootstrap.memory_lock: true

Note: On Linux systems, memory locking requires setting memlock limits in /etc/security/limits.conf.

Warning: Disabling bootstrap.system_call_filter may expose the system to security risks.