Securing Production Deployments

1. Configuring Firewall Rules

Example: Basic iptables Rule for ELK Stack

# Allow traffic on port 9200 (Elasticsearch API)
sudo iptables -A INPUT -p tcp --dport 9200 -j ACCEPT

# Allow traffic on port 5601 (Kibana UI)
sudo iptables -A INPUT -p tcp --dport 5601 -j ACCEPT

# Allow traffic on port 9200 (Elasticsearch API) from specific IP
sudo iptables -A INPUT -p tcp --dport 9200 -s 192.168.1.100 -j ACCEPT

# Drop all other traffic
sudo iptables -A INPUT -j DROP

Rule	Description	Port	Protocol
Elasticsearch API	Allows communication with Elasticsearch REST API	`9200`	TCP
Kibana UI	Allows access to Kibana web interface	`5601`	TCP
Elasticsearch Transport	Allows internal communication between nodes in the cluster	`9300`	TCP

Note: Ensure that the firewall rules are applied consistently across all nodes in the ELK stack to maintain network integrity.

Warning: Opening ports without proper authentication can expose your system to unauthorized access. Always implement secure authentication mechanisms before opening any external ports.

2. Disabling Dynamic Scripting

Configuration	Value	Description
script.allowed_types	`stored`	Allows only stored scripts, disables inline script execution
script.inline	`false`	Disables inline script execution in queries and templates
script.index	`false`	Disables inline script execution in index requests

Example: Elasticsearch Configuration

script:
  allowed_types: [stored]
  inline: false
  index: false

Note: Disabling dynamic scripting is crucial for production environments to prevent code injection vulnerabilities.

3. Implementing Network Segmentation

Example: Basic Network Segmentation Setup

# Example: Using iptables to segment ELK stack components
iptables -A INPUT -p tcp --dport 9200 -j ACCEPT
iptables -A INPUT -p tcp --dport 9200 -s 10.0.1.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 9200 -s 10.0.2.0/24 -j ACCEPT
iptables -A INPUT -p tcp --dport 9200 -j DROP

Component	Network Zone	Port	Access Control
Elasticsearch	Internal	`9200`	Restricted to Kibana and Logstash
Kibana	Internal	`5601`	Restricted to authorized users
Logstash	Internal	`5044`	Open to Beats and other log sources
Beats	External	`5044`	Securely send logs to Logstash

Network Segmentation Best Practices

Use firewalls to isolate sensitive components like Elasticsearch from external access.
Implement strict access control lists (ACLs) for each component's port.
Segment internal services (e.g., Kibana, Elasticsearch) from external log sources (e.g., Beats).
Use private subnets for internal services and public subnets for external-facing components.

4. Using Reverse Proxy

Proxy Configuration Notes

Nginx

Proxy	Configuration	Notes
Nginx	`location / { proxy_pass http://elasticsearch:9200; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }`	Basic proxy configuration for ELK stack
Apache	`ProxyPass /elasticsearch/ http://localhost:9200/ ProxyPassReverse /elasticsearch/ http://localhost:9200/`	Reverse proxy using mod_proxy

location / {
    proxy_pass http://elasticsearch:9200;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

Basic proxy configuration for ELK stack

Apache

ProxyPass /elasticsearch/ http://localhost:9200/
ProxyPassReverse /elasticsearch/ http://localhost:9200/

Reverse proxy using mod_proxy

Example: Nginx with SSL

server {
    listen 443 ssl;
    server_name elasticsearch.example.com;

    location / {
        proxy_pass https://elasticsearch:9200;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_ssl_verify off;
    }
}

Note: Ensure SSL certificates are properly configured for secure communication.

Warning: Do not expose Elasticsearch directly to the internet without a reverse proxy and authentication.

5. Configuring CORS

Property	Value	Description
http.cors.enabled	`true`	Enables or disables CORS support
http.cors.allow-origin	`/https?:\/\/localhost(:[0-9]+)?/`	Comma-separated list of allowed origins. Can use regex patterns
http.cors.allow-credentials	`false`	Allows sending credentials (cookies, authorization headers) with CORS requests
http.cors.max-age	`86400`	Maximum age (in seconds) for preflight requests
http.cors.allow-methods	`GET, POST, PUT, DELETE`	Comma-separated list of allowed HTTP methods
http.cors.allow-headers	`X-Requested-With, Content-Type, Content-Length, Authorization`	Comma-separated list of allowed headers

Example: Enabling CORS for Kibana

http.cors.enabled: true
http.cors.allow-origin: "https://kibana.example.com"
http.cors.allow-credentials: true
http.cors.max-age: 3600

Note: When using regex patterns in http.cors.allow-origin, ensure they are properly escaped to avoid unexpected behavior.

6. Implementing Rate Limiting

Component	Configuration	Description
elasticsearch.http.rate_limiting.enabled	`true`	Enables rate limiting for Elasticsearch HTTP requests
elasticsearch.http.rate_limiting.requests_per_second	`1000`	Maximum number of requests per second allowed
elasticsearch.http.rate_limiting.burst	`2000`	Maximum burst size before rate limiting kicks in

Example: Configuring Rate Limiting in elasticsearch.yml

elasticsearch:
  http:
    rate_limiting:
      enabled: true
      requests_per_second: 1000
      burst: 2000

Note: Rate limiting is applied per node and does not consider cluster-wide traffic. For cross-node rate limiting, consider using a reverse proxy or API gateway.

Feature	Status	Notes
Rate Limiting	NEW in 8.12	Introduced to prevent resource exhaustion in high-traffic environments

7. Using VPN Access

Example: Configuring VPN for ELK Stack Access

# Example: Setting up OpenVPN for secure ELK access
# Install OpenVPN server
sudo apt update
sudo apt install openvpn easy-rsa

# Generate certificates
make-certificate.sh

# Configure OpenVPN server
sudo nano /etc/openvpn/server.conf

Component	Description	Security Level
OpenVPN	Secure tunneling protocol for remote access	High
Certificate Authority (CA)	Root of trust for all certificates	High
Client Certificates	Authentication for individual users	High

Note: VPN access should be combined with additional security measures like firewall rules and user authentication.

Warning: Misconfigured VPNs can expose your entire network to unauthorized access. Ensure proper certificate management and regular audits.

8. Hardening JVM Settings

Setting	Description	Recommended Value	Notes
-Xms	Initial heap size	1g	Set to same as -Xmx for stability
-Xmx	Maximum heap size	1g	Limit to 50% of available RAM for optimal GC performance
-XX:+UseG1GC	Garbage collector	true	Recommended for large heaps and low-latency requirements
-XX:MaxGCPauseMillis	Maximum GC pause time	200ms	Helps maintain responsiveness under load
-XX:G1HeapRegionSize	Size of G1 regions	32m	Optimal for large heaps (16GB+)
-XX:+UseStringDeduplication	String deduplication	true	Reduces memory usage in applications with many duplicate strings
-XX:+UseCompressedOops	Compressed object pointers	true	Enables compressed references for 32-bit JVMs
-Djava.security.egd=file:/dev/urandom	Entropy source for secure random	true	Improves entropy availability in containerized environments

Example: JVM Settings in elasticsearch.yml

bootstrap.memory_lock: true
network.host: localhost
http.port: 9200
discovery.type: single-node

Note: Ensure that the user running Elasticsearch has sufficient permissions to lock memory (ulimit -l) and that the system's /etc/security/limits.conf is configured to allow this.

Warning: Setting -Xms and -Xmx to different values can cause frequent GC pauses. Always set them equal for stable performance.

9. Scanning for Vulnerabilities

Tool	Description	Integration	Notes
Nessus	Network vulnerability scanner	Can be integrated with ELK via log forwarding	Provides detailed vulnerability reports and remediation guidance
OpenVAS	Open-source vulnerability scanner	Supports ELK log ingestion through syslog	Free alternative to commercial tools with extensive vulnerability database
Nmap	Network discovery and security auditing tool	Logs can be parsed and sent to ELK for analysis	Useful for identifying open ports and services that may be vulnerable
Qualys	Cloud-based vulnerability management platform	Supports API integration with ELK stack	Real-time scanning and compliance reporting capabilities

Example: Nmap Scan Integration with ELK

nmap -sV -oN /var/log/nmap_scan.log <target>

Note: Ensure that the logs from vulnerability scanners are properly parsed and indexed in Elasticsearch to enable effective analysis.

Key Points for Vulnerability Scanning

Regular scanning is essential to detect new vulnerabilities in the environment
Integrate scanner logs into ELK for centralized monitoring and alerting
Use tools like Nmap or Nessus to identify network-level vulnerabilities
Compliance and security policies should guide the frequency and scope of scans

10. Implementing Secrets Management

Component	Secrets Storage	Access Control	Rotation Policy
Elasticsearch	`elasticsearch.keystore`	Role-based access control (RBAC)	Manual or scripted rotation
Logstash	Environment variables or config files	File permissions and user groups	Manual on configuration change
Kibana	Encrypted config files or external stores	Authentication and session management	Automated via plugins or scripts

Example: Storing Elasticsearch Passwords in Keystore

elasticsearch-keystore create
elasticsearch-keystore add xpack.security.authc.password_hashing.algorithm
elasticsearch-keystore add xpack.security.authc.token.enabled

Note: Ensure keystore files are protected with strict file permissions (e.g., chmod 600) to prevent unauthorized access.

Warning: Never store sensitive credentials in plain text configuration files or version-controlled repositories.

Secret Type	Storage Method	Encryption	Access Logs
API Keys	Internal Elasticsearch index	AES-256	Enabled via audit logs
Passwords	Keystore or Vault	Encrypted at rest	Logged in audit trail
Certificates	File system or PKI store	PKI-based encryption	Tracked via certificate logs

Best Practices for Secrets Management

Use centralized secret stores like HashiCorp Vault or AWS Secrets Manager
Implement automatic rotation for all secrets
Enforce least privilege access to secret data
Log and audit all secret access attempts

11. Configuring Security Headers

Header	Description	Example Value	Notes
Content-Security-Policy	Prevents cross-site scripting and other code injection attacks	`default-src 'self'; script-src 'self' 'unsafe-inline'; object-src 'none'`	Essential for preventing XSS attacks
X-Content-Type-Options	Prevents browsers from interpreting files as a different MIME type	`nosniff`	Helps prevent MIME type sniffing
X-Frame-Options	Prevents clickjacking by controlling frame embedding	`DENY`	Can also be `SAMEORIGIN`
X-XSS-Protection	Enables XSS filtering in browsers	`1; mode=block`	Deprecated in modern browsers but still useful for older ones
Strict-Transport-Security	Enforces secure (HTTPS) connections to the server	`max-age=31536000; includeSubDomains`	Must be used with HTTPS to prevent downgrade attacks

Example: Setting Security Headers in Nginx

server {
    listen 443 ssl;
    server_name example.com;

    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; object-src 'none'";
    add_header X-Content-Type-Options "nosniff";
    add_header X-Frame-Options "DENY";
    add_header X-XSS-Protection "1; mode=block";
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";

    location / {
        root /var/www/html;
        index index.html;
    }
}

Example: Setting Security Headers in Express.js

const express = require('express');
const helmet = require('helmet');
const app = express();

app.use(helmet({
  contentSecurityPolicy: {
    directives: {
      defaultSrc: ["'self'"],
      scriptSrc: ["'self'", "'unsafe-inline'"],
      objectSrc: ["'none'"]
    }
  },
  xContentTypeOptions: { setTo: "nosniff" },
  xFrameOptions: { action: "DENY" },
  xXssProtection: { setTo: "1; mode=block" },
  hsts: {
    maxAge: 31536000,
    includeSubDomains: true
  }
}));

app.get('/', (req, res) => {
  res.send('Hello World!');
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

12. Using Least Privilege Principle

Component	Required Permissions	Notes
Elasticsearch	`cluster:monitor/main`, `cluster:monitor/health`	Minimal read-only access for monitoring
Logstash	`cluster:monitor/main`, `indices:admin/exists`	Read access to indices and cluster state
Kibana	`cluster:monitor/main`, `indices:admin/exists`, `indices:data/read/search`	Read access to indices and search capabilities

Example: Elasticsearch Role Configuration

{
  "elasticsearch": {
    "cluster": [
      "monitor/main",
      "monitor/health"
    ],
    "indices": [
      {
        "names": ["logs-*"],
        "privileges": ["read"]
      }
    ]
  }
}

Note: Apply the principle by granting only necessary permissions to each component. Avoid using all or manage roles unless absolutely required.