Tuning Kernel Parameters

1. Increasing File Descriptors

Parameter	Default Value	Description	Notes
fs.file-max	`1024 * 1024`	Maximum number of file handles that the kernel can allocate	System-wide limit. Increase for high-concurrency setups.
fs.nr_open	`2097152`	Maximum number of file handles each process can open	Must be less than or equal to `fs.file-max`.

Example: Increasing File Descriptors

echo 'fs.file-max = 2097152' >> /etc/sysctl.conf
echo 'fs.nr_open = 2097152' >> /etc/sysctl.conf
sysctl -p

Note: Changes to fs.file-max and fs.nr_open require a system reboot or a reload of the kernel parameters to take effect.

Warning: Setting these values too high can lead to resource exhaustion and system instability. Ensure sufficient RAM is available.

2. Configuring Backlog Queue

Parameter	Description	Default Value	Notes
net.core.somaxconn	Maximum number of connections that can be queued for acceptance	128	Should be set to a value higher than the expected number of concurrent connections
net.core.netdev_max_backlog	Maximum number of packets queued for processing by network device drivers	1000	Higher values help handle burst traffic but consume more memory
net.ipv4.tcp_max_syn_backlog	Maximum number of remembered SYN connections	1024	Increases the number of half-open connections that can be queued

Example: Increasing Backlog Limits

sudo sysctl -w net.core.somaxconn=2048
sudo sysctl -w net.core.netdev_max_backlog=2000
sudo sysctl -w net.ipv4.tcp_max_syn_backlog=2048

Note: Changes made with sysctl are temporary and will be reset after a reboot. To make them permanent, add the settings to /etc/sysctl.conf or /etc/sysctl.d/99-nginx.conf.

3. Setting TCP Buffer Sizes

Parameter	Default Value	Description	Notes
net.core.rmem_max	`134217728`	Maximum receive socket buffer size in bytes	Recommended to set to at least 128MB for high-performance setups
net.core.wmem_max	`134217728`	Maximum send socket buffer size in bytes	Same as rmem_max for optimal performance
net.ipv4.tcp_rmem	`4096 87380 6291456`	Min, default, and max TCP receive buffer sizes	Values in bytes; affects connection handling
net.ipv4.tcp_wmem	`4096 65536 16777216`	Min, default, and max TCP send buffer sizes	Values in bytes; affects data transmission
net.ipv4.tcp_mtu_probing	`0`	Enable MTU probing for TCP connections	Set to `1` or `2` for better performance in high-latency networks

Example: Setting TCP Buffer Sizes in sysctl.conf

# Increase socket buffer sizes
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728

# Configure TCP buffer sizes
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 65536 16777216

# Enable MTU probing (optional)
net.ipv4.tcp_mtu_probing = 1

Note: Changes to these parameters require a system reboot or sysctl -p to take effect. Monitor performance after tuning.

4. Enabling TCP Fast Open

Parameter	Values	Description
tcp_fast_open	`0`, `1`, `2`	Enables or disables TCP Fast Open (TFO) for incoming connections. `0` disables TFO, `1` enables it for all connections, `2` enables it only for TLS connections.

Example: Enabling TCP Fast Open for TLS

server {
    listen 443 ssl http2;
    tcp_fast_open 2;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;
}

Note: TCP Fast Open requires kernel support (Linux 3.7+). Ensure /proc/sys/net/ipv4/tcp_fast_open is set to a value that supports TFO (e.g., 3).

5. Configuring Keepalive Parameters

Parameter	Description	Default Value	Notes
keepalive_timeout	Timeout for keep-alive connections	`75s`	Time to keep connections open for reuse
keepalive_requests	Maximum number of requests per keep-alive connection	`100`	After this, connection is closed and reopened
keepalive_disable	Disable keep-alive for specific user agents	`none`	Can be set to `msie6`, `safari`, etc.

Example: Setting Keepalive Parameters

server {
    listen 80;
    server_name example.com;

    keepalive_timeout 65s;
    keepalive_requests 100;
}

Note: Adjust keepalive_timeout based on client behavior and expected traffic patterns to optimize resource usage.

6. Setting Connection Tracking

Parameter	Values	Description
net.netfilter.nf_conntrack_max	`65536`	Maximum number of connections tracked by the connection tracking subsystem
net.netfilter.nf_conntrack_tcp_timeout_established	`600`	Timeout for established TCP connections in seconds
net.netfilter.nf_conntrack_tcp_timeout_time_wait	`60`	Timeout for TIME_WAIT state connections in seconds

Example: Adjusting Connection Tracking Limits

sudo sysctl -w net.netfilter.nf_conntrack_max=131072
sudo sysctl -w net.netfilter.nf_conntrack_tcp_timeout_established=1200
sudo sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=120

Note: These values should be adjusted based on your server's expected load and available memory.

7. Optimizing Memory Settings

Setting	Description	Recommended Value	Notes
worker_connections	Maximum number of connections per worker process	65535	Higher values allow more concurrent connections but use more memory
worker_rlimit_nofile	Maximum number of file descriptors per worker process	65535	Must be set to at least `worker_connections`
worker_rlimit_mem	Maximum memory limit per worker process (in bytes)	1073741824	1GB per worker; adjust based on system RAM
client_body_buffer_size	Buffer size for client request body	1k	Adjust based on typical payload sizes
client_max_body_size	Maximum allowed size of client request body	10m	Adjust based on application needs
client_header_buffer_size	Buffer size for client request headers	1k	Default is usually sufficient
large_client_header_buffers	Number and size of buffers for large client headers	4 8k	Adjust if headers exceed default size

Example: Memory Optimization Configuration

worker_connections 65535;
worker_rlimit_nofile 65535;
worker_rlimit_mem 1073741824;

client_body_buffer_size 1k;
client_max_body_size 10m;
client_header_buffer_size 1k;
large_client_header_buffers 4 8k;

Note: These settings should be tuned based on actual usage patterns and system resources. Monitor memory usage to ensure optimal performance.

8. Using AIO and DirectIO

Directive	Description	Values	Notes
aio	Enables or disables AIO	`on`, `off`	Available in Nginx 1.25.0+
directio	Bypasses the kernel's page cache for file reads	`off`, `size`	Use with `aio` for better performance

Example: Configuring AIO and DirectIO

http {
    aio on;
    directio 4k;

    server {
        listen 80;
        root /var/www/html;

        location / {
            try_files $uri $uri/ =404;
        }
    }
}

Note: AIO and DirectIO are particularly useful for high-throughput file serving, such as media streaming or large static asset delivery.

9. Configuring Thread Pool

Directive	Values	Description
thread_pool	`name threads max_connections`	Defines a thread pool for handling connections. NEW in Nginx 1.25.0
thread_pool	`name threads max_connections`	Defines a thread pool for handling connections. NEW in Nginx 1.25.0

Example: Defining a Thread Pool

thread_pool my_thread_pool threads=32 max_connections=1024;

Example: Using the Thread Pool

server {
    listen 80 thread_pool=my_thread_pool;
    location / {
        proxy_pass http://backend;
    }
}

10. Setting Accept Mutex

Directive	Values	Description
accept_mutex	`on` \| `off`	Controls whether the accept mutex is enabled. Default is `on`.

Note: The accept mutex is used to prevent thundering herd problem when multiple worker processes try to accept new connections simultaneously.

Example: Enabling Accept Mutex

events {
    accept_mutex on;
}

Example: Disabling Accept Mutex

events {
    accept_mutex off;
}

11. Configuring Multi Accept

Directive	Values	Description
multi_accept	`on` \| `off`	Controls whether the listening socket accepts multiple connections at once. When set to `on`, Nginx will attempt to accept multiple connections per event loop iteration, which can improve performance under high load.

Example: Enabling multi_accept

events {
    multi_accept on;
    worker_connections 1024;
}

Note: The multi_accept directive is available since Nginx 1.15.4 and requires the EPOLL event model to be used.

12. Setting Worker RLimit Nofile

Directive	Values	Description
worker_rlimit_nofile	`integer`	Sets the maximum number of file descriptors that can be opened by a worker process

Example: Configuring worker_rlimit_nofile

events {
    worker_connections 1024;
    worker_rlimit_nofile 2048;
}

http {
    server {
        listen 80;
        root /var/www;
    }
}

Note: The value should be set higher than worker_connections to avoid file descriptor limits being hit under high load.