Working with Nginx Streams (TCP/UDP)

1. Configuring Stream Block

Directive	Values	Description
stream	`on` \| `off`	Enables or disables the stream module. Default is `off`.
listen	`port` \| `ip:port`	Specifies the port or IP and port to listen on for TCP/UDP traffic.
proxy_pass	`address`	Specifies the backend server address to forward traffic to.
proxy_timeout	`time`	Sets the timeout for proxy connections. Default is `1s`.
proxy_responses	`number`	Defines how many responses to expect from the backend server.

Example: Basic Stream Configuration

stream {
    server {
        listen 12345;
        proxy_pass backend;
    }
}

upstream backend {
    server 192.168.1.10:8080;
}

2. Implementing TCP Load Balancing

Directive	Description	Values
upstream	Defines a group of servers to be load balanced	`name`
server	Defines a server in the upstream group	`address [parameters]`
least_conn	Load balancing method that selects the server with the least number of active connections	`none`
ip_hash	Load balancing method that uses the client IP address to determine server selection	`none`
keepalive	Number of keepalive connections to maintain per worker process	`number`

Example: Basic TCP Load Balancing Configuration

upstream backend {
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
}

stream {
    server {
        listen 8080;
        proxy_pass backend;
    }
}

Example: TCP Load Balancing with Sticky Sessions

upstream backend {
    ip_hash;
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
}

stream {
    server {
        listen 8080;
        proxy_pass backend;
    }
}

Example: TCP Load Balancing with Health Checks

upstream backend {
    server 192.168.1.10:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.12:8080 max_fails=3 fail_timeout=30s;
}

stream {
    server {
        listen 8080;
        proxy_pass backend;
    }
}

3. Implementing UDP Load Balancing

Directive	Description	Values	Notes
upstream	Defines a group of servers for load balancing	`name`	Used in `stream` context
server	Defines a server in the upstream group	`address [port]`	Can include `weight`, `max_fails`, `fail_timeout`
keepalive	Enables keep-alive connections to upstream servers	`size`	Optional parameter for connection reuse

Example: Basic UDP Load Balancing Configuration

stream {
    upstream udp_backend {
        server 192.168.1.10:53;
        server 192.168.1.11:53;
    }

    server {
        listen 53 udp;
        proxy_pass udp_backend;
    }
}

Example: UDP Load Balancing with Weighted Servers

stream {
    upstream udp_backend {
        server 192.168.1.10:53 weight=3;
        server 192.168.1.11:53 weight=1;
    }

    server {
        listen 53 udp;
        proxy_pass udp_backend;
    }
}

Note: UDP load balancing in Nginx is supported only in the stream module and does not support HTTP-specific features.

Warning: UDP is connectionless, so session persistence and state tracking are not supported.

4. Setting Stream Proxy

Directive	Values	Description
proxy_pass	`stream_backend`	Specifies the upstream server or group to which the stream request should be proxied.
proxy_timeout	`timeout`	Sets the timeout for connecting to the upstream server and reading the response.
proxy_bind	`address`	Binds the connection to a specific IP address and port.

Example: Basic Stream Proxy Configuration

stream {
    upstream backend {
        server 192.168.1.10:8080;
        server 192.168.1.11:8080;
    }

    server {
        listen 8080;
        proxy_pass backend;
        proxy_timeout 1s;
    }
}

5. Configuring Stream SSL

Directive	Values	Description
ssl_certificate	`path/to/cert.pem`	Path to the SSL certificate file used for stream connections
ssl_certificate_key	`path/to/key.pem`	Path to the private key file for the SSL certificate
ssl_protocols	`SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3`	SSL/TLS protocols to enable; defaults to `TLSv1.2 TLSv1.3`
ssl_ciphers	`ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDH-RSA-AES256-GCM-SHA512`	SSL ciphers to use; defaults to a secure set
ssl_prefer_server_ciphers	`on`, `off`	Whether to prefer server-side cipher preferences over client-side
ssl_session_cache	`shared:SSL:10m`	Shared memory cache for SSL sessions; `off` to disable
ssl_session_timeout	`5m`	Timeout for SSL session reuse; default is `5m`

Example: Basic SSL Stream Configuration

stream {
    server {
        listen 443 ssl;
        ssl_certificate /etc/ssl/certs/nginx.crt;
        ssl_certificate_key /etc/ssl/private/nginx.key;
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512;
        proxy_pass backend;
    }
}

Note: Ensure that the certificate and key files are readable by the Nginx process.

6. Using SNI Routing

SNI (Server Name Indication) routing allows Nginx to route TCP/UDP traffic based on the SNI extension in TLS handshakes. This is particularly useful for handling multiple SSL/TLS services on a single IP address.

Directive	Description	Values	Notes
stream	Defines a stream context block	`on` \| `off`	Enables or disables stream processing
ssl_preread	Enables or disables SNI reading	`on` \| `off`	Must be enabled to use SNI-based routing
server_name	Specifies the server name to match	`string`	Matches the SNI hostname in TLS handshake

Example: Basic SNI-based TCP Stream Configuration

stream {
    upstream backend1 {
        server 192.168.1.10:443;
    }

    upstream backend2 {
        server 192.168.1.11:443;
    }

    server {
        listen 443 ssl;
        ssl_preread on;

        # Route based on SNI hostname
        if ($ssl_preread_server_name = "api.example.com") {
            proxy_pass backend1;
        }

        if ($ssl_preread_server_name = "secure.example.com") {
            proxy_pass backend2;
        }

        # Default fallback
        proxy_pass backend1;
    }
}

Note: The ssl_preread directive must be enabled to read the SNI hostname. The $ssl_preread_server_name variable is only available in the stream context.

Warning: Ensure that all backend services support the same TLS version and cipher suites to avoid handshake failures.

7. Setting Stream Access Control

Directive	Values	Description
allow	`address` \| `cidr` \| `all`	Allows access from the specified address or network.
deny	`address` \| `cidr` \| `all`	Denies access from the specified address or network.

Example: Basic Stream Access Control

stream {
    upstream backend {
        server 192.168.1.10:8080;
        server 192.168.1.11:8080;
    }

    server {
        listen 12345;
        proxy_pass backend;

        allow 192.168.1.0/24;
        deny all;
    }
}

Note: The allow and deny directives are processed in order, with the last matching rule taking effect.

8. Configuring Stream Logging

Directive	Values	Description
error_log	`path` [ `level` ]	Set the path to the error log and the logging level. Default is `error`.
access_log	`path` [ `format` ]	Set the path to the access log and the format. Default is `combined`.
log_format	`name` `format`	Define a custom log format for use in access logs.

Example: Basic Stream Logging Configuration

error_log /var/log/nginx/error.log info;
access_log /var/log/nginx/access.log combined;

log_format stream_combined &#x27;$remote_addr - $remote_user [$time_local] $status &#x27;
&#x27;$body_bytes_sent $request_time &#x27;
&#x27;$upstream_addr $upstream_response_time&#x27;;

access_log /var/log/nginx/stream_access.log stream_combined;

Note: Stream logging directives are similar to HTTP logging but tailored for TCP/UDP traffic. The access_log directive supports custom formats defined with log_format.

Warning: Ensure that the log paths are writable by the Nginx process to avoid permission errors.

9. Implementing Health Checks

Example: Basic TCP Health Check Configuration

upstream backend {
    server 192.168.1.10:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 80;
    proxy_pass backend;
    proxy_timeout 1s;
    proxy_next_upstream_timeout 1s;
}

Directive	Description	Default	Version
max_fails	Number of failed requests before marking server as down	1	0.8.20+
fail_timeout	Time to consider server as unhealthy after max_fails	10s	0.8.20+
keepalive	Number of keepalive connections per worker	0	1.13.12+
backup	Mark server as backup (only used if all non-backup servers are down)	none	0.8.21+

Note: Health checks in Nginx are passive and rely on connection or response timeouts rather than active probing.

Example: UDP Health Check with Custom Timeout

stream {
    upstream dns_servers {
        server 8.8.8.8:53 max_fails=2 fail_timeout=10s;
        server 8.8.4.4:53 max_fails=2 fail_timeout=10s;
    }

    server {
        listen 53 udp;
        proxy_pass dns_servers;
        proxy_timeout 1s;
        proxy_responses 1;
    }
}

Health Check Best Practices

Set max_fails to at least 2 to avoid flapping due to transient issues
Use fail_timeout of 30s or more to prevent rapid reconnection attempts
Monitor nginx -T or use stub_status to observe upstream server states
Use keepalive for persistent connections to reduce overhead

10. Setting Stream Timeouts

Directive	Values	Description
stream_timeout	`time`	Sets the timeout for stream connections. Default is `60s`.
stream_connect_timeout	`time`	Sets the timeout for establishing a connection to a backend server. Default is `60s`.
stream_send_timeout	`time`	Sets the timeout for sending data to a client. Default is `60s`.
stream_read_timeout	`time`	Sets the timeout for reading data from a client. Default is `60s`.

Example: Configuring Stream Timeouts

stream {
    upstream backend {
        server 192.168.1.10:8080;
        server 192.168.1.11:8080;
    }

    server {
        listen 12345;
        proxy_pass backend;
        proxy_timeout 30s;
        proxy_connect_timeout 10s;
        proxy_send_timeout 20s;
        proxy_read_timeout 20s;
    }
}

11. Using Stream Map

Directive	Context	Description	Default
map	stream	Defines a mapping from one string to another, used for stream processing.	None

Example: Basic Stream Map Usage

stream {
    map $remote_addr $backend {
        default          backend1;
        192.168.1.100    backend2;
    }

    server {
        listen 8080;
        proxy_pass $backend;
    }
}

Note: The map directive in stream context is used to dynamically route traffic based on a condition, such as client IP address.

12. Implementing Stream Rate Limiting

Directive	Values	Description
limit_conn	`zone`, `number`	Limits the number of connections to a zone per client IP
limit_rate	`rate`	Limits the rate of response transmission per client
limit_req	`zone`, `burst`, `nodelay`	Limits the request rate to a zone

Example: Limiting Connections per IP

http {
    limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
    
    server {
        listen 8080;
        limit_conn conn_limit 10;
        
        location / {
            proxy_pass http://backend;
        }
    }
}

Example: Rate Limiting Requests

http {
    limit_req_zone $binary_remote_addr zone=req_limit:10m rate=1r/s;
    
    server {
        listen 8080;
        limit_req zone=req_limit burst=5 nodelay;
        
        location / {
            proxy_pass http://backend;
        }
    }
}

Note: The limit_rate directive can be used to limit the rate of response transmission per client. It is useful for controlling bandwidth usage in stream contexts.