Debugging Nginx

1. Testing Configuration

Command	Description	Notes
nginx -t	Test the Nginx configuration for syntax errors	Does not reload the server
nginx -T	Test configuration and display the full effective configuration	Shows merged config from all files

Example: Testing Nginx Configuration

nginx -t

Note: If the configuration test passes, Nginx will output a message like nginx: the configuration file /etc/nginx/nginx.conf syntax is ok and nginx: configuration file /etc/nginx/nginx.conf test is successful.

Warning: Running nginx -t does not apply changes to the running server. Use nginx -s reload to apply configuration changes after testing.

2. Enabling Debug Log

Directive	Values	Description
error_log	`logs/error.log debug`	Specifies the log file and level. Use `debug` to enable detailed logging.

Example: Enabling Debug Log

error_log /var/log/nginx/error.log debug;

Note: Debug logs can be very verbose and may impact performance. Use only when necessary for troubleshooting.

3. Using Rewrite Log

Directive	Values	Description
rewrite_log	`on` \| `off`	Enables or disables the rewrite log. Default is `off`.

Example: Enabling Rewrite Log

server {
    listen 80;
    server_name example.com;

    rewrite_log on;

    location / {
        rewrite ^/old-path/(.*)$ /new-path/$1 permanent;
    }
}

Note: The rewrite log is written to the error_log with the info level when rewrite_log on is set.

Warning: Enabling rewrite_log can significantly impact performance in high-traffic environments.

4. Checking Error Logs

Log Type	Location	Description
error.log	`/var/log/nginx/error.log`	Default location for Nginx error logs. Captures errors, warnings, and critical issues.
error.log (custom)	`/path/to/custom/error.log`	Custom location specified via `error_log` directive.

Example: Viewing Nginx Error Logs

tail -f /var/log/nginx/error.log

Example: Filtering Logs by Time

grep "2026/01/01" /var/log/nginx/error.log

Note: Ensure the Nginx user has read permissions on the log file to avoid permission errors.

5. Debugging SSL Issues

Issue	Description	Solution
Certificate not trusted	Certificate is self-signed or issued by an untrusted CA	Install the CA certificate in the client's trust store or use a valid certificate from a recognized CA
SSL handshake failure	Client and server cannot agree on a cipher or protocol	Check supported SSL protocols and ciphers in both client and server config; ensure compatibility
Invalid certificate chain	Missing intermediate certificates or incorrect order	Ensure the full certificate chain is provided in the correct order in `ssl_certificate`
Certificate expired	Server certificate has passed its validity date	Renew or replace the certificate with a valid one

Example: SSL debug logging

error_log /var/log/nginx/error.log debug;

server {
    listen 443 ssl;
    ssl_certificate /etc/ssl/certs/nginx.crt;
    ssl_certificate_key /etc/ssl/private/nginx.key;

    # Enable detailed SSL logging
    error_log /var/log/nginx/ssl_error.log debug;
}

Note: Use openssl s_client -connect example.com:443 to test SSL handshake and certificate details from the command line.

Warning: Avoid using ssl_protocols SSLv2 or SSLv3 as they are insecure and deprecated.

Common SSL Debugging Steps

Check certificate validity and expiration date
Verify the certificate chain includes all intermediate certificates
Ensure SSL protocols and ciphers are compatible between client and server
Enable debug logging to capture detailed SSL handshake errors
Use tools like openssl s_client to simulate and inspect SSL connections

6. Testing with Curl

Example: Basic curl request to Nginx

curl http://localhost

Option	Values	Description
-X	`GET`	Specify HTTP method (default is GET)
-H	`Accept: text/html`	Add custom header to request
-v	`true`	Verbose output showing request/response details
-I	`true`	Fetch only headers (HEAD request)

Note: Use -v to inspect headers and status codes for debugging.

Example: Simulate a POST request

curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' http://localhost/api

7. Using Nginx Debug Build

Example: Enabling debug logging in Nginx

error_log /var/log/nginx/error.log debug;

server {
    listen 80;
    server_name example.com;

    location / {
        # Debug-level logging for this location
        error_log /var/log/nginx/access.log debug;
        return 200 "Hello from Nginx debug build!";
    }
}

Directive	Description	Values	Notes
error_log	Set the error log file and level	`error_log /path/to/error.log debug;`	Debug level enables detailed logging
debug	Log level for debug messages	`debug`	Most verbose logging level

Note: Debug builds of Nginx are not enabled by default in most distributions. You must compile from source or use a debug-enabled package.

Warning: Enabling debug logging can significantly impact performance and disk space usage. Use only for debugging purposes.

8. Analyzing Core Dumps

Step	Description	Action
Enable Core Dumps	Ensure Nginx is configured to generate core dumps on crash	`worker_processes auto; error_log /var/log/nginx/error.log debug; debug_points on;`
Core Dump Location	Set core dump path in system limits	`* soft core unlimited * hard core unlimited`
Generate Core Dump	Force a crash to generate core dump (e.g., send SIGSEGV)	`kill -SEGV <nginx_pid>`

Example: Analyzing Core Dump with GDB

gdb /usr/sbin/nginx <core_file>
(gdb) bt
(gdb) info registers

Note: Ensure gdb is installed and Nginx binary has debug symbols for detailed analysis.

9. Debugging Proxy Issues

Issue	Description	Solution
502 Bad Gateway	Proxy server received an invalid response from upstream server	Check backend service status, increase `proxy_connect_timeout`
504 Gateway Timeout	Proxy server did not receive a timely response from upstream	Increase `proxy_read_timeout`, check backend performance
500 Internal Server Error	Backend server error, often misconfiguration	Review `error_log`, check proxy headers and backend config
403 Forbidden	Proxy blocked request due to access control	Verify `allow`/`deny` rules, check `proxy_pass` permissions

Example: Enable Detailed Proxy Logging

error_log /var/log/nginx/error.log debug;

location / {
    proxy_pass http://backend;
    proxy_connect_timeout 30s;
    proxy_read_timeout 30s;
    proxy_send_timeout 30s;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Note: Use debug level in error_log only temporarily as it can impact performance.

Common Proxy Debugging Steps

Check backend server status and logs
Verify proxy timeouts are sufficient
Validate headers and URL paths
Review error_log for detailed error messages

10. Using tcpdump for Network Analysis

Example: Capture Nginx Traffic

tcpdump -i any -s 0 -w nginx.pcap port 80 and host example.com

Option	Description	Notes
-i any	Interface to capture packets on	Use `any` to capture on all interfaces
-s 0	Snapshot length	0 means capture full packets
-w nginx.pcap	Output file	Saves captured packets to a PCAP file
port 80 and host example.com	Filter criteria	Captures only traffic to port 80 from example.com

Note: Ensure tcpdump is installed and you have sufficient permissions to capture network traffic.

Warning: Capturing full packets can generate large files quickly. Monitor disk space.

11. Debugging Location Matching

Directive	Description	Example
error_log	Specifies the file to which error logs are written. Use `debug` level for detailed location matching info.	`error_log /var/log/nginx/error.log debug;`
location	Defines how to handle requests based on URI. Use `debug` logging to trace matching.	`location /api/ { proxy_pass http://backend; }`

Example: Debugging Location Matching

error_log /var/log/nginx/error.log debug;

server {
    listen 80;
    server_name example.com;

    location /api/ {
        proxy_pass http://backend;
    }

    location = /index.html {
        root /var/www;
    }
}

Note: Enable debug logging to see detailed trace of how Nginx matches locations. This is useful for troubleshooting complex location blocks.

12. Using Echo Module for Testing

Example: Basic Echo Module Usage

server {
    listen 80;
    location /echo {
        echo "Hello, Nginx!";
        echo "Current time: $time_local";
    }
}

Directive	Description	Usage
echo	Echoes a string to the client	`echo "Hello World"`
echo -n	Echoes without a trailing newline	`echo -n "No newline"`
echo_flush	Forces output to be flushed immediately	`echo_flush`

Note: The echo module is part of the openresty/echo-nginx-module and must be compiled into Nginx to be used.

Warning: The echo module is primarily for testing and debugging. It should not be used in production environments.