How to debug SSL_do_handshake() failed errors

My issue:
I am using nginx:alpine docker image and using it as a webserver as well as reverse proxy. I am trying to reverse proxy: api.hume.ai and specifically websocket.

How I encountered the problem:
When I run this I get: wss://foo_domain/api/v0/evi/chat?fernSdkLanguage=JavaScript&fernSdkVersion=0.9.12&apiKey=&verbose_transcription=false

Solutions I’ve tried:
I have tried curl and openssl command from within my docker container and they all provide response. I even tried adding ssl_ciphers, proxy_ciphers settings etc. but nothing works (I am new to this). Later I removed these trial and errors from my config assuming that I am doing too much. I would really appreciate if someone could help here.

My config:
Main nginx file:

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    log_format custom_log '$remote_addr - $remote_user [$time_local] "$request" '
                      'status=$status body_bytes_sent=$body_bytes_sent '
                      '"$http_referer" "$http_user_agent" '
                      '"$http_x_forwarded_for" "$http_host" '
                      'request_uri="$request_uri" '
                      'query_string="$query_string" '
                      'request_headers="$http_user_agent, $http_accept, $http_accept_encoding" '
                      'proxied_request="$scheme://$proxy_host$request_uri"';

    # access_log  /var/log/nginx/access.log  main;

    access_log /var/log/nginx/access.log custom_log;


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    # Define DNS resolver
    resolver 8.8.8.8 8.8.4.4 valid=300s;

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    include /etc/nginx/conf.d/*.conf;
}

Nginx Server File:

server {
    listen       443 ssl;
    server_name  foo_domain;

    ssl_certificate /etc/nginx/ssl/fullchain.pem;  # Path to your fullchain.pem
    ssl_certificate_key /etc/nginx/ssl/server.key;  # Path to your server.key

    # Content Security Policy (CSP) header for the entire website
    add_header Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self' wss: ws:;" always;

    error_log /var/log/nginx/error.log debug;
    error_log /dev/stdout debug;

    location /api/ {

        # Capture the entire query string
        set $intercepted_requesturi $request_uri;

        # If api_key is present, replace its value with "secret_value"
        if ($intercepted_requesturi ~ "^(.*[&?])*apiKey=(.*)$") {
            # Reconstruct the query string with api_key=secret_value
            set $firstvar $1;
            set $secondvar $2;
            set $intercepted_requesturi "${firstvar}apiKey=secret_value${secondvar}";
        }
        
        proxy_pass https://third_pary_domain/$intercepted_requesturi;  # Replace with the backend server's URL
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

    }

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }
}

server {
    listen       80;
    server_name  foo_domain;

    return 301 https://$host$request_uri;
}

Hi,

the general recommendation to enable a debugging log, A debugging log

Linux packages built by NGINX project itself usually contains two binaries:

  • nginx
  • nginx-debug

There’re two services available:

  • nginx - for PRODuction purposes
  • nginx-debug - for development and debugging purposes

Hope that helps.

Thank you.

I tried using nginx-debug and it provides some debugging logs now. I am still not able to figure out exact root cause. (I suspect that out of 4 IP Addresses (2 Pv6 and 2 IPv4) some of the IPs do get some transient connectivity issues). But one thing I did noticed is that my querystring parameters don’t get modified as per logs. So are the settings that I provided even correct that nginx supports websocket proxying and also with url rewriting? I see some SSL_get_error: 2, is there a way to debug/fix this further?

I have an update, I modified my nginx settings by adding below lines in proxy_pass section:

		proxy_ssl_protocols TLSv1.3;
        proxy_ssl_ciphers 'TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256';
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";

        # WebSocket specific configurations:
        proxy_read_timeout 86400;
        proxy_send_timeout 86400;

I also executed below commands in my docker container for nginx:latest
openssl ciphers provides below list

TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA

but when I run my specific third party domain:

openssl s_client -connect third_party_domain:443 -tls1_3 -cipher "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256"

I get errors:

Call to SSL_CONF_cmd(-cipher, TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256) failed
800B0D68D17F0000:error:0A0000B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match:../ssl/ssl_lib.c:2779:

Any ideas what I am missing.

In short SSL Server Test (Powered by Qualys SSL Labs) reports that TLS 1.3 is supported with above mentioned ciphers which are supported but then I get errors.