Hello, to give some context, I encountered a weird error in production when working on a new project: I have a NGINX to NGINX mTLS communication setup that goes over public internet. Interested in the PQC topic, I tried to make this NGINX config create an mTLS tunnel that’s PQC resistant, and seem to have succeeded, but I encountered, after a few hours, an error that blocked all traffic and I had to rollback, and I would like to understand what happened.
My issue: After some time, the configuration I created triggered the following error on server side (error logs in debug mode): [info] 70#70: *521602 SSL_do_handshake() failed (SSL: error:0A000065:SSL routines::no suitable key share) while SSL handshaking, client: x.x.x.x, server: 0.0.0.0:9003
How I encountered the problem: I have modified my NGINX configuration to add 2 lines with the following directive ssl_ecdh_curve X25519MLKEM768; which has seemed all right, after restarting (and not only reloading) my NGINX process, I could see all connections being TLS1.3 negotiated on this X25519MLKEM768 curve. But after a few hours, it stopped correctly negotiating and began to show in the logs the above message, leading to TLS negotiation failures.
Solutions I’ve tried: I have removed the constraint on the curve, and immediately began to see that TLS negotiation were fixed (and funnily enough, the negotiated curve was systematically also being X25519MLKEM768). Here’s an example of access log message seen after the fix : x.x.x.x - - [26/May/2026:07:15:17 +0000] “POST / HTTP/1.1” 200 42 “-” “domain.example.com” “User-Agent” “y.y.y.y” “zzzzzzzzzzzzzzzzz” “TLSv1.3” “TLS_AES_256_GCM_SHA384” “X25519MLKEM768”, corresponding to the format: log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$server_name" '
'"$http_user_agent" "$http_x_forwarded_for" "$x_request_id" '
'"$ssl_protocol" "$ssl_cipher" "$ssl_curve"';
Version of NGINX or NGINX adjacent software (e.g. NGINX Gateway Fabric): NGINX OSS on docker image nginx:1.31.0-trixie, deployed only with docker-compose, not in a k8s cluster.
Deployment environment:
Minimal NGINX config to reproduce your issue (preferably running on https://tech-playground.com/playgrounds/nginx for ease of debugging, and if not as a code block): (Tip → Run nginx -T to print your entire NGINX config to your terminal.)
This is the configuration from the NGINX Local conf (the remote one did not have its conf changed to trigger or remove the issue).
nginx.conf:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
map $http_x_request_id $x_request_id {
default $http_x_request_id;
'' $request_id;
}
map $sent_http_x_request_id $sent_x_request_id {
default "";
'' $request_id;
}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$server_name" '
'"$http_user_agent" "$http_x_forwarded_for" "$x_request_id" '
'"$ssl_protocol" "$ssl_cipher" "$ssl_curve"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 75;
gzip on;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $proxy_host; # uses upstream name instead of received host header
proxy_ssl_server_name on;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Real-IP $remote_addr;
ssl_protocols TLSv1.3;
ssl_session_cache shared:SSL_HTTPS:1m;
ssl_ecdh_curve X25519MLKEM768;
ssl_session_timeout 4h;
ssl_certificate /etc/nginx/ssl/cert.crt;
ssl_certificate_key /etc/nginx/ssl/cert.key;
ssl_trusted_certificate /etc/nginx/ssl/ca.crt;
ssl_verify_client on;
include /etc/nginx/conf.d/*.http.conf;
}
stream {
log_format basic '"$remote_addr" [$time_local] '
'"$protocol" "$status" "$bytes_sent" "$bytes_received" "$session_time" '
'"$ssl_protocol" "$ssl_cipher" "$ssl_curve"';
access_log /var/log/nginx/access.log basic;
ssl_protocols TLSv1.3;
ssl_ecdh_curve X25519MLKEM768;
ssl_session_cache shared:SSL_STREAM:1m;
ssl_session_timeout 4h;
ssl_handshake_timeout 30s;
ssl_certificate /etc/nginx/ssl/cert.crt;
ssl_certificate_key /etc/nginx/ssl/cert.key;
ssl_trusted_certificate /etc/nginx/ssl/ca.crt;
ssl_verify_client on;
include /etc/nginx/conf.d/*.stream.conf;
}
Where the conf.d folder contains files only defining only simple upstream and server blocks, such as:
upstream upsteam1 {
zone upstream1 64k;
server x.x.x.x:yyyy;
}
server {
listen 9003 ssl;
server_name server.example.com;
proxy_ssl on;
proxy_pass upstream1;
}
or
upstream upstream2 {
zone upstream2 64k;
server z.z.z.z:aaaa;
keepalive 10;
}
server {
listen 9002 ssl;
http2 on;
server_name otherserver.example.com;
client_max_body_size 100m;
location / {
proxy_pass https://upstream2;
}
}
NGINX access/error log: (Tip → You can usually find the logs in the /var/log/nginx directory.)
These were already shown earlier.
After looking online, I found this: https://serverfault.com/questions/932102/nginx-ssl-handshake-error-no-suitable-key-share, where we seem to agree that there is an issue when negotiating and the client does not seem to be able to use an X25519MLKEM768 hybrid curve, but here the client is another NGINX that I can control and configure, which is not the case of this user, plus I know from the logs that these 2 can use this curve, but it seems that the client does not present it on all SSL negotiations.
Also another user develops this here [nginx] SSL: logging level of "no suitable key share". , but I don’t actually know how NGINX behaves, when it initiates a TLS communication with upstream and which curves it present to the server? This could likely help to understand the issue.
Thanks for any help here! 