Uneven requests distribution in connections when proxy to internal listener

Troubleshooting
,
1

My issue: uneven distribution of requests per connection, so poor connection reuse:
there are connections with thousands of requests and connections with one request only.
How I encountered the problem: tested by sending traffic to nginx, proxy requests to internal (sandwich) listener (same nginx) and upstream server.

Has anyone experienced the same? Any suggestions?
Thanks in advance!

My config:
http {
map $http_upgrade $connection_upgrade {
default “”;
websocket upgrade;
}
map $DEBUG $should_log {
“false” 0;
“true” 1;
}
upstream internal {
server 127.0.0.1:8096 max_fails=0 fail_timeout=0;
keepalive 2000;
keepalive_time 50m;
keepalive_timeout 55m;
keepalive_requests 10000;
}
upstream upstream_server {
server perf.test.svc.cluster.local max_fails=0 fail_timeout=0;
keepalive 2000;
keepalive_time 50m;
keepalive_timeout 55m;
keepalive_requests 10000;
}
server {
listen 8086 backlog=1024;
access_log /tmp/8086.log 8086 if=true;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_http_version 1.1;
location / {
proxy_pass http://internal;
}
}
server {
listen 8096 backlog=1024;
access_log /tmp/8096.log 8096 if=true;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_http_version 1.1;
location / {
proxy_pass http://upstream_server;
}
}
}

1 Like
2

Try using fifo and lower your keepalive:

http {
    map $http_upgrade $connection_upgrade {
        default "";
        websocket upgrade;
    }
    
    upstream internal {
        server 127.0.0.1:8096 max_fails=0 fail_timeout=0;
        keepalive 200;
        keepalive_time 50m;
        keepalive_timeout 55m;
        keepalive_requests 1000;
        keepalive_mode fifo;
    }
    
    upstream upstream_server {
        server perf.test.svc.cluster.local max_fails=0 fail_timeout=0;
        keepalive 200;
        keepalive_time 50m;
        keepalive_timeout 55m;
        keepalive_requests 1000;
        keepalive_mode fifo; 
    }
    
    # Rest of your config remains the same
}
1 Like
3

Thanks for the answer, unfortunately, “keepalive_mode” is not available out of my nginx (error: unknown directive). I’m using opensource version.

4

From the config, each upstream only has a single node in it, so all the requests would be sent to that one endpoint. From looking at your config, there is simply more traffic hitting one server block than the other. It seems like this would be expected behavior, unless I am missing something about how requests are coming into the NGINX node.

1 Like
5

Traffic is coming from test tool to 1st listener :8086 then all requests proxied to upstream which is internal listener :8096 (sandwich). From test tool to 1st listener requests are balanced evenly. However, NGINX distribute requests internally totally uneven, where some connections have thousands of requests, but some connections have only one request, then seems these connections are not identified by nginx as available for reuse.

1 Like