Nginx UDP Load Balancing with Proxy Responses

Troubleshooting
1

My issue:
I’m having some issues with getting proxy_responses to work with my UDP load balancing setup. I am attempting to load balance RADIUS traffic to multiple ISE servers and I want the Nginx Proxy to be transparent to the connection.

I followed this guide here which works for RADIUS MAB traffic, which is a single access-request from the client and a single access-accept from the ISE server.

My issue is that for RADIUS EAP-TLS traffic there will be responses (about 10 packets exchanged) between the ISE server and the client.

However, when I enable ‘proxy_responses ;’ in the configuration the return traffic from the ISE servers fails to be processed through the Nginx proxy.

How I encountered the problem:

I’m using the Nginx Proxy as a NAT router and having all the ISE nodes set their default gateway to the Nginx proxy.
Detailed under this section in the guide

Router NAT – Rewriting the Egress Packets on an Intermediate Router

In the captures below ps-sprt-1 is the RADIUS client and ISE103 and ISE104 are the backend servers that Nginx is load balancing to.
All capture below are taken on the Nginx Proxy as all traffic from client to server will be processed through the Nginx proxy.

This is a working MAB connection packet capture taken on the Nginx proxy.
tester@nginx-proxy-1:/etc/nginx$ sudo tcpdump -i any udp port 1812
12:01:17.582534 ens3 In IP ps-sprt-1.test.com.42891 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0x93 length: 182
12:01:17.582805 ens3 Out IP ps-sprt-1.test.com.42891 > ISE103.test.com.radius: RADIUS, Access-Request (1), id: 0x93 length: 182
12:01:17.629331 ens3 In IP ISE103.test.com.radius > ps-sprt-1.test.com.42891: RADIUS, Access-Accept (2), id: 0x93 length: 146
12:01:17.629336 ens3 Out IP ISE103.test.com.radius > ps-sprt-1.test.com.42891: RADIUS, Access-Accept (2), id: 0x93 length: 146

If I leave the proxy reponses at 0 the issue is that response traffic will be sent to a different ISE node even if the port is the same.
This is a failing EAP-TLS capture taken on the Nginx Proxy. It fails because the second access-request sent from the client goes to a new ISE node ISE104 instead of returning to ISE103.
tester@nginx-proxy-1:/etc/nginx/stream.d$ sudo tcpdump -i any udp port 1812
22:03:24.122711 ens3 In IP ps-sprt-1.test.com.33658 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xff length: 148
22:03:24.123031 ens3 Out IP ps-sprt-1.test.com.33658 > ISE103.test.com.radius: RADIUS, Access-Request (1), id: 0xff length: 148
22:03:24.125951 ens3 In IP ISE103.test.com.radius > ps-sprt-1.test.com.33658: RADIUS, Access-Challenge (11), id: 0xff length: 158
22:03:24.125956 ens3 Out IP nginx-proxy-1.radius > ps-sprt-1.test.com.33658: RADIUS, Access-Challenge (11), id: 0xff length: 158
!
22:03:24.139913 ens3 In IP ps-sprt-1.test.com.33658 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0x00 length: 464
22:03:24.139987 ens3 Out IP ps-sprt-1.test.com.33658 > ISE104.test.com.radius: RADIUS, Access-Request (1), id: 0x00 length: 464
22:03:24.144318 ens3 In IP ISE104.test.com.radius > ps-sprt-1.test.com.33658: RADIUS, Access-Reject (3), id: 0x00 length: 44
22:03:24.144321 ens3 Out IP nginx-proxy-1.radius > ps-sprt-1.test.com.33658: RADIUS, Access-Reject (3), id: 0x00 length: 44

Solutions I’ve tried:

When I enable proxy responses so that the Nginx sends the second access-request packet from the client to the same ISE server for some reason the return traffic from ISE fails to be processed by the Nginx Proxy.
This packet capture is taken on the Nginx Proxy when ‘proxy_responses 10;’ is set in the configuration file.
tester@nginx-proxy-2:/etc/nginx/stream.d$ sudo tcpdump -i any udp port 1812
04:36:12.512124 ens3 In IP ps-sprt-1.test.com.54542 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0x44 length: 148
04:36:12.512441 ens3 Out IP ps-sprt-1.test.com.54542 > ISE103.test.com.radius: RADIUS, Access-Request (1), id: 0x44 length: 148
04:36:12.516570 ens3 In IP ISE103.test.com.radius > ps-sprt-1.test.com.54542: RADIUS, Access-Challenge (11), id: 0x44 length: 155

For some reason when proxy responses is enabled the ISE return traffic does not get NATTED and sent back to the client.

If I remove the other backened ISE servers from the configuration and just leave ISE103 as the only server to load balance to the EAP-TLS connection works so I know there is no issue with the processing of RADIUS EAP-TLS traffic through the Nginx proxy.

My config:

Configuration has been sanatized.
The IP of the Nginx host is 10.10.10.50
The IP of the ISE servers are 10.10.10.103 and 10.10.10.104

tester@nginx-proxy-1:~$ cat /etc/nginx/stream.d/udp-proxy.conf
stream {
upstream radius_auth {
hash $remote_addr:$remote_port consistent;
server ISE103.test.com:1812 max_fails=3 fail_timeout=30s;
server ISE104.test.com:1812 max_fails=3 fail_timeout=30s;
least_conn;
}

upstream radius_accounting {
    server ISE103.test.com:1813;
    server ISE104.test.com:1813;
    least_conn;
}

server {
    listen 1812 udp;
    proxy_pass radius_auth;
    proxy_bind $remote_addr:$remote_port transparent;
    proxy_responses 0;
}

server {
    listen 1813 udp;
    proxy_pass radius_accounting;
    proxy_bind $remote_addr transparent;
    proxy_responses 0;
}

}
!
tester@nginx-proxy-1:/etc/nginx$ cat /proc/sys/net/ipv4/ip_forward
1
!
tester@nginx-proxy-2:/etc/nginx/stream.d$ sudo tc filter show dev ens3
filter parent 10: protocol ip pref 10 u32 chain 0
filter parent 10: protocol ip pref 10 u32 chain 0 fh 800: ht divisor 1
filter parent 10: protocol ip pref 10 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 terminal flowid not_in_hw
match 38c98167/ffffffff at 12
match 07140000/ffff0000 at 20
action order 1: nat egress 10.10.10.103/32 10.10.10.50 pass
index 1 ref 1 bind 1

filter parent 10: protocol ip pref 10 u32 chain 0 fh 800::801 order 2049 key ht 800 bkt 0 terminal flowid not_in_hw
match 38c98168/ffffffff at 12
match 07140000/ffff0000 at 20
action order 1: nat egress 10.10.10.104/32 10.10.10.50 pass
index 2 ref 1 bind 1

I have enabled and reviewed Nginx debugs. While I am no expert at reading these debugs from the best I can tell it looks like the connection times out. Which makes me think the packet is not making it into the Nginx processes when proxy responses are enabled but I’m not sure why.

This is a failed connection with EAP-TLS when proxy_responses are set to 1

ter@nginx-proxy-1:~ tester@nginx-proxy-1:~ sudo tail -f /var/log/nginx/error.log
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECD83E0
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECD8210
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECD3B50, unused: 3
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECCF4D0, unused: 8
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECBF230, unused: 0
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECB9E60, unused: 8
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEECF2880, unused: 8
2025/02/04 09:42:41 [debug] 13286#13286: *5 free: 00005FBAEED44610, unused: 80
2025/02/04 09:42:41 [debug] 13286#13286: worker cycle
2025/02/04 09:42:41 [debug] 13286#13286: epoll timer: -1
2025/02/04 09:43:23 [debug] 13286#13286: epoll: fd:5 ev:0001 d:0000799D416A7010
2025/02/04 09:43:23 [debug] 13286#13286: recvmsg on 0.0.0.0:1812, ready: 0
2025/02/04 09:43:23 [debug] 13286#13286: posix_memalign: 00005FBAEED44610:256 @16
2025/02/04 09:43:23 [debug] 13286#13286: posix_memalign: 00005FBAEECF2880:256 @16
2025/02/04 09:43:23 [debug] 13286#13286: posix_memalign: 00005FBAEECB9E60:256 @16
2025/02/04 09:43:23 [debug] 13286#13286: *7 recvmsg: 10.11.11.91:39668 fd:5 n:148
2025/02/04 09:43:23 [debug] 13286#13286: add cleanup: 00005FBAEED446E0
2025/02/04 09:43:23 [debug] 13286#13286: posix_memalign: 00005FBAEECBF230:256 @16
2025/02/04 09:43:23 [info] 13286#13286: *7 udp client 10.11.11.91:39668 connected to 0.0.0.0:1812
2025/02/04 09:43:23 [debug] 13286#13286: *7 posix_memalign: 00005FBAEECCF4D0:256 @16
2025/02/04 09:43:23 [debug] 13286#13286: *7 generic phase: 0
2025/02/04 09:43:23 [debug] 13286#13286: *7 generic phase: 1
2025/02/04 09:43:23 [debug] 13286#13286: *7 generic phase: 2
2025/02/04 09:43:23 [debug] 13286#13286: *7 generic phase: 3
2025/02/04 09:43:23 [debug] 13286#13286: *7 generic phase: 4
2025/02/04 09:43:23 [debug] 13286#13286: *7 ssl preread handler
2025/02/04 09:43:23 [debug] 13286#13286: *7 proxy connection handler
2025/02/04 09:43:23 [debug] 13286#13286: *7 malloc: 00005FBAEECD8210:448
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream script var: “10.11.11.91”
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream script copy: “:”
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream script var: “39668”
2025/02/04 09:43:23 [debug] 13286#13286: *7 malloc: 00005FBAEECD83E0:16384
2025/02/04 09:43:23 [debug] 13286#13286: *7 init least conn peer
2025/02/04 09:43:23 [debug] 13286#13286: *7 get least conn peer, try: 2
2025/02/04 09:43:23 [debug] 13286#13286: *7 get least conn peer, many
2025/02/04 09:43:23 [debug] 13286#13286: *7 dgram socket 4
2025/02/04 09:43:23 [debug] 13286#13286: *7 epoll add connection: fd:4 ev:80002005
2025/02/04 09:43:23 [debug] 13286#13286: *7 connect to 10.10.10.104:1812, fd:4 #8
2025/02/04 09:43:23 [debug] 13286#13286: *7 connected
2025/02/04 09:43:23 [debug] 13286#13286: *7 proxy connect: 0
2025/02/04 09:43:23 [info] 13286#13286: *7 udp proxy 10.11.11.91:39668 connected to 10.10.10.104:1812
2025/02/04 09:43:23 [debug] 13286#13286: *7 malloc: 00005FBAEED6E1E0:16384
2025/02/04 09:43:23 [debug] 13286#13286: *7 posix_memalign: 00005FBAEECD3B50:256 @16
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream proxy add preread buffer: 148
2025/02/04 09:43:23 [debug] 13286#13286: *7 write new buf t:1 f:0 00005FBAEECB9E80, pos 00005FBAEECB9E80, size: 148 file: 0, size: 0
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream write filter: l:0 f:1 s:148
2025/02/04 09:43:23 [debug] 13286#13286: *7 sendmsg: 148 of 148
2025/02/04 09:43:23 [debug] 13286#13286: *7 stream write filter 0000000000000000
2025/02/04 09:43:23 [debug] 13286#13286: *7 event timer add: 5: 10000:425422864
2025/02/04 09:43:23 [debug] 13286#13286: timer delta: 42081
2025/02/04 09:43:23 [debug] 13286#13286: worker cycle
2025/02/04 09:43:23 [debug] 13286#13286: epoll timer: 10000
2025/02/04 09:43:23 [debug] 13286#13286: epoll: fd:4 ev:0004 d:0000799D416A75E1
2025/02/04 09:43:23 [debug] 13286#13286: *7 event timer: 5, old: 425422864, new: 425422864
2025/02/04 09:43:23 [debug] 13286#13286: timer delta: 0
2025/02/04 09:43:23 [debug] 13286#13286: worker cycle
2025/02/04 09:43:23 [debug] 13286#13286: epoll timer: 10000
2025/02/04 09:43:23 [debug] 13286#13286: epoll: fd:4 ev:0004 d:0000799D416A75E1
2025/02/04 09:43:23 [debug] 13286#13286: *7 event timer: 5, old: 425422864, new: 425422864
2025/02/04 09:43:23 [debug] 13286#13286: timer delta: 0
2025/02/04 09:43:23 [debug] 13286#13286: worker cycle
2025/02/04 09:43:23 [debug] 13286#13286: epoll timer: 10000
2025/02/04 09:43:33 [debug] 13286#13286: timer delta: 10010
2025/02/04 09:43:33 [debug] 13286#13286: *7 event timer del: 5: 425422864
2025/02/04 09:43:33 [error] 13286#13286: *7 upstream timed out (110: Connection timed out) while proxying connection, udp client: 10.11.11.91, server: 0.0.0.0:1812, upstream: “10.10.10.104:1812”, bytes from/to client:148/0, bytes from/to upstream:0/148
2025/02/04 09:43:33 [debug] 13286#13286: *7 finalize stream proxy: 502
2025/02/04 09:43:33 [debug] 13286#13286: *7 free rr peer 2 4
2025/02/04 09:43:33 [debug] 13286#13286: *7 free rr peer failed: 00005FBAEECE7F90 1
2025/02/04 09:43:33 [debug] 13286#13286: *7 close stream proxy upstream connection: 4
2025/02/04 09:43:33 [debug] 13286#13286: *7 reusable connection: 0
2025/02/04 09:43:33 [debug] 13286#13286: *7 finalize stream session: 502
2025/02/04 09:43:33 [debug] 13286#13286: *7 stream log handler
2025/02/04 09:43:33 [debug] 13286#13286: *7 close stream connection: 5
2025/02/04 09:43:33 [debug] 13286#13286: *7 reusable connection: 0
2025/02/04 09:43:33 [debug] 13286#13286: *7 run cleanup: 00005FBAEED446E0
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEED6E1E0
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECD83E0
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECD8210
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEED44610, unused: 3
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECF2880, unused: 8
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECB9E60, unused: 0
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECBF230, unused: 8
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECCF4D0, unused: 8
2025/02/04 09:43:33 [debug] 13286#13286: *7 free: 00005FBAEECD3B50, unused: 80
2025/02/04 09:43:33 [debug] 13286#13286: worker cycle
2025/02/04 09:43:33 [debug] 13286#13286: epoll timer: -1

Any help on this would be greatly appreciated.

Thanks

2 Likes
2

Is there a specific need for IP transparency in this usecase? It might be interesting to test the same NGINX configuration with a basic proxy setup without the IP transparency. I believe I have tested a similar setup without the IP transparency and it should work, but that was a long time ago and my memory isn’t great. With the current setup it can be hard to identify if the issue is with the NGINX config or the network setup. Another good troubleshooting step would be to test it with a single node in the upstream group.

3

Hey Damian,

Thanks for the reply.

The reason for enabling transparency is so that ISE can identify the actual IP address of the authenticating user. Without transparency, every connection appears to originate from the Nginx load balancer, meaning all users connecting to ISE appear to have the IP of the load balancer.

I removed transparency from the configuration, but the initial traffic from the client seems to stop at Nginx and is never forwarded to the ISE node.
tester@nginx-proxy-2:/etc/nginx/stream.d$ sudo tcpdump -i any udp port 1812
16:29:27.926152 ens3 In IP ps-sprt-1.test.com.39547 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xb3 length: 182
16:29:33.186988 ens3 In IP ps-sprt-1.test.com.53858 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xb4 length: 182

When testing with a single node in the upstream group, I am able to complete an EAP-TLS authentication as long as proxy responses are disabled. However, if I enable proxy responses, the return traffic from the ISE node reaches the Nginx host but is not forwarded back to the client.

Testing with a single node upstream and proxy_responses 0
tester@nginx-proxy-1:/etc/nginx/stream.d$ sudo tcpdump -i any udp port 1812
16:35:56.532206 ens3 In IP ps-sprt-1.test.com.44668 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xe7 length: 182
16:35:56.532424 ens3 Out IP ps-sprt-1.test.com.44668 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xe7 length: 182
16:35:56.738277 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.44668: RADIUS, Access-Accept (2), id: 0xe7 length: 143
16:36:01.752581 ens3 In IP ps-sprt-1.test.com.51619 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xe8 length: 182
16:36:01.752846 ens3 Out IP ps-sprt-1.test.com.51619 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xe8 length: 182
16:36:01.793505 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.51619: RADIUS, Access-Accept (2), id: 0xe8 length: 143
16:36:06.944339 ens3 In IP ps-sprt-1.test.com.54819 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xe9 length: 182
16:36:06.944549 ens3 Out IP ps-sprt-1.test.com.54819 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xe9 length: 182
16:36:06.984898 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.54819: RADIUS, Access-Accept (2), id: 0xe9 length: 143
16:36:12.133755 ens3 In IP ps-sprt-1.test.com.52348 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xea length: 182
16:36:12.133963 ens3 Out IP ps-sprt-1.test.com.52348 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xea length: 182
16:36:12.174074 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.52348: RADIUS, Access-Accept (2), id: 0xea length: 143
16:36:17.321289 ens3 In IP ps-sprt-1.test.com.49854 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xeb length: 182
16:36:17.321500 ens3 Out IP ps-sprt-1.test.com.49854 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xeb length: 182
16:36:17.361625 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.49854: RADIUS, Access-Accept (2), id: 0xeb length: 143

Testing with a single node upstream and proxy responses 10;
tester@nginx-proxy-1:/etc/nginx/stream.d$ sudo tcpdump -i any udp port 1812
16:36:47.513345 ens3 In IP ps-sprt-1.test.com.59160 > nginx-proxy-1.radius: RADIUS, Access-Request (1), id: 0xfd length: 148
16:36:47.513562 ens3 Out IP ps-sprt-1.test.com.59160 > ISEEAPSN103.test.com.radius: RADIUS, Access-Request (1), id: 0xfd length: 148
16:36:47.516400 ens3 In IP ISEEAPSN103.test.com.radius > ps-sprt-1.test.com.59160: RADIUS, Access-Challenge (11), id: 0xfd length: 155

I wouldn’t rule out a networking issue, but since it works with a single upstream host and MAB connections (which consist of a single request and response) function correctly, I believe the networking is fine. It’s really odd that when proxy_responses is set, Nginx seems unable to handle the responses at all.

Any insights would be greatly appreciated.

Thanks,
Cameron