Load balancing causing performance degradation

Troubleshooting
1

I tried load balancing , with both default and least-conn distribution, but there is a slight degradation in overall performance of my system.

Setup is as:

React client uploads files to a node backend server which copies them to disk. File count is huge(around 15k) and some file sizes are in range of 1-2 gb. So my approach is to batch files with their cumulative size less than 250mb, and for files bigger than 250mb I upload them as chunks.

Without nginx, time taken is around 1.2 minutes. With nginx and load balancing(2 backend instances), it takes 1.3 mins on average.

What can be the missing link here?

1 Like
2

Hey Manish! I’ve moved your post to the Troubleshooting category so that it gets more traction.

3

Thanks a lot.

1 Like
4

What’s the resource utilization during uploads?

5

Can you share the config that you are using? There are some parameters that can be tuned to help, but this also could be caused by the added hop of transferring the content through nginx. Loadbalancing in this usecase is mostly going to help with resiliency and spreading the load across multiple nodes, and not provide a increase in transfer speeds.

6

Here is my config:

worker_processes  1;

pid ..../logs/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;


    sendfile        on;
    keepalive_timeout  65;
	
	upstream node_backend {
		least_conn;
        server 127.0.0.1:3000;
        server 127.0.0.1:3002;
    }

    server {
        listen       80;
        server_name  localhost;

        location / {
			root   ....\build;
			index  index.html;
			try_files $uri $uri/ /index.html;  # This ensures React routing works
		}
		
		location /upload {
			client_max_body_size 300M;
			proxy_pass http://node_backend/upload;
			proxy_http_version 1.1;
			proxy_set_header Upgrade $http_upgrade;
			proxy_set_header Connection 'upgrade';
			proxy_set_header Host $host;
			proxy_request_buffering off;
			proxy_buffering off;
		}

		location /upload-chunk {
			client_max_body_size 300M;
			proxy_pass http://node_backend/upload-chunk;
			proxy_http_version 1.1;
			proxy_set_header Upgrade $http_upgrade;
			proxy_set_header Connection 'upgrade';
			proxy_set_header Host $host;
			proxy_request_buffering off;
			proxy_buffering off;
		}
		
		....
		....
    }
}
1 Like