Best NGINX settings for handling many ESP32 HTTP requests?

[Aria12]

Hi all, I have a quick question: I’m planning to have dozens of ESP32 devices post small sensor payloads to an NGINX edge server and then proxy them to a backend. During my testing of https://www.theengineeringprojects.com/2021/11/create-a-web-server-with-esp32.html, which demonstrates how devices issue HTTP requests, I’m trying to replicate that pattern at scale. Has anyone tuned NGINX for this kind of workload (many small, frequent POSTs)? I’m especially interested in recommended values or pitfalls for worker_connections, client_body_buffer_size, proxy_buffering, keepalive_timeout, and rate-limiting when used with IoT clients, and whether a lightweight MQTT broker in front of the backend would be a better pattern. Any real-world configs or gotchas would be appreciated.

[route443]

Hi @Aria12 ,

Nginx uses event-driven, non-blocking architecture, so you don’t really have hard limits. However, you need to understand not only the number of requests per second but also the nature of the workload in order to choose appropriate hardware and tune the configuration properly.
I don’t know your hardware, network capacity, os, etc, so here are some general recommendations:

1. Start with moderate worker_connections values, something like 4096–8192.
2. Enable multi_accept on.
3. Configure worker_rlimit_nofile, this can be in the range of 100000–200000.
4. Tune keepalive, but don’t set it too high. You need a balance between the cost of handshakes and memory/fd usage.
5. Keep tiny payloads in memory (to avoid temp files). Read the docs for client_max_body_size and client_body_buffer_size.
6. proxy_buffering and proxy_request_buffering should be enabled.
7. nginx will allocate 16KB buffer chunks per connection, in your case this may be 2x higher. So pay attention to client_header_timeout and client_body_timeout (~10s should be ok)

[route443]

@Aria12

rate-limiting when used with IoT clients

regarding limits, I really can’t recommend anything specific, it all depends entirely on your expected load. The only important point is to try to choose a unique key, ideally some custom header that is unique for each esp32 devs. If you can’t find such a header, fall back to the IP addr:

map $http_x_esp_id $iot_key {
    default $http_x_esp_id;
   "" $binary_remote_addr;
}