Environment
NIC v5.5.x (OSS image), Helm-deployed, single VirtualServer on a host with ~110 routes, most delegated to VirtualServerRoutes in application charts. Many routes are regex (`~*`), which we can’t avoid due to case-insensitivity and locale-prefix requirements.
We’ve spent time tracing how virtual server to virtual server route references behave during coordinated changes (same Helm release updating both objects) and would like to (a) confirm our reading is correct and (b) hear how others handle this.
What We Believe We’ve Established
1. Regex/exact delegation requires an exact path-set match.
For prefix routes, subroutes must be contained by the parent path. But for regex and exact routes, validation requires the paths to match bidirectionally – every VS route path must have a matching subroute and vice versa. A regex path change must touch both objects, byte-identically.
2. There is no easy order-safe way to change a shared regex path.
Example: we want to change a route like `~*/user/(\d+)/profile` to `~*/user/(\d+)/(profile|edit)`
Updating the virtual server first fails validation against the old virtual server route; updating the virtual server route first fails against the old virtual server. Since two objects can’t be written atomically in Kubernetes, every coordinated regex path change transits an invalid intermediate state in *any* write order.
The only way around this we have found is to:
- first add in a new virtual server route and deploy it.
- update the virtual server to point at the new virtual server route and remove the old reference, deploy
- remove the old virtual server route and deploy.
This is rather cumbersome to handle manually and difficult to automate across multiple repositories. It’s also an extremely common operation for our teams.
3. One invalid subroute rejects the whole virtual server route.
Using new regex subroutes still have several of these limitations, and by placing them in the same virtual server route, it seems like validation fails for all routes when it fails for any of them. So this just increases the blast radius of a configuration error.
4. Reload batching narrows but does not close the window.
The design doc says reloads pause “When NGINX Ingress Controller LTS receives a number of concurrent requests from the Kubernetes API, it will pause NGINX reloads until the task queue is empty.” That’s reactive:
If the worker dequeues the virtual server event before the virtual server write reaches the informer (Helm applies them sequentially, ~one API round-trip apart), there’s nothing to batch — the intermediate config reloads immediately, and a second reload fixes it moments later. Each controller replica races independently, so the result is a partial brownout across pods. Counterintuitively, an *idle* controller is more likely to hit this than a busy one. Is this a correct assessment?
Ultimately, depending on this isn’t a guarantee of no downtime if we deploy the virtual server route and updated virtual server together when changing the paths on both together.
Questions
1. Can maintainers/others confirm intended behavior?
2. Is there a recommended procedure for changing a regex path that’s delegated to a VSR without a traffic-loss window? If the VS and VSR are deployed separately, the workflow in (2) requires three separate deployments to update a regex. This is a very common operation for our delegated teams.
3. How are others running regex-heavy, multi-team routing on NIC?
Patterns we’re considering
-
keeping all regex routes directly in the Virtual server (atomic, but single-repo ownership)
-
prefix-only delegation with edge-side case normalization. (Could require a lot of change)