Traffic Split for K8s dashboard using oauth2-proxy using NGF

[khannasumit961]

Hi Team,

I have a requirement where i need to deploy canary deployments for my K8s dashboard which uses authentication as (oauth2-proxy) and i want to split the traffic like canary between stable and canary version. Below are my findings and I would really appreciate any leads.

User → HTTPRoute → kube-oauth2-proxy (Service, port 80) → kubernetes-dashboard (Service, port 443)

My HTTPRoute only points to kube-oauth2-proxy. The actual dashboard canary is hidden behind the proxy’s --upstream configuration. I want the canary logic to apply to the dashboard, not the proxy.

• My oauth2-proxy only knows about the kubernetes-dashboard service (via its --upstream flag).

• The HTTPRoute only routes to the proxy, not the dashboard services directly.

• And Ido NOT want to change the oauth2-proxy config or run multiple oauth2-proxy instances.

So I asked chatgpt and it suggested this:

You cannot use HTTPRoute’s weighted backendRefs to split dashboard traffic:
Your HTTPRoute only sees the oauth2-proxy, so all splitting is “hidden” behind that proxy.

• You cannot do a true traffic split between two dashboard versions at the HTTPRoute level without also splitting at the proxy level.

• If you want a true, precise canary split for the dashboard (not the proxy), you must:

  1. Split the traffic at the Service or ServiceMesh level (using TrafficSplit, which you do not have), OR

  2. Run two oauth2-proxy Deployments and split at the HTTPRoute (not what you want), OR

  3. Use the pod-weighted canary approach behind a single Service (less precise).

Your Options Based on Current Setup

Option 1: Pod-Weighted Canary (Simple, Not Precise)

• Deploy both dashboard versions with the same selector/labels, both behind the kubernetes-dashboard Service.

• Control the traffic split by the ratio of pods (e.g., 4 old + 1 canary ≈ 80/20).

• Drawback: NGINX’s “least_conn” means the split won’t be exact.

Option 2: HTTPRoute Split (Precise, Needs Two Proxies)

• Deploy two oauth2-proxy instances:

  • One upstreams to kubernetes-dashboard

  • One upstreams to kubernetes-dashboard-canary-kong-proxy

• Modify your HTTPRoute to split between the two oauth2-proxy services, e.g.:

  backendRefs:
- name: kube-oauth2-proxy
port: 80
weight: 90
- name: kube-oauth2-proxy-canary
port: 80
weight: 10

• Drawback: More complex; requires canary oauth2-proxy deployment.

Option 3: SMI TrafficSplit (Precise, Needs SMI Support)

• If you can install and use SMI TrafficSplit (with a service mesh), you can split traffic at the Service level without touching the proxy or HTTPRoute.

  Could you please suggest? When I install using traffic split, it throws this error:

  k apply -f trafficsplit.yaml
  error: resource mapping not found for name: “kubernetes-dashboard” namespace: “kubernetes-dashboard” from “trafficsplit.yaml”: no matches for kind “TrafficSplit” in version “split.smi-spec.io/v1alpha2”
  ensure CRDs are installed first

  cat trafficsplit.yaml
  apiVersion: split.smi-spec.io/v1alpha2
  kind: TrafficSplit
  metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
  spec:
  service: kubernetes-dashboard
  backends:

  • service: kubernetes-dashboard
    weight: 90

  • service: kubernetes-dashboard-canary-kong-proxy
    weight: 10

[salonichf5]

Hey @khannasumit961 thanks for opening this discussion.

To summarize: you’d like NGF to send requests to an external authorization proxy first, and then forward authorized requests to the backend applications (as you confirmed). Today, NGF doesn’t natively support externalAuth, but we’re actively discussing its prioritization. The tracking issue is here

In the meantime, you can achieve external auth in your current NGF setup using SnippetsFilter. This relies on the NGINX http_auth_request module to implement the external authorization check: ngx_http_auth_request

We have this module enabled in our NGINX builds, so you should be able to start using it right away. If you’re able to get it working, please share details—this would help others looking for the same solution until we support it natively in NGF.

@Micheal_Kingston

[Nikoolayy1]

Yes that will be great as in the future I may have a use case as well!