Migrating for k8s nginx ingress to f5 nginx ingress (ingress api) - no response

Projects NGINX Ingress Controller
1

Hello,

currently doing my first tests in a test cluster for migrating from the soon EOL k8s nginx ingress to f5 ingress controller (non fabric).

The helm chart looks fairly straight forward and i’am deploying it like this

controller:
  defaultHTTPListenerPort: 80
  defaultHTTPSListenerPort: 443
  service:
    externalIPs: 
      - publicIpServer1
      - publicIpServer2
    externalTrafficPolicy: Local

  ingressClass:
    create: true
    name: nginx-external
    setAsDefaultIngress: false

  replicaCount: 2

  enableSnippets: false
  enableCertManager: true

I’am running

  • calico as an CNI with VXLAN enc
  • 2 servers, 2 workers (test cluster only)

When i deploy the ingress controller, i see the pods coming up, i see the services with the right external ips.

But when i try to curl/telnet to 80/443 the connection is just stalled. I see the packets coming in (tcpdump), but no response.

Doing the same from within the server1, telnet on the publicIpServer1 works, but not from an external IP.

For comparison, the setup that works with the k8s nginx ingress has this configuration:

controller:
  ingressClassByName: true
  watchIngressWithoutClass: false
  service:
    externalIPs: 
      - publicIpServer1
      - publicIpServer2
    ports:
      http: 80
      https: 443
    externalTrafficPolicy: Local
    internal:
      externalTrafficPolicy: Local

  ingressClassResource:
    enabled: true
    name: nginx-external
    default: false
    controllerValue: "k8s.io/external-ingress-nginx"

  replicaCount: 2
  allowSnippetAnnotations: false

You can rule out any firewall or network policy issue, since it works with the legacy ingress controller, removing it and just installing the f5 one - not response.

Any ideas what i’am doing wrong?

1 Like
2

hey @EugenMayer , would you mind posting the output of:

kubectl get pods -n <IC_namespace>

kubectl logs <IC_PodName>

kubectl describe ing

you can also create an issue on GitHub · Where software is built

1 Like
3

Sorry cant debug it any longer (do not have it installed), moved on to Traeffik. Since it supports nginx-ingress,ingress,gateway-api and custom CRDs in one deployment, has far superior middlewares it is at least the better choice for us. Also, they support the official gateway standard, not a customer nginx one, which seemed the right choice.

Thank you for helping though!

4

Glad you see you have a path forward. As you have mentioned the GatewayAPI, it is worth noting that Nginx Gateway Fabric is an implementation of that API, not Nginx Ingress Controller. Many users are using the ingress2gateway project to migrate straight to GatewayAPI implementations like NGF. If your current approach throws up unexpected issues, it may be an option worth considering. Best of luck with your migration either way.

5

Thank you for the details

The point is, i can use Traefik Ingress CRD, Ingress-API, Gateway API or even nginx-ingress with Traefik 3.x without deploying anything else - i can use them at the same time and even mix those. Traefik also offers a similar to to analyze the migration and see what annotations could be problematic with The Ultimate Ingress NGINX Migration Kit (Free Tool)

But this or that, the Gateway-API/Fabric is nothing i ill consider. The way certificates are handled for HTTPRoutes is - for my use cases - unusable. Having to re-configure/add a new gateway-listener per HTTPRoute in the gateway-api, instead of just deploying an app that creates a "Ingress” object and that gets a ssl-terminated http ingress router (Ingress-API or Traefic-CRD) - this was a do or die in the end.

In addition, the option to easy define middlewares and combine them with Ingress-API or CRD routes gives a lot of flexibility, without requiring snippet support or having a missing feature, since Ingress-API is frozen and wont get any new feature.

6

We really appreciate the feedback @EugenMayer. These are great points to ensure the roadmap can help for these scenarios.

We made the decision to build a 100% dedicated Gateway API implementation rather than trying to fit Gateway API specifications into NGINX Ingress Controller, as that would add a lot more complexity to the existing controller. This approach also works much better for release cycles, allowing us to have a separate, independent NGINX Gateway Fabric releases that fully conforms to the Gateway API spec.

Regarding annotations, the plan is to support more of them in NGINX Ingress Controller, especially those that are popular in ingress-nginx. We have a list we’re actively working through to help folks migrate. NGINX Ingress Controller has very robust CRDs that we encourage folks to use as an alternative to the Ingress API. These are more mature and don’t rely on annotations (we know what problems arise with those!).

Finally, there’s also a new Gateway API feature called “ListenerSets,” which we’ve roadmapped for this year. It should help with the challenge you mentioned around single Gateway listeners per HTTPRoute. ListenerSets will allow a shared list of listeners to be attached to a single Gateway.

I know you’ve gone with Traefik for now, but I thought it would be worth sharing these updates in case you explore other options in the future. There are some interesting things on the roadmap, and we’re listening closely to feedback. Thanks again for taking the time to evaluate!

3 Likes
7

You never say never! :slight_smile:

Your answer is very appropriated - i’am glad you take the time to elaborate and draw a picture also for every other reader. We all have different needs and use cases and thus also different conclusions.

For that ListenerSets - i cannot see those really make it any better to be honest. I assume what makes it better is one needs to add a new Lister in addition to a HTTPRoute in some kind of CRD (1:1 relation), which i could handle via our terraform modules as a macro. This is practical, but way more complicated and harder to maintain if compared to ingress-api.

Maybe this is a path, i will look at the gateway-api from now and then - but honestely, it is currently just not mature enough. To many things are beta, non RFC, experimental or “yet to be defined” - it seems like it takes another 6-12 months at least to get anyway. But things take time, that’s fine - and some of those will be great than. I will be looking at it!

4 Likes
8

Thanks @EugenMayer ! Understood regarding ListenerSets. I believe they allow listeners to be defined separately from the Gateway resource (1-64 per ListenerSet) while still sharing the same underlying NGINX data plane. It’s experimental right now, but we will implement it at this year to conform to the Gateway API spec.

You’re right, Gateway API is still maturing and a lot of features are experimental, but after the ingress-nginx retirement, we are expecting adoption to increase quite a lot this year, so you’re right to keep it in mind. Please feel free to add any feature requests or issues to our repo. :slight_smile: GitHub - nginx/nginx-gateway-fabric: NGINX Gateway Fabric provides an implementation for the Gateway API using NGINX as the data plane.

Again, thanks for taking the time!

1 Like