Kubernetes Ingress & Service API Demystified

The Ingress Object itself already has a long history with K8s. It is still considered beta, which is kinda surprising for something that has been so long present in K8s. But why is that? And when will that change?

With the release of Kubernetes 1.18, some improvements have been made to Ingress, which have been overdue for a long time. However, the changes introduced are minor, and some of the issues we’ll be covering in this blog post have gone untackled. In addition to covering the issues mentioned above, we’ll be exploring the new Service API aimed at solving these issues.

Issues

In this blog post we’ll cover some of the long-standing issues with the current state of Ingress in Kubernetes, including these topics:

Inflexible HTTP routing definitions
Schema differences across vendors
Extensibility of Ingress

Inflexible HTTP routing definitions

A simple Ingress object example is the following:

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
spec:
  rules:
  - host: myhost.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80

The above Ingress object will route HTTP requests with the URI GET http://myhost.com/testpath and forward the request internally to the service called test on port 80. So far, so good.

The primary focus of an ingress resource is on solving simple HTTP routing cases, similar to the concept of Virtual Hosts with a Path Based routing extension. This leads us to the first issue: How would you configure cases like a redirect?

Schema differences across vendors

Further configuration is usually done through annotations on the Ingress resource. Annotations are like key-value pairs stored in the metadata of an object.

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: myhost.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80

The above example shows that the Ingress Controller (nginx) would rewrite all requests to / (slash) before forwarding to the backend.

Consider that the same configuration in Traefik would look like this:

--- 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata: 
  name: rewrite-slash
spec: 
  replacePath: 
    path: /
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: whoami-ingress
  annotations:
    kubernetes.io/ingress.class: traefik              
    traefik.ingress.kubernetes.io/router.middlewares: default-rewrite-slash@kubernetescrd
spec:
  rules:
  - host: whoami.localhost
    http:
      paths:
      - path: /foobar
        backend:
          serviceName: whoami
          servicePort: web

As you can see, it’s totally different! Which brings us to another issue: extensibility.

Extensibility of Ingress

Extending Ingress Objects is a requirement and the de-facto standard for that is using annotations. However, annotations often differ between the many different implementations of Ingress Controllers and it's therefore hard to manage for an end-user.

This unmanageability also translates back to the providers who must maintain their ingress controller implementation, who are often constrained by the simplicity of the key / value pair approach.

Service API aka Ingress V2

Announced at Kubecon NA in 2019 by Google there has been substantial effort in creating an “Ingress V2” which is now known as the Service API.

This specification aims to solve a few problems:

Provide clean separation and role-based control
Uplevel the Ingress specification
Specify standard methods of extending the Ingress specification

To do so, the specification currently consists of 4 different CRD Ressources:

GatewayClass

The GatewayClass is meant to be a Cluster Scoped resource, which is meant to represent a “category” of gateways. It’s similar to the former `ingress.class` annotation or the now included IngressClass resource.

The expected use-case is to have more than one GatewayClass per Ingress Controller provider. These classes may have a variety of default settings, which are inherited by the cluster-level gateway. Also, this can be used to pass additional configuration down to that gateway. Since it is a cluster scoped resource, it's expected to be managed by the Infrastructure provider.

---
kind: GatewayClass
metadata:
  name: cluster-gateway
spec:
  controller: "acme.io/gateway-controller"
  parametersRef:
    apiVersion: core/v1
    kind: ConfigMap
    namespace: acme-system
    name: internet-gateway

Gateway

The Gateway has a life-cycle which is tied with the infrastructure. For instance, one Gateway could be running an instance of Traefik or one AWS ELB. As already mentioned, it’s linked to a Gateway class for inferring configuration.

The gateway sets listener bindings (Address, Ports, TLS…) and the routes served by the Gateway.

---
kind: Gateway
name: my-app-gw
namespace: my-app
spec:
  class: from-internet
  listeners:
  - address:
      ip: 1.2.3.4
    protocols: ["http"] # implies port 80.
    routes:
      ...
  - address:
      ip: 1.2.3.4
    protocols: ["https"] # implies port 443
    certificates:
    - name: my-secret
    - apiGroup: certmanager.k8s.io
      kind: Certificate               
      name: lets-encrypt-cert
    routes:
      - route:
          name: http-app-1
          namespace: app-1
          kind: HTTPRoute

It also has some sane default protocols like: HTTP, HTTPS, TCP, etc. which map to predefined ports.

Route

Last but not least, a route is used to describe a way to handle traffic given protocol level descriptions.

A route can be of a different resource (HTTPRoute, TLSRoute, TCPRoute…) and can therefore have different protocol level descriptions taken into consideration for routing. Each of these protocols have different attributes which could be used for route matching. It can also be used to delegate to other Route resources in a multi-tenant scenario. For example, one team might offer a global authentication service to which you want to forward requests from within your current scope.

---
kind: HTTPRoute
name: delegate-1
namespace: other-team
rules:
- match:
    http:
      host: bar.com
      path:
        prefix: /store
  action:
    backend:                           
      name: delegate-1                  
      namespace: other-team             
      kind: HTTPRoute
      apiGroup: networking.k8s.io

In the above HTTPRoute example it’s listening for traffic with a host of bar.com and looking for a path prefix of /store, which is close to some of the examples we’ve explored on traditional Ingress but allows for some additional specifics.

Extensibility

Taking what we just learned to the next level, the question that arrives now is: “How can we extend that, for example, to implement rewrites?”

For that purpose, the specification currently offers three different levels of support:

Core
Extended
Custom

Core functionality is guaranteed between all solutions respecting a specific set of API’s. Extended is a standardised API, but functionality is not guaranteed between all solutions.

For special use-cases, which might be vendor specific, there’s a custom layer where the implementation can be custom built to meet specific requirements. Usually, these will end up in CRD’s or custom annotations.

Conclusion

Service API is a new set of forward-looking API’s attempting to solve some issues that have become apparent over the evolving usage of Ingress. However, as it’s a bit more complex and might not solve all the simple use cases Ingress is capable of solving, it’s not meant to replace Ingress but rather provide an alternative for complex use cases.

Eventually, Traefik itself will support the new Service API spec. In the meantime, we offer support for both native Kubernetes Ingress and have extended support for Ingress through the use of CRDs. Learn more about how we do both and empower developers with flexible and easy to use Kubernetes Ingress.