Rolling Out EKS Clusters with Traefik Proxy as the Ingress Controller

Ingress Controllers are simply reverse proxies tightly tied to the Kubernetes API, allowing them to interact with Ingress resource types. They play a key part in any Kubernetes cluster.

You may think to yourself, why do I need an Ingress Controller? I can just expose services directly using the Service resource! Technically, you can do this, but it will be expensive (especially if using Load Balancers in a cloud environment), and you'll miss out on a lot of features an ingress controller can provide you with, such as:

• Centralized Monitoring: Traefik Proxy supports multiple metrics backends, giving you centralized network statistics for all the applications deployed to your cluster.
• SSL Certificate Management: Traefik Proxy can use Let's Encrypt to generate Certificates for your Ingress routes automatically.
• Load Balancing: Ingress Controllers, like Traefik Proxy, provide a simple way of load balancing pods without paying for a cloud-provided load balancer for each application.
• Authentication: Using Traefik plugins, you can add authentication to any route to help secure your applications.

On EKS, AWS provides an Ingress Controller through the AWS Load Balancer Controller Add-on. When you create an Ingress resource, it creates an Application Load Balancer (ALB); this creates an external load balancer in AWS and configures it based on your Ingress resource. This can get expensive very fast, and you miss out on a lot of features that a more advanced Ingress Controller, such as Traefik Proxy, can provide.

In this guide, I'll be setting up an EKS cluster with Traefik Proxy as the Ingress Controller. I'll walk through provisioning an EKS cluster using eksctl, configuring Traefik Proxy, exposing the dashboard, and setting up a route with authentication.

Prerequisites

To follow this walkthrough, you need to have a few things set up:

• eksctl
• helm
• kubectl

Provisioning the cluster

I’ll start by provisioning an EKS cluster using eksctl — there are many ways of provisioning and managing EKS clusters, but today, I’ll be using eksctl for its simplicity. The command below creates an EKS cluster with x2 t3.medium nodes.

$ eksctl create cluster --region eu-west-1 --instance-types t3.medium --name traefik-eks-test

Once the cluster is provisioned (this may take around 10-15 minutes), check that you have access by running kubectl nodes, you should get a similar output to this:

$ kubectl get nodes
NAME                                           STATUS   ROLES    AGE    VERSION
ip-192-168-15-145.eu-west-1.compute.internal   Ready    <none>   4m9s   v1.22.6-eks-7d68063
ip-192-168-44-23.eu-west-1.compute.internal    Ready    <none>   4m9s   v1.22.6-eks-7d68063

Deploying Traefik Proxy

I’ll be using Helm to deploy Traefik Proxy onto our newly provisioned cluster. Before you do anything, you need to add the Traefik Helm repository to your client:

$ helm repo add traefik https://helm.traefik.io/traefik
$ helm repo update

Now, you’re ready to deploy Traefik Proxy onto your shiny new cluster! You also need to configure Traefik Proxy as your default IngressClass, this way you don’t have to manually specify each time you want to use Traefik Proxy on an Ingress resource.

To do this, you need to add the following values to your helm deployment. You can create a values.yaml file like seen below.

Note: You can find all the default and available values here.

# values.yaml
---
ingressClass:
  enabled: true
  isDefaultClass: true
  fallbackApiVersion: v1
ingressRoute:
  dashboard:
    enabled: false
service:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
globalArguments:
- "--api.insecure=true"

To deploy Traefik proxy, all you need to do is run the below command, this will deploy Traefik Proxy using the Helm chart inside the traefik namespace; it will also create the namespace for you:

$ helm install traefik traefik/traefik --create-namespace --namespace=traefik --values=values.yaml

You can now validate your deployment:

$ kubectl get pods -n traefik
NAME                       READY   STATUS    RESTARTS   AGE
traefik-7fc5f7dfc7-rmxbv   1/1     Running   0          12m
whoami-75d5976d8d-vmpfw    1/1     Running   0          23s

Configuring Traefik Proxy

Now that Traefik Proxy is up and running, it’s time to start exposing some services. Let’s start by creating an IngressRoute for the dashboard on the cluster. First, you need to create a basic auth middleware to secure your dashboard so that no bad actors can access it.

Create the following 3 YAML files with the below content:

# secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: basic-auth-creds
  namespace: traefik
type: kubernetes.io/basic-auth
stringData:
  username: test
  password: password

# middleware.yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: basic-auth
  namespace: traefik
spec:
  basicAuth:
    secret: basic-auth-creds

# ingress-route.yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: dashboard
  namespace: traefik
spec:
  entryPoints:
    - web
  routes:
    - match: PathPrefix(`/dashboard`, `/dashboard/`) || PathPrefix(`/api`, `/api/`)
      kind: Rule
      services:
        - name: api@internal
          kind: TraefikService
      middlewares:
        - name: basic-auth
          namespace: traefik

You can apply the YAML files you created above using:

$ kubectl apply -f middleware.yaml
$ kubectl apply -f ingress-route.yaml
$ kubectl apply -f secret.yaml

Now that you’ve deployed everything, you should be able to access the Traefik dashboard. If you describe services in the traefik namespace, you should see that EKS has created a load balancer for you, this will point to your Traefik instance.

Note: It may take a few minutes for the NLB to begin serving traffic.

$ kubectl get service -n traefik
NAME      TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)                      AGE
traefik   LoadBalancer   10.100.194.78   a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com   80:31693/TCP,443:31136/TCP   72s

You can now access the Traefik dashboard by visiting the following URL and entering the basic auth credentials you set above:

http://a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com/dashboard/

Exposing a service

Now let’s deploy a basic whoami service and expose it using the Kubernetes Ingress class instead of the Traefik IngressRoute CRD.

IngressRoute vs Ingress

So, you might wonder what the difference between the Ingress and IngressRoute object is. The IngressRoute is a CRD provided by Traefik Proxy which translates directly into Traefik configuration, it’s a lot more powerful and ideal for more complex configurations. Ingress is a Kubernetes native object which is universal no matter which Ingress Controller you decide to deploy, we can use annotations on the Ingress object to manage Traefik features such as middleware, certificates, and entrypoints.

Situations where you may want to use Ingress over IngressRoute would most likely be in a Helm Chart where the chart developer would bundle the Ingress object instead of the Traefik IngressRoute.

Deploying whoami

For the purpose of this walkthrough, I’m going to use traefik/whoami as an example service, it’s a small, lightweight webserver that prints out some debug information.

# whoami.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: traefik
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: whoami

You can deploy it simply by running:

$ kubectl apply -f whoami.yaml

Then you can validate the deployment is running:

$ kubectl get pods -n traefik
NAME                       READY   STATUS    RESTARTS   AGE
traefik-7fc5f7dfc7-rmxbv   1/1     Running   0          12m
whoami-75d5976d8d-vmpfw    1/1     Running   0          23s

Creating the Ingress

# ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  namespace: traefik
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
    traefik.ingress.kubernetes.io/router.middlewares: traefik-basic-auth@kubernetescrd
spec:
  rules:
  - http:
      paths:
      - path: /whoami
        pathType: Prefix
        backend:
          service:
            name: whoami
            port:
              number: 80

Apply the manifest using:

$ kubectl apply -f ingress.yaml

Validate you can access the newly deployed service:

$ curl -u test:password http://a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com/whoami

Tidy up

You can now tidy up all the resources you’ve created by running:

$ eksctl delete cluster --region eu-west-1 traefik-eks-test

Conclusion

Ingress Controllers are powerful and can give you granular control over the networking on your Kubernetes cluster. Out-of-the-box Ingress Controllers that cloud providers ship with can be a convenient way of getting started. Although, you may find that your needs grow over time and you need a wider feature set. Luckily, it’s usually quite simple to change Ingress Controllers, especially if you’re using the provider-agnostic Ingress resource.

Whitepaper: Making the Most of Kubernetes with Cloud Native NetworkingExplore key traffic management strategies for success with microservices in K8s environments.Download It Now