Rolling Out EKS Clusters with Traefik Proxy as the Ingress Controller
Ingress Controllers are simply reverse proxies tightly tied to the Kubernetes API, allowing them to interact with Ingress resource types. They play a key part in any Kubernetes cluster.
You may think to yourself, why do I need an Ingress Controller? I can just expose services directly using the Service resource! Technically, you can do this, but it will be expensive (especially if using Load Balancers in a cloud environment), and you'll miss out on a lot of features an ingress controller can provide you with, such as:
- Centralized Monitoring: Traefik Proxy supports multiple metrics backends, giving you centralized network statistics for all the applications deployed to your cluster.
- SSL Certificate Management: Traefik Proxy can use Let's Encrypt to generate Certificates for your Ingress routes automatically.
- Load Balancing: Ingress Controllers, like Traefik Proxy, provide a simple way of load balancing pods without paying for a cloud-provided load balancer for each application.
- Authentication: Using Traefik plugins, you can add authentication to any route to help secure your applications.
On EKS, AWS provides an Ingress Controller through the AWS Load Balancer Controller Add-on. When you create an Ingress resource, it creates an Application Load Balancer (ALB); this creates an external load balancer in AWS and configures it based on your Ingress resource. This can get expensive very fast, and you miss out on a lot of features that a more advanced Ingress Controller, such as Traefik Proxy, can provide.
In this guide, I'll be setting up an EKS cluster with Traefik Proxy as the Ingress Controller. I'll walk through provisioning an EKS cluster using eksctl, configuring Traefik Proxy, exposing the dashboard, and setting up a route with authentication.
Prerequisites
To follow this walkthrough, you need to have a few things set up:
Provisioning the cluster
I’ll start by provisioning an EKS cluster using eksctl — there are many ways of provisioning and managing EKS clusters, but today, I’ll be using eksctl for its simplicity. The command below creates an EKS cluster with x2 t3.medium nodes.
$ eksctl create cluster --region eu-west-1 --instance-types t3.medium --name traefik-eks-test
Once the cluster is provisioned (this may take around 10-15 minutes), check that you have access by running kubectl nodes, you should get a similar output to this:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-192-168-15-145.eu-west-1.compute.internal Ready <none> 4m9s v1.22.6-eks-7d68063
ip-192-168-44-23.eu-west-1.compute.internal Ready <none> 4m9s v1.22.6-eks-7d68063
Deploying Traefik Proxy
I’ll be using Helm to deploy Traefik Proxy onto our newly provisioned cluster. Before you do anything, you need to add the Traefik Helm repository to your client:
$ helm repo add traefik https://helm.traefik.io/traefik
$ helm repo update
Now, you’re ready to deploy Traefik Proxy onto your shiny new cluster! You also need to configure Traefik Proxy as your default IngressClass, this way you don’t have to manually specify each time you want to use Traefik Proxy on an Ingress resource.
To do this, you need to add the following values to your helm deployment. You can create a values.yaml file like seen below.
Note: You can find all the default and available values here.
# values.yaml
---
ingressClass:
enabled: true
isDefaultClass: true
fallbackApiVersion: v1
ingressRoute:
dashboard:
enabled: false
service:
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: nlb
globalArguments:
- "--api.insecure=true"
To deploy Traefik proxy, all you need to do is run the below command, this will deploy Traefik Proxy using the Helm chart inside the traefik namespace; it will also create the namespace for you:
$ helm install traefik traefik/traefik --create-namespace --namespace=traefik --values=values.yaml
You can now validate your deployment:
$ kubectl get pods -n traefik
NAME READY STATUS RESTARTS AGE
traefik-7fc5f7dfc7-rmxbv 1/1 Running 0 12m
whoami-75d5976d8d-vmpfw 1/1 Running 0 23s
Configuring Traefik Proxy
Now that Traefik Proxy is up and running, it’s time to start exposing some services. Let’s start by creating an IngressRoute for the dashboard on the cluster. First, you need to create a basic auth middleware to secure your dashboard so that no bad actors can access it.
Create the following 3 YAML files with the below content:
# secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
name: basic-auth-creds
namespace: traefik
type: kubernetes.io/basic-auth
stringData:
username: test
password: password
# middleware.yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: basic-auth
namespace: traefik
spec:
basicAuth:
secret: basic-auth-creds
# ingress-route.yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: dashboard
namespace: traefik
spec:
entryPoints:
- web
routes:
- match: PathPrefix(`/dashboard`, `/dashboard/`) || PathPrefix(`/api`, `/api/`)
kind: Rule
services:
- name: api@internal
kind: TraefikService
middlewares:
- name: basic-auth
namespace: traefik
You can apply the YAML files you created above using:
$ kubectl apply -f middleware.yaml
$ kubectl apply -f ingress-route.yaml
$ kubectl apply -f secret.yaml
Now that you’ve deployed everything, you should be able to access the Traefik dashboard. If you describe services in the traefik namespace, you should see that EKS has created a load balancer for you, this will point to your Traefik instance.
Note: It may take a few minutes for the NLB to begin serving traffic.
$ kubectl get service -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik LoadBalancer 10.100.194.78 a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com 80:31693/TCP,443:31136/TCP 72s
You can now access the Traefik dashboard by visiting the following URL and entering the basic auth credentials you set above:
http://a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com/dashboard/
Exposing a service
Now let’s deploy a basic whoami service and expose it using the Kubernetes Ingress class instead of the Traefik IngressRoute CRD.
IngressRoute vs Ingress
So, you might wonder what the difference between the Ingress and IngressRoute object is. The IngressRoute is a CRD provided by Traefik Proxy which translates directly into Traefik configuration, it’s a lot more powerful and ideal for more complex configurations. Ingress is a Kubernetes native object which is universal no matter which Ingress Controller you decide to deploy, we can use annotations on the Ingress object to manage Traefik features such as middleware, certificates, and entrypoints.
Situations where you may want to use Ingress over IngressRoute would most likely be in a Helm Chart where the chart developer would bundle the Ingress object instead of the Traefik IngressRoute.
Deploying whoami
For the purpose of this walkthrough, I’m going to use traefik/whoami as an example service, it’s a small, lightweight webserver that prints out some debug information.
# whoami.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
namespace: traefik
spec:
replicas: 1
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: traefik/whoami
---
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: traefik
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: whoami
You can deploy it simply by running:
$ kubectl apply -f whoami.yaml
Then you can validate the deployment is running:
$ kubectl get pods -n traefik
NAME READY STATUS RESTARTS AGE
traefik-7fc5f7dfc7-rmxbv 1/1 Running 0 12m
whoami-75d5976d8d-vmpfw 1/1 Running 0 23s
Creating the Ingress
# ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: whoami
namespace: traefik
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: traefik-basic-auth@kubernetescrd
spec:
rules:
- http:
paths:
- path: /whoami
pathType: Prefix
backend:
service:
name: whoami
port:
number: 80
Apply the manifest using:
$ kubectl apply -f ingress.yaml
Validate you can access the newly deployed service:
$ curl -u test:password http://a2c924a3d56184f39b81cc5b0dcb2758-474757780.eu-west-1.elb.amazonaws.com/whoami
Tidy up
You can now tidy up all the resources you’ve created by running:
$ eksctl delete cluster --region eu-west-1 traefik-eks-test
Conclusion
Ingress Controllers are powerful and can give you granular control over the networking on your Kubernetes cluster. Out-of-the-box Ingress Controllers that cloud providers ship with can be a convenient way of getting started. Although, you may find that your needs grow over time and you need a wider feature set. Luckily, it’s usually quite simple to change Ingress Controllers, especially if you’re using the provider-agnostic Ingress resource.
