Getting Started with Traefik Proxy and the New Kubernetes Gateway API 0.4
It was in June 2020 when we first discussed the movement inside the Kubernetes community to develop an improved method of defining and managing ingress traffic for Kubernetes. We introduced initial support for the Gateway API in Traefik Proxy 2.4 in January 2021 and exactly one year ago we published our first getting started guide on how to use the Gateway API, GatewayClass, and HTTPRoute APIs with Traefik Proxy to route requests to services.
With the release of Traefik Proxy 2.6 and support for the latest version of Gateway API, we thought it was about time we updated this guide to include all the latest information.
In this article, you’ll find a summary of all the changes introduced in Gateway API 0.4, a walkthrough on how to install and configure Traefik Proxy to use the Gateway API, as well as how to migrate your services from v1alpha1 to v1alpha2.
Let’s get started!
Kubernetes Gateway API 0.4: what has changed?
In October 2021, the new release for Gateway API went live. The 0.4 release introduced big updates including the change of the API version from v1alpha1 to v1alpha2. The new release also brought a few breaking changes, highly-anticipated additions, and some bug fixes. Below is a quick summary of the most important changes.
Version and APIGroup changes — The API version of the Gateway API has been changed from v1alpha1 to v1alpha2. Also, the APIGroup has moved from networking.x-k8s.io to gateway.networking.k8s.io.
Gateway-route binding improvements — The 0.4 release introduces three major change:
- Gateways can now specify what kind of routes are allowed to be attached to that gateway, and where they are located.
- Routes have to directly attach to a gateway (and optionally to a listener).
- The route becomes attached only when the specifications intersect.
Safe(r) cross-namespace references — Following CVE-2021-25740, the Gateway API needed some changes in the process of cross-namespace references. In the 0.4 version, references across namespaces are not allowed by default. The only way to get cross-namespace references working is by creating a Reference Policy in the target namespace. This reference policy needs to specify what references are accepted.
No more certificate references on Routes — Previously, HTTPRoute objects had a way to reference TLS Key Pairs. This was done to allow app owners to provide TLS Key Pairs inside their own (or other) namespace and use them on their own routes. However, there were plenty of edge-cases which were hard to overcome with that model. WIth v1alpha2, if you want to have those references again, you will have to re-use the ReferencePolicy model mentioned above. By utilizing that, you can create secure cross namespace references for TLS certificates which should hit most of those edge cases.
RouteForwardTo to BackendRef — Previously, on an HTTPRoute, you were able to use the RouteForward to select where things should be forwarded to. This has been replaced with a new backend ref, to have a clear definition upstream to make it clear where to put connections to.
Object Policy — This feature allows you to add policies to objects. It is an easy to use mechanismen that enables implementations, such as Traefik Proxy, to work on a given set of policies on a Route, and do what is configured. A first class example, as shown in the Gateway API docs is a Timeout Policy.
There are many more small changes and additions included in this release. You can take a look at the full release notes for the detailed list.
Traefik Proxy 2.6 and Gateway API 0.4
Now that you had a look at the Gateway API 0.4 release highlights, let’s get started and see how you can use the new Gateway API with the latest Traefik Proxy version.
Prerequisites
To follow through with this guide, you'll need some things before you begin:
- A working Kubernetes cluster, which this guide assumes is running on
localhost - The
kubectlcommand-line tool, installed and configured to access your cluster
You do not need to have Traefik installed on your cluster before you begin. You'll take care of that in the next few steps.
You'll also want the set of configuration files that accompanies this post, which are available on GitHub. Clone this repository:
git clone https://github.com/traefik-tech-blog/k8s-gateway-api
Installing the CRDs
Installing the CRDs
Gateway API is not installed on Kubernetes clusters by default. Support for the clusters depends upon a set of custom resource definitions (CRDs), and you should make sure these are installed before enabling support in Traefik Proxy.
It's a safe bet to use the latest version 0.4.0:
kubectl apply -k "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.4.0"
Install and configure Traefik Proxy to use the Gateway API
To install Traefik Proxy v2.6 and have it configured to enable the new provider, the best way is to use the official Helm chart:
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install traefik --set experimental.kubernetesGateway.enabled=true traefik/traefik
Note: the --set experimental.kubernetesGateway.enabled=true flag. This will install Traefik 2.6, enable the new Gateway API provider, and also create GatewayClasses and a Gateway instance.
More customization options for the installation, such as the label selector, or TLS certificates (which you'll use later in this guide) are visible in the Helm chart's values file.
To verify that the new features are enabled, use port forwarding to expose the Traefik Proxy dashboard:
kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name) 9000:9000
When you use your browser to access the dashboard via http://localhost:9000/dashboard/, you should see that the KubernetesGateway provider is activated and ready to serve.
Setup a dummy service
You'll need a target where Traefik Proxy can route requests, so quickly install the famous whoami service using kubectl apply -f, in order to have something to use for testing purposes:
# 01-whoami.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
spec:
replicas: 2
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: traefik/whoami:v1.6.0
ports:
- containerPort: 80
name: http
---
apiVersion: v1
kind: Service
metadata:
name: whoami
spec:
selector:
app: whoami
ports:
- port: 80
targetPort: http
Deploying a simple host
Now that everything is set up, you're ready to start the action! Where previously you would have created an Ingress or IngressRoute, here you'll deploy your first simple HTTPRoute:
# 02-whoami-httproute.yaml
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
name: http-app-1
namespace: default
spec:
parentRefs:
- name: traefik-gateway
hostnames:
- whoami
rules:
- matches:
- path:
type: Exact
value: /
backendRefs:
- name: whoami
port: 80
weight: 1
This HTTPRoute will catch requests made to the whoami hostname and forward them to the whoami service you deployed earlier. If you now make a request for that hostname, you will see typical whoami output, which looks something like this:
curl -H "Host: whoami" http://localhost
Hostname: whoami-9cdc57b6d-pfpxs
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.13
IP: fe80::9c1a:a1ff:fead:2663
RemoteAddr: 10.42.0.11:33658
GET / HTTP/1.1
Host: whoami
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-74d7f586dd-xxr7r
X-Real-Ip: 10.42.0.1
Simple host with paths
The example above can easily be enhanced to only react on a given subpath:
# 03-whoami-httproute-paths.yaml
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
name: http-app-1
namespace: default
spec:
parentRefs:
- name: traefik-gateway
hostnames:
- whoami
rules:
- matches:
- path:
type: Exact
value: /foo
backendRefs:
- name: whoami
port: 80
weight: 1
With the modified HTTPRoute, you'll see that the previous request now returns a 404 error, while requesting the /foo path suffix returns success:
curl -H "Host: whoami" http://localhost/foo
Hostname: whoami-9cdc57b6d-pfpxs
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.13
IP: fe80::9c1a:a1ff:fead:2663
RemoteAddr: 10.42.0.11:34424
GET /foo HTTP/1.1
Host: whoami
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-74d7f586dd-xxr7r
X-Real-Ip: 10.42.0.1
More information about what parts of a request can be matched can be found in the official Service API documentation.
TLS with static certificates
So far, you have created a simple HTTPRoute. For the next step, you'll want to secure this Route through TLS. For that, you need to create a Kubernetes Secret with a dummy certificate. You'll find one included in the configuration files in this post's GitHub repository:
# 04-tls-dummy-cert.yaml
---
apiVersion: v1
kind: Secret
metadata:
name: mysecret
namespace: default
type: kubernetes.io/tls
data:
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVVVENDQXJtZ0F3SUJBZ0lRV2pNZ2Q4OUxOUXIwVC9WMDdGR1pEREFOQmdrcWhraUc5dzBCQVFzRkFEQ0IKaFRFZU1Cd0dBMVVFQ2hNVmJXdGpaWEowSUdSbGRtVnNiM0J0Wlc1MElFTkJNUzB3S3dZRFZRUUxEQ1JxWW1SQQpaSEpwZW5wMElDaEtaV0Z1TFVKaGNIUnBjM1JsSUVSdmRXMWxibXB2ZFNreE5EQXlCZ05WQkFNTUsyMXJZMlZ5CmRDQnFZbVJBWkhKcGVucDBJQ2hLWldGdUxVSmhjSFJwYzNSbElFUnZkVzFsYm1wdmRTa3dIaGNOTWpBeE1qQTAKTVRReE1qQXpXaGNOTWpNd016QTBNVFF4TWpBeldqQllNU2N3SlFZRFZRUUtFeDV0YTJObGNuUWdaR1YyWld4dgpjRzFsYm5RZ1kyVnlkR2xtYVdOaGRHVXhMVEFyQmdOVkJBc01KR3BpWkVCa2NtbDZlblFnS0VwbFlXNHRRbUZ3CmRHbHpkR1VnUkc5MWJXVnVhbTkxS1RDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUIKQU12bEc5d0ZKZklRSWRreDRXUy9sNGhQTVRQcmVUdmVQOS9MZlBYK2h2ekFtVC90V1BJbGxGY2JJNnZzemp0NQpEWlZUMFFuQzhHYzg0K1lPZXZHcFpNaTg0M20zdTdFSUlmY3dETUF4WWQ0ZjJJcENLVW9jSFNtVGpOaVhDSnhwCjVNd2tlVXdEc1dvVVZza1RxeVpOcWp0RWVIbGNuQTFHaGZSa3dEUkZxd1QxeVhaUTBoZHpkQzRCeFhhaVk0VEQKaFQ1dnFXQmlnUlh0M1VwSkhEL1NXUG4wTEVQOHM3ckhjUkZPY0RhV3ZWMW1jTkxNZUpveWNYUTJ0M2Z1Q0Fsegp3UWZOSjFQSk45QWlLalFJcXJ1MGFnMC9wU0kyQ3NkbEUzUTFpM29tZGpCQkZDcmxNMTZyY0wwNDdtWXZKOEVvCjFMdDVGQkxnVURBZktIOFRsaXU0ZG9jQ0F3RUFBYU5wTUdjd0RnWURWUjBQQVFIL0JBUURBZ1dnTUJNR0ExVWQKSlFRTU1Bb0dDQ3NHQVFVRkJ3TUJNQXdHQTFVZEV3RUIvd1FDTUFBd0h3WURWUjBqQkJnd0ZvQVV5cWNiZGhDego3Nm4xZjFtR3BaemtNb2JOYnJ3d0VRWURWUjBSQkFvd0NJSUdkMmh2WVcxcE1BMEdDU3FHU0liM0RRRUJDd1VBCkE0SUJnUUFzWlBndW1EdkRmNm13bXR1TExkWlZkZjdYWk13TjVNSkk5SlpUQ1NaRFRQRjRsdG91S2RCV0gxYm0Kd003VUE0OXVWSHplNVNDMDNlQ294Zk9Ddlczby94SFZjcDZGei9qSldlYlY4SWhJRi9JbGNRRyszTVRRMVJaVApwNkZOa3kvOEk3anF1R2V2b0xsbW9KamVRV2dxWGtFL0d1MFloVCtudVBJY1pGa0hsKzFWOThEUG5WaTJ3U0hHCkIwVU9RaFdxVkhRU0RzcjJLVzlPbmhTRzdKdERBcFcwVEltYmNCaWlXOTlWNG9Ga3VNYmZQOE9FTUY2ZXUzbW0KbUVuYk1pWFFaRHJUMWllMDhwWndHZVNhcTh1Rk82djRwOVVoWHVuc3Vpc01YTHJqQzFwNmlwaDdpMTYwZzRWawpmUXlYT09KY0o2WTl2a2drYzRLYUxBZVNzVUQvRDR1bmd6emVWQ3k0ZXhhMmlBakpzVHVRS3JkOFNUTGNNbUJkCnhtcXVKZXFWSEpoZEVMNDBMVGtEY1FPM1NzOUJpbjRaOEFXeTJkdkR1a1gwa084dm9IUnN4bWVKcnVyZ09MVmIKamVvbTVQMTVsMkkwY3FKd2lNNHZ3SlBsb25wMTdjamJUb0IzQTU5RjZqekdONWtCbjZTaWVmR3VLM21hVWdKegoxWndjamFjPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRREw1UnZjQlNYeUVDSFoKTWVGa3Y1ZUlUekV6NjNrNzNqL2Z5M3oxL29iOHdKay83Vmp5SlpSWEd5T3I3TTQ3ZVEyVlU5RUp3dkJuUE9QbQpEbnJ4cVdUSXZPTjV0N3V4Q0NIM01BekFNV0hlSDlpS1FpbEtIQjBwazR6WWx3aWNhZVRNSkhsTUE3RnFGRmJKCkU2c21UYW83UkhoNVhKd05Sb1gwWk1BMFJhc0U5Y2wyVU5JWGMzUXVBY1Yyb21PRXc0VStiNmxnWW9FVjdkMUsKU1J3LzBsajU5Q3hEL0xPNngzRVJUbkEybHIxZFpuRFN6SGlhTW5GME5yZDM3Z2dKYzhFSHpTZFR5VGZRSWlvMApDS3E3dEdvTlA2VWlOZ3JIWlJOME5ZdDZKbll3UVJRcTVUTmVxM0M5T081bUx5ZkJLTlM3ZVJRUzRGQXdIeWgvCkU1WXJ1SGFIQWdNQkFBRUNnZ0VCQUl5SWpvbzQxaTJncHVQZitIMkxmTE5MK2hyU0cwNkRZajByTVNjUVZ4UVEKMzgvckZOcFp3b1BEUmZQekZUWnl1a1VKYjFRdUU2cmtraVA0S1E4MTlTeFMzT3NCRTVIeWpBNm5CTExYbHFBVwpEUmRHZ05UK3lhN2xiemU5NmdaOUNtRVdackJZLzBpaFdpdmZyYUNKK1dJK1VGYzkyS1ZoeldSa3FRR2VYMERiCnVSRXRpclJzUXVRb1hxNkhQS1FIeUVITHo2aWVVMHJsV3IyN0VyQkJ4RlRKTm51MnJ1MHV1Ly8wdG1SYjgzZWwKSUpXQnY1V1diSnl4dXNnMkhkc0tzTUh0eEVaYWh1UlpTNHU2TURQR3dSdjRaU0xpQm1FVVc3RUMwUEg3dCtGaAoxUDcrL0Yyd1pGSDAvSzl6eXUyc0lOMDJIbTBmSWtGejBxb09BSzQ5OXhrQ2dZRUE2SC9nVUJoOG9GUSt2cmZKCnQvbXdMeFBHZHhWb3FWR1hFVjhlQzNWbmxUSXJlREpNWm81b1hKZHNuQ0d2S1NaWUhXZ3o3SVpwLzRCL29vSWsKTDl4TEJSVTJwS0d1OGxBT1ZhYnpaVDk0TTZYSE1PTGQ0ZlUrS3ZqK1lLVm5laEM3TVNQL3RSOWhFMjN1MnRKZwp1eUdPRklFVlptNHZxS1hEelU3TTNnU0R5WXNDZ1lFQTRJRVFyZDl2MXp0T2k5REZ6WEdnY05LVmpuYmFTWnNXCm9JNm1WWFJZS1VNM1FyWUw4RjJTVmFFM0Y0QUZjOXRWQjhzV0cxdDk4T09Db0xrWTY2NjZqUFkwMXBWTDdXeTMKZXpwVEFaei9tRnc2czdic3N3VEtrTW5MejVaNW5nS3dhd3pRTXVoRGxLTmJiUi90enRZSEc0NDRrQ2tQS3JEbQphOG40bUt6ZlRuVUNnWUFTTWhmVERPZU1BS3ZjYnpQSlF6QkhydXVFWEZlUmtNSWE2Ty9JQThzMGdQV245WC9ICk12UDE4eC9iNUVMNkhIY2U3ZzNLUUFiQnFVUFQ2dzE3OVdpbG9EQmptQWZDRFFQaUxpdTBTOUJUY25EeFlYL3QKOUN5R1huQkNEZy9ZSE1FWnFuQ1RzejM4c0VqV05VcSt1blNOSkVFUmdDUVl0Y2hxSS9XaWxvWGQyd0tCZ1FEQworTlBYYlBqZ1h5MHoxN2d4VjhFU3VwQVFEY0E5dEdiT1FaVExHaU9Ha2sxbnJscG9BWnVZcWs0Q0pyaVZpYUlyCkJvREllWWpDcjVNK3FnR3VqU3lPUnpSVU40eWRRWkdIZjN1Zkp3NEM3L1k3SlY0amlzR3hSTSt3Rk9yQ0EydmIKVEdGMEZLcThaN0o2N3dQRVliUUNobDB4TmJkcVIvK1ZGTzdGQ1QxV0VRS0JnQThUaE9hZmNEUmdpd0IxRFdyRgozZ1lmT3I0dERENExrNjRYZlF6ajdtRXQyYlJzOFNEYXYwVGZPclVUUlpFTTkyTVFZMnlrbzhyMDJDbmpndmxCCm1aYnZCTEFYaVZLa0laai9TTkNYUnhzOFZkZ3psTkpzYVNZTUtsNloxK1Z3MnZUdDNQSnI0TXlhRWpHYUxlSmMKRGRTQjdYOU9ESk5acW10bGpoRzc5eXpQCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=
With that secret in place, you can start securing. First, reconfigure the gateway to create a TLS listener with mysecret certificate. To do this, use the upgrade option with the Helm chart to add a certificates section to your Traefik Proxy configuration:
helm upgrade traefik -f 05-values.yaml traefik/traefik
# 05-values.yaml
---
experimental:
kubernetesGateway:
enabled: true
certificate:
group: core
kind: Secret
name: mysecret
Once Traefik Proxy has restarted, you can check the results. The same HTTPRoute is still in effect, only now you can access it with HTTPS:
curl --insecure -H "Host: whoami" https://localhost/foo
Hostname: whoami-9cdc57b6d-pfpxs
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.13
IP: fe80::9c1a:a1ff:fead:2663
RemoteAddr: 10.42.0.11:53158
GET /foo HTTP/1.1
Host: whoami
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-74d7f586dd-xxr7r
X-Real-Ip: 10.42.0.1
Note that you must pass the --insecure parameter to curl in order to use the unsigned dummy certificate.
Canary releases
Another feature that Traefik Proxy can support via the Service API specification is canary releases. Suppose you want to run two different services (or two versions of the same service) on one endpoint and route a portion of requests to each. You can achieve this by modifying your HTTPRoute.
First, you'll need a second service to run. For the sake of this example, you can quickly spawn an Nginx instance:
# 06-nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
name: http
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- port: 80
targetPort: http
The HTTPRoute resource has a weight option, and you can assign different values to each of the two services:
# 07-whoami-nginx-canary.yaml
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
name: http-app-1
namespace: default
spec:
parentRefs:
- name: traefik-gateway
hostnames:
- whoami
rules:
- backendRefs:
- name: whoami
port: 80
weight: 3
- name: nginx
port: 80
weight: 1
Now, you'll once again be able to access your whoami service at http://localhost (without the foo/ path suffix). Keep in mind, however, that 25% of the time you'll see a response from Nginx, rather than from whoami.
Migrating from earlier versions: v1alpha1 to v1alpha2
If you already have a running setup with the previous version of Gateway API, there are a few steps you need to take to migrate your applications. The previous version of Gateway API utilized the previous APIGroup of networking.x-k8s.io/v1alpha1. As the 0.4 release uses gateway.networking.k8s.io/v1alpha2 APIGroup, objects created with the previous versions are not compatible. Even the build-in Kubernetes API Server can’t handle that for you. This makes the migrating process is a bit cumbersome.
My recommendation is to follow a simple three-step process:
- Install the new CRDS first.
- Duplicate your previous HTTPRoutes and recreate them with the new API version.
- Run Helm upgrade to install Traefik 2.6 and use the new version of Gateway API:
helm upgrade traefik traefik/traefik --set experimental.kubernetesGateway.enabled=true
These steps ensure that after upgrading, your services will still be accessible.
Known limitations and the future
Currently, Traefik's implementation of the Gateway API is focused on HTTP and HTTPS, TCP Routing functionality, as well as handling TLS. More advanced capabilities such as filters — such as requestHeaderModifier or requestRedirect — will be added in the future.
We are also evaluating how we can use the Policy attachment mentioned above to potentially enable more Traefik Proxy features set with the Gateway API. And, of course, we have our eyes on utilizing the ReferencePolicy as safer to do cross namespace references following that pattern.
In the meantime, you can get started using the Kubernetes Gateway API with Traefik 2.6 today. Explore away, and be sure to share your feedback. And as always, we welcome your contributions! If you have ideas on how to improve Traefik Proxy, don’t hesitate to open a pull request for us.
If you want to learn more about the new features introduced in Traefik Proxy 2.6, check these articles:
- Announcing Traefik Proxy 2.6
- Traefik Proxy 2.6 and HTTP: Trust, Verification, and the Future
- How to use Traefik Proxy and HTTP/3 on AWS EKS
- Getting Started With Traefik Proxy and HashiCorp Consul
You can also watch on-demand our recent webinar, where we explored all the new features.
