What is multi-orchestrator architecture?

Multi-orchestrator architectures consist of applications running as containers within more than one container orchestrator. This could take the form of different orchestrators running within the same cloud or environment, and it could also overlap with a hybrid or multi-cloud architecture. Multi-orchestrator architectures must be unified, as it’s very difficult to network traffic without a consolidated solution. In a unified multi-orchestrator architecture, a reverse proxy sits in layer one to distribute traffic to reverse proxies and ingress controllers associated with different orchestrators.

Run APIs Easily. Anywhere. | Traefik Labs

Q: What is unified multi-cloud architecture?

A unified multi-cloud architecture is a more evolved multi-cloud architecture that adds a reverse proxy in the front that can be thought of as an edge reverse proxy or a layer one reverse proxy. The reverse proxy handles the initial routing to individual clusters and unifies routing. A unified multi-cloud architecture allows for far more sophisticated load balancing schemes. For example, you can expose and route unified API host names across multiple clouds by leveraging path-based routing. You can abstract the location of the actual services away from client requests.

One Platform. One Gateway. Every Request Governed.

Troy Topnik, Sudeep Goswami — Fri, 17 Apr 2026 21:30:30 GMT

How Traefik Labs and SUSE power unified application delivery governance across VM and container estates, covering VMware migration, Kubernetes ingress, and AI agent control.

Three waves are hitting enterprise infrastructure simultaneously, and they are converging on the same answer.

The first wave is VMware displacement. Two years after Broadcom completed its acquisition of VMware, the predicted mass exodus has not materialized, but a measured, sustained unwind is well underway. CloudBolt's January 2026 survey of 302 enterprise IT decision-makers found that 86 percent of organizations are actively reducing their VMware footprint, while 88 percent report that anticipated future price increases are already shaping infrastructure decisions. Most organizations saw price increases of 25-49%; outlier cases have been dramatically higher. The organizations evaluating SUSE Virtualization, Nutanix, and OpenShift as destinations are not making minor adjustments. They are re-platforming their entire estate, without realizing that infrastructure migration and application delivery migration are two different problems.

The second wave is Kubernetes networking consolidation. The Kubernetes community retired ingress-nginx in March 2026, ending all maintenance, bug fixes, and security patches for one of the most widely deployed components in production infrastructure. SUSE formalized Traefik as the default ingress controller for RKE2 starting with v1.36, extending what K3s has delivered for years. IBM Cloud, Nutanix, OVHcloud, and TIBCO each made the same choice independently.

The third wave is enterprise AI governance. Organizations are deploying AI inference workloads, autonomous agents, and MCP-connected tool pipelines into production. API gateway, AI gateway, and MCP governance are absent across every segment of the enterprise infrastructure landscape today. They have to be added, and the architecture of how they are added determines how well-governed, auditable, and portable the result is.

The joint architecture described in this post addresses all three waves with a single platform and a single upgrade path.

The Architecture at a Glance

Traefik Hub sits in front of both SUSE estates: the VM estate (SUSE Virtualization/Harvester) and the Kubernetes estate (RKE2 + K3s), both managed by SUSE Rancher Suite.

Two Estates, One Management Plane

A common misconception is that SUSE Rancher Prime is a Kubernetes-only management platform and SUSE Virtualization is a VM-only platform. Neither is correct, and the distinction matters significantly for how Traefik is positioned in this architecture.

SUSE Rancher Prime is the unified management control plane for both estates. It manages Kubernetes clusters (RKE2 and K3s) and SUSE Virtualization (Harvester) VM clusters from a single interface and API surface. Platform teams operating mixed environments do not need separate consoles for virtualization and Kubernetes.

SUSE Virtualization (Harvester), the VM platform, is itself built on Kubernetes. It runs RKE2 internally, and VMs are managed as Kubernetes objects via KubeVirt. This means VMs have a Kubernetes API surface: they can be declared as YAML manifests, exposed as Kubernetes services, and managed through the same GitOps workflows as containerized workloads. The current release of Harvester defaults to ingress-nginx. Traefik becomes the default ingress controller in Harvester v1.9, which is in active development. Once v1.9 ships, the Traefik application delivery layer will be consistent across both SUSE estates by default, with no additional configuration required.

On the Kubernetes estate, RKE2 defaults to Traefik from v1.36, and K3s has defaulted to Traefik for years. Together with the Harvester v1.9 roadmap, this means Traefik is converging as the standard ingress and application delivery layer across the entire SUSE estate, from edge K3s clusters through enterprise RKE2 to Harvester-managed VM infrastructure.

The Gateway Layer: Traefik Hub Across Both Estates

One of the consistent findings in analyzing application delivery across VMware customer segments is how wide the gap is. Across every segment reviewed, vSphere-only, VVF, and VCF with or without Avi, API gateway is absent, AI gateway is absent, and MCP governance is absent or limited to transport-layer session persistence. These gaps do not close when organizations migrate to SUSE. They close when organizations deploy Traefik Hub. The Triple Gate architecture addresses all three with a single deployment, a single control plane, and a single audit surface, applied uniformly to both the VM and Kubernetes estates.

API Gateway

The API governance gap is older and deeper than most organizations realize. A typical VMware estate routes API traffic through NGINX or F5 without any catalog of what APIs exist in production, no record of which consumers are calling which endpoints, no deprecation workflow, and no API-level access controls. When organizations begin migrating workloads, they frequently discover that their API surface is both larger and less understood than anyone believed. The migration moment is when the gap becomes operationally expensive rather than theoretically concerning.

Traefik's API Gateway closes this gap at the ingress boundary, before any backend on either estate receives a request. The Coraza WAF is a Go-native implementation of the OWASP Core Rule Set, the same ruleset used in ModSecurity-compatible deployments, but without the C/C++ data plane that regulators are asking vendors to move away from. It applies equally to traffic destined for VM-hosted services on Harvester and container-hosted services on RKE2. Authentication is enforced at the gateway: JWT validation, OAuth2 flows, API key management, and policy-based access control are all handled before the request propagates. Rate limiting operates at the consumer level: per user, per team, and per application, using token bucket or sliding window algorithms. API lifecycle management provides the developer portal, access plans, versioning, and deprecation controls that transform a collection of endpoints into a governed API platform.

The regulated industry argument is direct. PCI-DSS requires a WAF for card data environments. HIPAA requires access logging and breach notification capability. The combination of Traefik's API Gateway for north-south enforcement and SUSE Security's container security for east-west micro-segmentation provides the two-perimeter architecture that zero-trust frameworks prescribe: enforce at the boundary, enforce between workloads. The same deployment that handles API governance also closes the memory safety compliance gap, since Coraza is written entirely in Go.

AI Gateway

AI inference infrastructure creates two simultaneous problems that traditional ingress controllers were never designed to solve.

The first is cost. GPU inference is expensive: a frontier model call that involves a long context window can cost orders of magnitude more than a typical API request, and an application making unconstrained inference calls can generate thousands of dollars in cloud spend within hours. Without a gateway, that spend is invisible until the invoice arrives.
The second is risk. Model outputs can contain PII extracted from training data, prompt injection attacks can manipulate model behavior in ways that produce harmful or policy-violating responses, and models from different providers have different safety characteristics that need to be accounted for in regulated deployments.

Traefik's AI Gateway addresses both dimensions at the infrastructure layer. Multi-provider model routing with weighted failover involves defining a primary inference endpoint and a fallback chain. If the primary provider is rate-limited, unavailable, or over budget, traffic shifts to the next provider automatically, with no application code changes, no DNS updates, and no manual intervention. Token budget enforcement operates as a gateway policy: per-user, per-team, and per-application token budgets are enforced before inference happens. An HTTP 429 response is returned before GPU cycles are consumed, which means cost controls are enforced at the infrastructure layer rather than discovered after the fact through billing reconciliation.

Safety pipelines run as middleware in the request and response chain. NVIDIA Safety NIMs, purpose-built safety models covering PII detection, prompt injection detection, and content filtering, execute in parallel with the primary inference call. The safety check result can gate the response before it reaches the consumer. For organizations running AI workloads on NVIDIA GPU nodes in RKE2, this creates a coherent safety architecture: NVIDIA hardware for inference, NVIDIA safety models for guardrails, and Traefik as the orchestration and enforcement layer that applies those guardrails consistently across every inference endpoint on the platform.

MCP Gateway

The Model Context Protocol is the emerging standard for how AI agents call external tools: databases, APIs, file systems, shell interfaces, and internal services. The governance question MCP raises is different in kind from traditional API governance. A conventional API consumer is a deterministic program calling a known endpoint with known parameters. An MCP-connected AI agent is a probabilistic system whose tool calls are shaped by natural-language instructions, and whose behavior can be redirected by a sufficiently crafted prompt. The blast radius of a prompt injection attack scales directly with the breadth of tool access available to the targeted agent. An agent with unrestricted MCP access to production systems is, in effect, an insider threat that can be triggered from the outside.

Task-Based Access Control (TBAC) is Traefik's answer to this problem, and it operates at a different layer than anything in the traditional load-balancing market. The distinction is worth making precisely: Avi Networks announced MCP session persistence as a tech preview at VMware Explore 2025. Session persistence keeps the same agent talking to the same MCP server connection across requests: this is an L4 transport concern. TBAC defines what tasks an agent can perform, on what resources, under what conditions, at the application layer. A TBAC policy does not say "this agent can connect to this MCP server." It says, "this agent can call this MCP server to perform read operations on cluster configuration data, for clusters in the production namespace, at a rate not exceeding one hundred calls per hour, between 08:00 and 22:00 UTC." Every call outside those parameters is blocked and logged. Session persistence and task authorization are not competing capabilities; they operate at different layers of the stack and address entirely different problems.

The AI Assistant (aka “Liz”) in SUSE Rancher Prime illustrates the practical governance model. It uses MCP to call external tools to retrieve cluster state, diagnose anomalies, and execute remediation workflows. TBAC defines what the assistant can do: which data sources it can query, which cluster operations it can initiate, and under what conditions. A policy that allows the assistant to read monitoring data from a CMDB MCP server does not automatically allow it to write to a production configuration store, even if both are reachable MCP servers. Every tool call is logged: agent identity, tool called, parameters passed, response received, and timestamp, creating the audit trail that compliance teams require when AI is part of a production operational workflow. Critically, the governance layer is independent of the agent. Whether the deployment involves Liz, a third-party agent framework, or a custom-built pipeline, the same TBAC policies apply at the gateway without modifications to the agent itself. The control surface belongs to the platform operator, not to the agent vendor.

All three gates (API, AI, & MCP) run on the same Traefik Hub instance, activated with a single Helm chart upgrade from Traefik Proxy. No re-architecture, no migration of existing routing rules, no new deployment footprint. A platform team that deploys Traefik Proxy as the default ingress controller on RKE2 today has a direct path to closing the API governance gap, the AI cost and safety gap, and the agent authorization gap across both the VM and Kubernetes estates, through a single upgrade operation.

The VMware Exit: Traefik as the Application Delivery Constant

The most common entry point for this joint architecture in 2026 is not a greenfield deployment. It is a VMware exit.

A typical VMware estate runs F5 BIG-IP, NGINX, or HAProxy for load balancing and a standalone WAF from F5 or Cloudflare. Every one of those components is written in C or C++. The regulatory picture on this is no longer advisory. CISA's "Product Security Bad Practices" guidance set January 1, 2026 as the deadline for software manufacturers to publish a memory safety roadmap for any existing product written in memory-unsafe languages used in serving critical infrastructure. That deadline has passed. Any vendor supplying network-facing infrastructure to regulated environments that cannot produce a published memory-safety roadmap is now operating against explicit CISA guidance. A June 2025 joint NSA/CISA Cybersecurity Information Sheet reinforced the same position, identifying memory safety vulnerabilities as serious risks to national security and critical infrastructure. The EU Cyber Resilience Act adds a parallel European mandate. Every legacy application delivery component in the VMware ecosystem (F5, NGINX, HAProxy, Avi, and Envoy) has a C or C++ data plane. Traefik, written in Go, has no memory-safety debt to address. It is already compliant by design.

Both vSphere 7 (October 2025) and vSphere 8 (October 2027) are on fixed end-of-life timelines, with no standalone vSphere 9. VCF 9.0 removed general-purpose NSX load balancing from the base entitlement. The migration follows three stages, with Traefik as the constant across all of them:

Migrate (DAY 0)

Deploy Traefik in front of existing VMware workloads before any infrastructure moves. This replaces C/C++ tools with a single memory-safe Go binary and decouples application identity from infrastructure identity. SUSE VM Import controller migrates VMs from vCenter to SUSE Virtualization (Harvester). Traefik auto-discovers workloads at their new location because they are still exposed as Kubernetes services. Traffic continues to flow without any upstream configuration changes.

Modernize

As workloads are containerized on RKE2, Traefik can manage canary routing between VM-hosted and container-hosted services simultaneously. One policy set governs both estates. No big-bang cutover. RKE2 v1.36 arrives pre-integrated with Traefik, so new clusters join the same governance model immediately.

Transform

AI inference workloads on RKE2 with NVIDIA GPU nodes become a third estate behind the same gateway. Token-level rate limiting governs GPU resource costs. A single Helm chart upgrade from Traefik Proxy to Traefik Hub activates API Gateway, AI Gateway, and MCP Gateway across all three estates simultaneously.

The organizations that deploy Traefik on Day 0 of their VMware exit arrive at their destination with application delivery governance already in place, across VMs and containers alike.

Why Regulated Industries Should Pay Attention

For organizations in regulated environments, Traefik Hub provides a supported, sanctioned path to FIPS 140-2 compliance. The implementation uses BoringCrypto, a FIPS 140-2 validated cryptographic module derived from BoringSSL, compiled directly into the Go binary. This means FIPS compliance is not a configuration option or a wrapper: it is enforced at the cryptographic primitive level, covering TLS, JWT validation, API key handling, and HMAC operations. Organizations operating under FedRAMP, HIPAA, PCI-DSS, or EU CRA requirements can deploy Traefik Hub with a fully documented, auditable cryptographic baseline. Air-gapped deployments are supported through RKE2’s offline image tarball distribution, which bundles Traefik natively, combined with Traefik Hub’s offline activation model.

SUSE Security handles east-west security policy within clusters while Traefik handles north-south governance at the estate boundary. Together, they form the two-perimeter model that zero-trust frameworks require. This applies across both the VM and Kubernetes estates managed by Rancher Prime. For organizations that missed the January 1, 2026 CISA deadline for publishing a memory safety roadmap, the SUSE and Traefik migration is itself the roadmap: moving application delivery to a Go-based stack eliminates the class of vulnerability CISA is targeting, rather than scheduling a future plan to do so.

The Upgrade Path is the Commercial Insight

Every Kubernetes cluster on K3s or RKE2 already runs Traefik Proxy as the default ingress controller. Every Harvester VM cluster runs K3s with Traefik inside. Organizations already running Traefik Proxy across their SUSE estate are one Helm chart upgrade away from Traefik Hub, which adds API Gateway, AI Gateway, MCP Gateway, and full API lifecycle management with no changes to the underlying cluster or existing ingress resources.

For VMware exiters, the same logic applies from Day 0. Adopting Traefik before the first VM moves means the upgrade path to full AI governance is already established before the migration completes. No second integration project, no second vendor, no second control plane.

SUSECON 2026 and What Comes Next

SUSECON 2026 runs from April 20–23 in Prague. The theme is Shape Your Resilient Future, with tracks covering AI adoption, digital sovereignty, Cloud Native modernization, and the shift from legacy virtualization. The SUSE Sovereign Summit opens on April 20 and focuses on infrastructure independence and regulatory compliance. The VMware displacement motion that SUSE is leading into SUSECON is sovereign infrastructure by another name: open source, portable, auditable, not subject to the pricing decisions of a single acquirer.

The architecture described here is deployable today. Teams attending SUSECON can begin the first stage of the migration path before the conference ends.

Summary: The Three Waves are Converging

The joint Traefik and SUSE stack addresses VMware displacement, Kubernetes networking consolidation, and enterprise AI governance across both SUSE estates:

Traefik is already the default ingress inside both SUSE estates: K3s inside Harvester (VM estate) and RKE2/K3s (Kubernetes estate), providing a consistent application delivery model across the entire platform
A three-stage VMware migration path (Migrate, Modernize, Transform) with Traefik as the constant, replacing C/C++ application delivery tools with a memory-safe Go platform from Day 0
SUSE Rancher Prime as the unified management plane for both VM and Kubernetes estates, with Traefik Hub as the unified governance layer in front of both
A single Helm chart upgrade from Traefik Proxy to Traefik Hub activates API Gateway, AI Gateway, and MCP Gateway across all estates simultaneously
A two-perimeter security model: Traefik for north-south governance, SUSE Security for east-west policy, across both VM and container workloads
FIPS-certified, air-gap-ready sanctioned & supported images from Traefik Labs for regulated industry deployments
A memory-safe, Go-based implementation that is compliant by architecture with CISA's January 2026 memory safety roadmap deadline (now passed) and with NSA, FBI, and EU CRA guidance, replacing every C/C++ component in the legacy VMware application delivery stack

Platform teams, CTOs, and CISOs navigating VMware exits, Kubernetes modernization, and enterprise AI adoption in 2026 are not choosing between stability and modernization. With this architecture, they are choosing both on a single platform, across every estate, with a single upgrade path.

One Gateway. Three Estates. Zero Compromises.

Aniket Daptari, Sudeep Goswami — Fri, 03 Apr 2026 18:06:00 GMT

How Nutanix and Traefik Labs Are Unifying Application Delivery Across VMs, Containers, and AI.

A joint perspective from Nutanix and Traefik Labs.

The infrastructure world is in the middle of a tectonic shift, and it's not a single earthquake. It's three hitting at once.

Enterprise IT teams are navigating a VMware displacement wave triggered by the Broadcom acquisition. They're managing the coexistence of legacy applications on virtual machines alongside modern microservices running in Kubernetes. And they're under pressure to operationalize AI, embedding inference directly into production applications, not as a science experiment but as a business-critical capability.

Each of these shifts alone would be a multi-year transformation. Together, they're redefining what enterprise infrastructure must look like.

Add to this the rise of sovereign computing. Financial services firms bound by data residency mandates. Healthcare organizations navigating patient data regulations across geographies. Government agencies requiring air-gapped deployments. These organizations can't simply lift and shift to hyperscalers. They need infrastructure that keeps data, models, and governance under their control, on their terms, in their locations.

Here's what makes this moment unique: AI is not a separate workload category. It's becoming embedded inside every application. A single user action, logging into a banking app, checking an account balance, initiating a transfer, can touch a containerized frontend, a VM-hosted core system, and an AI inference endpoint within the same request flow. The user doesn't know. The user doesn't care. They just expect it to work.

The question isn't whether enterprises will run workloads across VMs, containers, and AI infrastructure simultaneously. They already do. The question is whether they have a unified way to route, observe, secure, and govern traffic across all three.

The Real Challenges Behind the Buzzwords

When we talk to enterprise infrastructure teams, the conversation quickly moves past technology preferences and into operational reality.

VMs and containers must coexist for years. The notion that every organization will "just move to Kubernetes" ignores the reality of core banking platforms, ERP systems, mainframe-adjacent workloads, and thousands of VM-based applications that will run for another decade. These aren't candidates for re-architecture. They're candidates for better management. At the same time, every new application, every new capability, is being built cloud-native. Infrastructure teams don't get to choose one world. They have to operate in both.

Complexity is the real enemy, not any single vendor. The skills gap in enterprise IT is well documented. Kubernetes expertise remains scarce. Networking teams trained on traditional load balancers are being asked to manage service meshes. Security teams need policy enforcement that spans environments they barely understand. Every additional tool, console, or abstraction layer compounds this problem. Organizations need fewer moving parts, not more.

Escaping one lock-in shouldn't create another. The VMware migration conversation is often framed as "pick a new hypervisor." But the smarter organizations are thinking beyond that. They want strategic decoupling: the ability to move workloads between infrastructure providers without rearchitecting their application delivery, security policies, or observability pipelines. The governance layer must be portable, even if the compute layer changes.

Day 0 preparation is more important than Day 1 migration. Most migration solutions focus on Day 1: getting workloads moved. But the riskiest moment in any migration is the cutover itself, when application traffic must shift from old infrastructure to new without disruption. Organizations that couple application identity directly to infrastructure identity (hardcoded IPs, DNS-dependent routing, infrastructure-specific load balancers) face downtime windows, DNS propagation delays, and split-brain risks every time a workload moves. The smarter approach is to decouple the application layer from the compute layer before migration begins, so the underlying infrastructure becomes interchangeable. And then comes Day 2 and beyond: who handles traffic management across a heterogeneous environment? How do you enforce consistent rate limiting, authentication, and access policies when workloads span VMs and Kubernetes? What happens when you need to add an AI inference call into an existing application flow without redesigning the entire pipeline?

AI insertion demands infrastructure readiness. The organizations best positioned to operationalize AI are not the ones with the most GPUs. They're the ones that already have unified ingress and governance across their infrastructure. If you can't consistently route, observe, and policy-govern traffic across VMs and containers today, you can't add AI inference as a third estate tomorrow. AI readiness is a side effect of good infrastructure, not a separate initiative.

Unified Application Intelligence: The Nutanix and Traefik Labs Joint Solution

Nutanix and Traefik Labs have built a joint solution designed around a simple but powerful principle: one application intelligence layer that follows your workloads, regardless of where they run.

Nutanix provides the foundational compute, storage, and Kubernetes platform. Traefik provides the unified Layer 7 ingress, routing, and governance that spans across it all. Together, they deliver something neither can offer alone: a consistent application delivery experience across VMs, containers, and AI workloads, managed from a single operational plane.

Here's what this looks like in practice.

Unified ingress across three estates. Traefik provides a single entry point for all application traffic, whether it's destined for a virtual machine on Nutanix AHV, a containerized service on Nutanix Kubernetes Platform (NKP), or an AI inference endpoint running on GPU-accelerated infrastructure. One gateway. One set of policies. One place to observe and troubleshoot. No stitching together separate load balancers, API gateways, and AI-specific proxies.

A lightweight footprint from edge to cloud. Traefik's single binary architecture means it runs everywhere: on a Nutanix cluster in a data center, on a small edge appliance at a retail location, or on a GPU node processing inference requests. There's no heavyweight sidecar tax or complex dependency chain. This matters enormously for organizations running Nutanix at the edge for use cases like manufacturing, retail, and healthcare imaging.

Default ingress in NKP. Traefik ships as the default ingress controller in Nutanix Kubernetes Platform. This isn't a marketplace listing or a "works with" certification. It's an engineering decision by Nutanix to make Traefik the native traffic management layer for every NKP deployment. Customers get enterprise-grade ingress out of the box, with a seamless upgrade path to advanced capabilities.

A clear upgrade path without rip-and-replace. Organizations can start with the open-source Traefik Proxy, which already powers millions of deployments worldwide with over 3.4 billion Docker Hub downloads, and upgrade to enterprise-grade capabilities as needs evolve: API gateway functionality, AI gateway with model routing and token-level governance, MCP gateway for agentic AI frameworks, and full API lifecycle management. The same ingress layer grows with the organization.

Under the Hood: How the Integration Works

The Nutanix and Traefik Labs integration goes deeper than simple co-deployment. It's a set of native integrations that make the unified ingress story operational.

Auto-Discovery Across AHV Virtual Machines

Traefik integrates directly with Nutanix Prism Central to auto-discover virtual machines and their associated services. When a new VM spins up or an existing service changes, Traefik detects it automatically and attaches the appropriate routing rules, security policies, and traffic management configurations. No manual reconfiguration. No configuration drift between what's running and what's being managed. This is critical for organizations managing hundreds or thousands of VMs as part of a VMware migration. As workloads land on AHV, they're immediately visible to and governed by Traefik.

Native Kubernetes Integration on NKP

On the Kubernetes side, Traefik integrates natively with NKP using standard Kubernetes Ingress resources and Traefik's own IngressRoute CRDs. Container workloads are auto-discovered as pods scale up and down, services are registered dynamically, and routing policies are applied consistently. Because Traefik is the default ingress in NKP, there's no separate installation or configuration step. It's ready from the first cluster deployment.

Complementing Flow Networking at Layer 7

Nutanix Flow Virtual Networking and the newly released Flow CNI provide robust Layer 4 network segmentation, microsegmentation, and connectivity across the Nutanix stack. Traefik complements this with Layer 7 application intelligence: content-based routing, header inspection, path-based policies, rate limiting, authentication, and API-level governance. Think of it as two layers working in concert. Flow handles the network plumbing and segmentation. Traefik handles the application-aware traffic decisions. Together, they provide defense in depth without requiring organizations to choose between network-level and application-level controls.

AI Workload Routing and Governance

For AI inference workloads, whether running on GPU-accelerated clusters on-premises, at the edge, or across hybrid environments, Traefik provides the same Layer 7 governance it applies to VMs and containers. This includes intelligent model routing (directing requests to the right model version or endpoint based on content, headers, or load), token-level rate limiting to manage expensive GPU resources, and security guardrails that ensure AI endpoints are subject to the same authentication and access policies as every other service. The key architectural insight: AI inference is just another endpoint behind Traefik's unified ingress. It doesn't need a separate gateway, a separate policy framework, or a separate observability pipeline.

What This Looks Like: A Real Application Flow

Consider a banking application, a pattern representative of what we see across financial services, healthcare, and other regulated industries.

A customer opens their mobile banking app. That initial request hits Traefik and is routed to the React frontend running as a containerized service on NKP. The user logs in, and Traefik routes the authentication request through middleware enforcement to identity services, also on NKP. Once authenticated, the dashboard loads account data. Traefik routes that request to the core banking platform, a VM-based system running on Nutanix AHV that has served the institution reliably for years. The user initiates a funds transfer. Before the transaction is processed, Traefik routes a fraud detection request to an AI inference endpoint for real-time scoring. The check passes, and Traefik routes the final transaction request back to the core banking system on AHV to complete the transfer.

Five steps. Three infrastructure types. One gateway managing the entire flow.

The user experienced a seamless interaction. The infrastructure team manages one ingress layer with one set of policies. The security team has one audit trail. The compliance team has one governance framework. And when the next capability gets added, whether it's a new AI model for personalized recommendations or a containerized notification service, it plugs into the same architecture without re-plumbing.

The Time to Act Is Now

For organizations evaluating their infrastructure strategy, three use cases make the Nutanix and Traefik Labs joint solution immediately actionable.

Migrate: Start with Day 0, not Day 1. Most migration strategies focus on Day 1: the moment a VM moves from one hypervisor to another. But the smartest organizations start at Day 0, before a single workload moves, by deploying Traefik as the unified ingress layer in front of their existing VMware environment.

Why? Because the riskiest part of any migration isn't moving the VM. It's maintaining application availability during and after the move. When application traffic is coupled directly to the infrastructure, a VM migration means DNS changes, propagation delays, potential downtime windows, and user-visible disruption. But when Traefik sits in front as the application's ingress layer, the VM becomes a backend detail. You move it from ESXi to Nutanix AHV. Traefik auto-discovers the new location via Prism Central integration. Traffic keeps flowing. The user never knows the underlying substrate changed.

This is the architectural principle that makes migration resilient by design: decouple the application identity from the infrastructure identity. Traefik owns the routing, the policies, and the traffic management. The hypervisor becomes interchangeable. No split-brain risk during cutover. No "please allow 24-48 hours for DNS propagation." No gap between migration and governance. The same ingress layer that manages your VMs on ESXi today manages them on AHV tomorrow and manages your containers and AI workloads the day after that.

Modernize: Decompose monoliths at your own pace. As applications evolve from monolithic VMs to containerized microservices on NKP, Traefik provides seamless traffic shifting between the old and new. Route 90% of traffic to the legacy VM. Shift 10% to the new container-based service. Validate. Adjust. No big-bang cutover. No separate tooling for each environment. The migration from VM to container happens behind the same gateway, with the same policies, at whatever pace the business requires.

AI-Enable: Insert intelligence without re-architecture. When the time comes to add AI capabilities, whether fraud detection, recommendation engines, document processing, or predictive analytics, Traefik's AI and MCP Gateway provides the same routing, governance, and observability for inference and MCP endpoints that it already provides for VMs and containers. There's no new gateway to deploy, no new policy framework to learn, and no new blind spot in your observability pipeline.

These aren't three separate journeys. They're stages of a single infrastructure evolution: prepare at Day 0, migrate at Day 1, modernize at your own pace, and AI-enable when the business demands it. The organizations that put the right application intelligence layer in place before the first VM moves will navigate each stage faster, with less risk, and with fewer late-night fire drills.

The infrastructure world has shifted. VMs, containers, and AI workloads aren't converging someday. They've already converged, inside your applications, inside your request flows, right now. The only question is whether you have one gateway managing all of it, or three separate tools creating three separate problems.

Nutanix and Traefik Labs joint solution chose one gateway. We think you should too.

To learn more about the Nutanix and Traefik Labs joint solution, visit https://traefik.io/solutions/nutanix-and-traefik.

To see the integration in action, join us at .NEXT 2026 in Chicago April 7^th- 9^th.

]]>

Breaking the Monolith: How an API Gateway Turns a Risky Migration into a Controlled Journey

Zaid Albirawi — Wed, 01 Apr 2026 20:01:19 GMT

The monolith isn't the villain. It got you to production, handled your first million users, and shipped features when the team was small enough to fit in one room. But somewhere between the third deploy queue of the day and the incident where a payment change broke the notification system, the conversation starts: "We need to break this thing apart."

The instinct is to plan a big rewrite. Spin up a Kubernetes cluster, redesign the domain model, rebuild from scratch. Eighteen months later, the new system handles 30% of the old one's functionality, and the monolith is still running in production.

There's a better way. Instead of a big-bang rewrite, you migrate incrementally, routing traffic, testing in production, and shifting workloads one piece at a time. The key ingredient is an API gateway that gives you fine-grained control over where every request goes. That's the thesis of this post: traffic control is migration control, and the right gateway turns a risky migration into a series of small, reversible, low-risk moves.

We'll walk through the journey using Traefik Hub as the gateway, but the patterns apply broadly. What makes Traefik particularly well-suited is that it combines routing, load balancing, identity-aware traffic splitting, and multi-platform connectivity into a single platform. Which means the same gateway that handles your Day 1 pass-through can handle your Day 3 or Day 100 canary deployment across VMs and Kubernetes. And as your architecture evolves beyond traditional microservices, the same gateway extends into Kubernetes Gateway API, AI model routing, and MCP, but more on that at the end.

The Gateway as the Starting Point

Before you break anything, you need a control plane for your traffic. If you don't have an API gateway in front of your monolith, that's step zero. If you have a basic load balancer, put Traefik in front of it or replace it entirely.

The setup is deliberately boring: point DNS at Traefik, configure a catch-all route that forwards everything to the monolith. Nothing changes for users. But now every request flows through a layer you own, and that layer is programmable.

This is the foundation. Every pattern that follows, strangler fig routing, canary deployments, and identity-based traffic splitting, depends on having this control point in place. It costs almost nothing to set up, it gives you options you didn't have before, and it works whether your monolith runs on Kubernetes, VMs, or bare metal.

Observability: See Before You Cut

Here's something teams often overlook: before you break anything apart, you need to understand what you have. The moment all traffic flows through the gateway, you gain a single observation point for your entire application, and that's worth pausing on before you start extracting services.

Traefik Hub emits OpenTelemetry metrics, traces, and logs natively. That means the same gateway you just stood up as a pass-through is already producing data you can pipe into Dash0, Datadog, Splunk, Grafana, Dynatrace, or whatever your observability stack looks like. You don't need to instrument the monolith. You don't need to retrofit tracing into a codebase that was never designed for it. The gateway sees every request, every response code, every latency spike, and it tells you about all of it.

This matters for the migration because it gives you a baseline. Before you extract a single service, you know: which endpoints get the most traffic, what the latency distribution looks like, where errors cluster, and which paths are candidates for extraction. You're building the dashboards and alerts before the migration, not scrambling to add them after something breaks.

It also pays off at every subsequent stage. When you start canary deployments, you're comparing the canary's error rate and latency against a baseline you already have. When you do A/B testing, you're measuring real business metrics through the same pipeline. When you shift traffic with identity-aware routing, you can see exactly how internal users experience the new service versus how everyone else experiences the monolith. Observability turns every migration decision from a guess into a data point.That visibility at the gateway is the foundation, but it’s not the finish line. As you start extracting services, tracing needs to extend with the architecture. Instrumenting your microservices as they’re built lets you carry that context forward, turning edge visibility into true end-to-end distributed tracing. Requests no longer stop at the gateway; they can be followed across every service boundary, every downstream dependency, and every internal interaction. That’s when observability stops being a perimeter view and becomes a full system understanding.

Additional Reference: Metrics, Tracing & Logs configuring OpenTelemetry export in Traefik Hub.

Centralizing Auth: Your First Win

With traffic flowing through the gateway, you get an immediate win that has nothing to do with microservices: centralized authentication.

Move auth to the edge. Traefik Hub supports OIDC and JWT middleware natively, so you can authenticate requests at the gateway before they ever reach the monolith. This means new services you extract later don't need to implement their own auth; the gateway handles it.

How you handle tokens downstream depends on your architecture. In a high-trust internal network, you might strip the token entirely and forward only the claim headers. In a zero-trust model, you forward the verified JWT to the backend so it can also make its own authorization decisions. Either way, the gateway becomes the centralized authentication point for all API traffic, ensuring that every request is consistently authenticated before reaching your services.

This matters for the migration story because the identity information flowing through the gateway is exactly what you'll use later to route specific users to new services. Setting up auth now pays dividends down the road.

Additional References:

OIDC Authentication setting up OpenID Connect middleware in Traefik Hub
JWT Authentication setting up JWT middleware in Traefik Hub

Bridging the Infrastructure Gap: No Re-Platforming Required

Before you start extracting services, there's a common blocker worth addressing: "Our monolith runs on VMs. We can't do any of this until we re-platform onto Kubernetes." That's wrong, and getting this piece in place now means every pattern that follows works regardless of where your monolith lives.

Traefik Hub's multi-cluster capability bridges VMs and Kubernetes natively. The architecture is straightforward:

A Traefik instance in Kubernetes acts as the parent cluster and the entry point for all traffic. A Traefik instance on the VMs (in the same network as the monolith) acts as a child cluster, advertising the monolith's workloads via uplinks. The parent discovers these workloads automatically and routes traffic to them. The connection between parent and child is secured with mutual TLS, and both sides verify each other against a trusted CA.

With this bridge in place, every pattern that follows, strangler fig routing, mirroring, canary deployments, and identity-aware routing, works across the VM-to-Kubernetes boundary. You can route specific paths to a new microservice in Kubernetes while everything else stays on the VM monolith. You can canary between the two, split traffic by identity, or mirror requests across the infrastructure divide. The gateway abstracts the boundary, and you migrate workloads at whatever pace makes sense for your team.

As confidence grows and services move to Kubernetes, you adjust the uplink weights. The VM cluster handles less and less traffic. Eventually, the monolith's uplink weight drops to zero. The fig has fully grown.

Additional Reference: Multi-Cluster Traffic Distribution setup guide for parent/child clusters, uplinks, mTLS, and cross-cluster routing patterns.

The Strangler Fig: Carving Out Your First Service

Now the actual decomposition begins. The pattern is called the strangler fig, after the tropical plant that grows around a tree and gradually replaces it. You don't rewrite the monolith, you intercept specific routes at the gateway and redirect them to new services, one endpoint at a time.

Say your monolith handles everything at example.com. You've extracted the orders domain into a new microservice. At the gateway, you add a route: requests to /api/orders go to the new service; all other requests go to the monolith. Clients don't know anything has changed.

The beauty of this approach is that it's incremental and reversible. If the new orders service has a problem, you revert to the monolith. No redeployment, no rollback, just a routing change. Over time, you extract more domains: payments, notifications, and user profiles. The monolith shrinks. The fig grows.

But there's a question lurking here: how do you know the new service actually works correctly under real traffic? You don't want to route 100% of order requests to an untested service and hope for the best. That's where traffic splitting comes in.

Testing in Production Safely

The strangler fig tells you where to route traffic. But how do you know the new service actually works before you commit? You don't flip a switch, you build confidence in stages: first, you observe, then you compare, then you roll out. Traefik gives you a different mechanism for each stage.

Mirroring: Zero-Risk Shadow Testing

The safest way to test a new service is to send it real traffic without affecting users. That's mirroring. Traefik copies a percentage of production requests to the new service but discards its responses; users only ever see the monolith's response. The new service processes real requests, real payloads, real edge cases, and you watch its logs and metrics for anything unexpected.

Since responses from the mirror are discarded, you won't see failures in your client-facing metrics. You need to watch the microservice directly, its logs, its traces, its error rates, and its latency. This is where the observability foundation you set up earlier pays off: the gateway's OpenTelemetry data gives you the monolith's baseline, and the microservice's instrumentation shows how it handles the same traffic. Compare the two. If the microservice is throwing errors on 2% of mirrored requests, you know exactly what to fix before any real user sees that service.

This is also where you catch the problems you'd never find in a staging environment: the malformed request from that one legacy client, the query parameter nobody documented, the header your integration partner sends at 3 am. Mirroring lets you discover it all without risking users. Start at 10%, ramp to 100% as the new service stabilizes.

A/B Testing: Comparing Old and New

Once the new service handles mirrored traffic cleanly, you're ready to send it to real users. A/B testing splits live traffic between the monolith and the microservice using weighted round robin (WRR), so you can compare their behavior side by side.

Sticky sessions matter here. If a user's first request goes to the new service, their subsequent requests should too; otherwise, they'll get inconsistent behavior mid-session.

Traefik supports cookie-based stickiness at the service level, so each user gets a consistent experience even during the split.
For situations where cookies aren't an option, API clients, mobile apps, or sessionless connections, Traefik also supports Highest Random Weight (HRW) hashing, which deterministically routes requests to the same backend based on attributes like the client's IP address. Same consistency guarantee, no cookies required.

The intent of A/B testing is comparison: which implementation is better? You're gathering data to make a decision.

Failover: The Monolith as Your Safety Net

All the patterns above assume you're actively watching and making decisions. But what about 3 am on a Saturday? Failover gives you an automatic safety net: set the new microservice as the primary and the monolith as the fallback. If the microservice goes down or fails health checks, Traefik automatically routes all traffic back to the monolith. No human intervention required.

This is different from weighted round robin. WRR splits traffic intentionally; you're choosing to send some percentage to each service. Failover is binary: use the new thing, but if it breaks, the old thing is still there. During a migration, this means you can go to 100% on the microservice while keeping the monolith warm as a fallback. It's the confidence boost that lets you make the final cutover without holding your breath.

Circuit Breaker: Automatic Damage Control

Failover handles the case where the microservice is completely down. But what about degraded performance, the service is technically responding, but 40% of requests are returning 500s? That's where the circuit breaker comes in.

Traefik's circuit breaker middleware monitors the service's error rate. When errors exceed a threshold you define (say, 50% of requests over a rolling window), the circuit "trips" and stops sending traffic to the failing service entirely. It periodically lets a few requests through to check if the service has recovered, and automatically closes the circuit once it has.

Pair this with failover, and you have a complete safety story: the circuit breaker detects degradation before it becomes an outage, and failover ensures traffic lands on a healthy node. During the migration, this combination means you can be aggressive about shifting traffic to new services, knowing that the system will self-heal if something goes wrong.

Canary Deployments: Shipping New Features Safely

It's worth noting that canary deployments aren't really a migration pattern; they're what you adopt after the migration. Once the orders microservice owns 100% of its traffic, every future release of that service uses canary: roll out v1.1 to 5% of users, watch the metrics, increase to 50%, then cut over. The same WRR mechanism that helped you compare monolith vs. microservice now helps you ship new features safely.

Health checks on the canary version gate each step automatically: if the canary fails its health check, Traefik stops sending traffic to it. No rollback deployment, just a weight shift back to 100% stable.

This is one of the compounding benefits of the gateway-first migration approach. The weighted routing infrastructure you built for the migration doesn't go away; it becomes your standard deployment strategy for every service you extract.

The full toolkit looks like: a mirror to validate with zero risk, an A/B test to compare monolith vs. microservice and decide which wins, a failover and a circuit breaker to catch problems automatically. Once the microservice owns its traffic, the same weighted routing mechanism becomes your canary strategy, same percentage splits, different purpose: now you're shipping new features safely, not choosing between old and new. Each pattern builds on the same gateway foundation, and every step is reversible.

Additional References:
Traffic Mirroring | A/B Testing | Canary Deployments configuration guides.

Identity-Aware Routing: Migrating by Who, Not Just What

Weighted round robin splits traffic randomly. But sometimes you don't want random, you want surgical. Instead of sending 10% of all users to the new service, you want to send specific users: your internal team, your beta testers, your most adventurous enterprise customers.

This is where Traefik Hub's multi-layer routing and identity integration come together. The idea is straightforward: the gateway already authenticates requests (remember the OIDC setup from earlier). The JWT claims contain information about who the user is, their role, their organization, and their tier. You use those claims to make routing decisions.

The mechanism is multi-layer routing. A parent router authenticates the request and extracts claims into headers. A child router matches on those headers and routes accordingly:

This gives you a migration strategy based on user segments. Internal team first, they're forgiving, and they file better bug reports. Then, beta users who opted in. Then a broader cohort. Enterprise customers on the monolith until you're absolutely confident. You control exactly who sees the new code, and you can change the targeting at any time without a deployment.

You can even combine this with a weighted round-robin: route 50% of beta users to the new service and 50% to the monolith. Identity routing for the who, weighted splitting for the how much.

Additional References:
Multi-Layer Routing | Identity-Based Routing identity-aware routing configuration and patterns.

The Triple Gate: API, AI, and MCP

The migration from monolith to microservices isn't the end of the journey; it's the first gate. The gateway you built to control your migration is the same gateway that controls what comes next, and what comes next is a three-gate architecture where a single platform governs all your traffic: services, AI models, and AI agents.

Gate 1: API Gateway. This is where you are now. This layer handles everything you've discussed throughout this post. Routing, authentication, observability, traffic splitting, and identity-aware decisions. Traefik Hub also supports the Kubernetes Gateway API standard, so as the ecosystem moves toward it, you have a native path to adopt it without re-architecting. This gate handles every request from users to services.

Gate 2: AI Gateway. Your newly extracted microservices will start calling LLMs. When they do, the gateway becomes the control plane for AI traffic too. Traefik Hub's AI Gateway routes requests to different model providers, enforces token-based rate limiting, applies content guardrails, and caches responses, all at the gateway layer. The same infrastructure that canary-tested your orders microservice now canary-tests your switch from GPT-4 to Claude. This gate handles every request from services to models.

Gate 3: MCP Gateway. The Model Context Protocol (MCP) is the emerging standard for connecting AI agents to tools and data sources. As your microservices become the tools that agents call, the gateway needs to understand MCP's stateful, session-based communication patterns. Traefik Hub's MCP Gateway routes agent-to-tool traffic, enforces tool-based access control, and maintains session affinity across requests. The services you extracted from the monolith become first-class tools in the agent ecosystem, and the gateway governs access to all of them. This gate handles every request from agents to tools.

The thread connecting all three gates is the one we started with: traffic control is migration control — and it doesn't stop at microservices. The same gateway that manages your monolith decomposition manages your AI model routing and your agent tool access. One platform, three gates, every request.

The Migration Is a Journey

The monolith doesn't die overnight, and it doesn't have to. The worst thing you can do is treat the migration as a binary event, old system off, new system on. The best thing you can do is build the infrastructure that lets you migrate incrementally, test continuously, and roll back instantly.

The API gateway is that infrastructure. It's the control plane for your migration, and for what comes after. Start with a simple pass-through. Set up observability to understand what you have. Centralize auth. Bridge your VMs and Kubernetes so the infrastructure is ready. Extract your first service behind a strangler fig route. Split traffic to test it. Use identity-aware routing to target specific user segments. And when you're ready, extend that same gateway through the triple gate, API, AI, MCP, into whatever your architecture becomes next. At every step, you're in control, and at every step, you can stop and come back to it later.

The monolith took years to build. Give yourself permission to take time to break it apart safely.

Every pattern in this guide, weighted routing, mirroring, canary releases, and load-balancing strategies, is available in Traefik Hub today. To try them on your own cluster, get started with Traefik Hub.

]]>

Implementing Forrester's AEGIS Framework at the Infrastructure Layer

Immánuel Fodor, Zaid Albirawi — Fri, 27 Mar 2026 13:50:51 GMT

Forrester's AEGIS framework (Agentic AI Enterprise Guardrails For Information Security) defines 39 controls across six domains for securing AI agents in the enterprise. It is the most comprehensive analyst framework for agentic AI security to date, with regulatory cross-mapping to NIST AI RMF, ISO/IEC 42001:2023, OWASP Top 10 for LLMs, the EU AI Act, and MITRE ATLAS. CISOs are adopting it as the evaluation standard for agent governance.

The framework is strong on what organizations need. It is intentionally silent on how to implement it. That's by design: AEGIS is vendor-neutral and architecture-agnostic. But it leaves platform engineers and security architects with a practical question: where do these 39 controls actually get enforced?

This post maps AEGIS's six domains and three core principles to Traefik Hub's Triple Gate architecture (API Gateway, AI Gateway, MCP Gateway), with configuration examples showing how specific AEGIS controls translate to Kubernetes CRDs that enforce at the infrastructure layer.

Key Takeaways

Forrester's AEGIS framework defines 39 controls across 6 domains. Its Zero Trust domain explicitly calls for "policy gateways" that all agent traffic must flow through.
Traefik Hub's Triple Gate architecture maps to 4 of 6 AEGIS domains (IAM, Data Security, Application Security, Zero Trust) with strong, product-native implementation from a single platform.
AEGIS's core principle, "least agency," is the design philosophy behind TBAC (Tools/Tasks/Transactions-Based Access Control): infrastructure-layer enforcement of what agents can do, independent of the agent runtime.
The remaining 2 domains (GRC and Threat Management) require organizational processes and complementary solutions, as expected for any infrastructure product.
We also published a companion post mapping infrastructure-layer enforcement to the OWASP Top 10 for Agentic Applications.

Why Infrastructure-Layer Enforcement Matters for AEGIS

Before mapping individual domains, it's worth understanding why AEGIS repeatedly calls for enforcement at the infrastructure layer rather than inside the agent runtime.

AEGIS's Zero Trust domain is explicit: "all access to APIs or data [must] flow through Zero Trust enforcement layers such as policy gateways or conditional access controls." The Application Security domain requires "channeling agent interactions through secure gateways with runtime policy enforcement and input sanitization."

The reason is architectural. Agent platforms are adding application-level controls: RBAC, audit logging, policy SDKs, signed skills. These operate inside the agent runtime. If the runtime is compromised through prompt injection, jailbreak, or privilege escalation, the governance is compromised with it. Application-layer controls and infrastructure-layer enforcement serve different purposes. Application-layer tools govern the agent's logic: what it should do. Infrastructure-layer enforcement governs the traffic: what it's allowed to do, regardless of what the runtime thinks is appropriate.

AEGIS treats both as necessary. This post focuses on the infrastructure layer. For a deeper look at how infrastructure-layer enforcement maps to specific agentic risks, see our companion post on the OWASP Agentic Top 10.

AEGIS Core Principles: Least Agency, Continuous Assurance, and Securing Intent

Before mapping individual domains, AEGIS is built on three core principles. Each maps to a Traefik Hub architectural decision.

Principle 1: Least Agency

AEGIS introduces "least agency" as its foundational principle. It extends the concept of least privilege beyond access control to cover decisions and actions. Agents should receive only the minimum permissions, capabilities, tools, and decision-making authority necessary for a specific task.

Traefik Hub implements least agency through TBAC (Tools/Tasks/Transactions-Based Access Control) in the MCP Gateway. TBAC enforces actions an agent identity is authorized to commit at the infrastructure layer: which tools it can invoke, which tasks it can perform, which transaction parameters are permitted, and under what conditions.

The enforcement is scoped to the agent's JWT identity. The same identity context that authenticates an API call (Gate 1), governs token budgets (Gate 2), and authorizes tool invocations (Gate 3). There's no gap where an agent can authenticate as one identity and act with the permissions of another.

Here's what least agency looks like in practice. Consider an inventory management agent that should be able to look up stock levels and create alerts, but should never be able to approve purchase orders:

Request flow for agent identity "inventory-agent-prod-01":

Gate 1 (API Gateway):
  → JWT validated: agent=inventory-agent-prod-01, team=warehouse, role=inventory-reader
  → Rate limit checked: 1,000 requests/hour for this identity based on the agent scope

Gate 2 (AI Gateway):
  → Token budget checked: 50,000 tokens/day remaining
  → Safety pipeline: all configured tiers execute in parallel
  → Token consumption recorded against this identity

Gate 3 (MCP Gateway):
  → TBAC evaluated:
    - Tool: inventory-lookup (authorized, read-only)
    - Tool: stock-alert (authorized, max 10 creates/hour)
    - Tool: purchase-order (DENIED, requires manager-agent role)
  → Agent tool call budget checked: 10,000 calls/day remaining
  → Invocation executed and logged with full parameter capture

The agent can look up inventory. It can create stock alerts, up to 10 per hour. It cannot approve purchases. These constraints are enforced at the traffic layer, below the agent runtime. If the agent is prompt-injected into believing it has purchase authority, TBAC still blocks the tool invocation because the authorization check happens in the gateway process, not in the agent process.

This is what AEGIS means by least agency: not just controlling access to systems, but controlling what an agent can do within those systems, enforced independently of the agent's own understanding of its permissions.

Principle 2: Continuous Risk Management

AEGIS's second principle replaces periodic audits with continuous evaluation of data, model, and agent integrity. Continuous assurance, not point-in-time compliance.

In Traefik Hub, this translates to per-request enforcement. The composable safety pipeline runs on every request and every response. TBAC evaluates every tool invocation. Token budgets are checked before every LLM interaction. There is no sampling, no periodic scan, no batch evaluation.

JWT validation happens on every request, not once per session. If a token expires, the next request is denied. If a TBAC policy is updated via a Kubernetes CRD change, the updated policy is enforced on the next request, with no redeployment required.

Deployment via Kubernetes CRDs enables GitOps-driven policy management. TBAC policies and safety pipeline configurations are version-controlled, auditable, and continuously reconciled by the Kubernetes controller. Drift from the declared state is automatically corrected. This means governance policies are not just continuously enforced; they're continuously reconciled to the desired state.

Principle 3: Securing Intent

AEGIS's third principle shifts the focus from infrastructure-centric controls to intent-centric controls. Organizations need to understand whether an agent's actions are malicious or benign, intentional or unintentional.

The Triple Gate architecture provides intent visibility across the full AI workflow:

Gate 1 (API Gateway): Who is this user or agent, and are they authorized to access this API?
Gate 2 (AI Gateway): What is the user asking the LLM? Is the content safe? Is the LLM's response factually grounded and free of harmful content?
Gate 3 (MCP Gateway): What tool is the agent invoking, with what parameters? Is this action authorized for this agent identity?

The same JWT identity is tracked across all three gates. When an incident occurs, the audit trail shows the full chain: who authenticated, what they asked, what the model responded, what tool the agent invoked, with what parameters, and what the governance decision was at every step. This is the evidence chain AEGIS requires for securing intent.

Mapping the Six AEGIS Domains

With the principles established, here's how each AEGIS domain maps to infrastructure-layer enforcement. Domains are covered in order. Coverage depth varies: four domains map strongly to product-native capabilities, two require organizational processes where we provide the enforcement layer but not the full domain.

Domain 1: Governance, Risk, and Compliance

AEGIS's GRC domain covers organizational governance: oversight functions, AI system inventories, risk classification, acceptable use policies, and cross-functional governance groups.

Coverage: Partial (enforcement layer). This is primarily an organizational domain, not a product-layer domain. No gateway product provides governance committees or risk classification taxonomies. What Traefik Hub provides is the enforcement infrastructure that makes GRC decisions machine-executable.

GRC policies that exist only in documents don't enforce themselves. A policy that says "agents must not access customer PII without explicit authorization" is only as strong as the system that enforces it. In Traefik Hub, that policy becomes a TBAC CRD that blocks unauthorized access at the traffic layer, plus a safety pipeline rule that masks PII in transit. The GRC team defines what should be allowed; the infrastructure enforces it.

For AI system inventory and risk classification, integration with an existing CMDB or governance platform is required. Traefik Hub's telemetry (which agents exist, what they access, how much they consume, what governance decisions were made) provides the operational evidence that feeds inventory and risk processes.

Domain 2: Identity and Access Management

AEGIS's IAM domain treats agents as a new identity class (neither fully human nor fully machine) that requires unique identification, just-in-time privileges, human oversight, and contextual authentication.

Coverage: Strong. Traefik Hub's implementation centers on JWT-based identity continuity. Every agent interacting with the Triple Gate is identified by its JWT claims. Token rate limiting, safety pipeline logging, TBAC authorization, and observability telemetry all track against this unique identity. This enables:

Per-agent token consumption tracking and budget enforcement
Per-agent distributed traces across all three gates, capturing authentication decisions, guard results, and tool invocations per call via OpenTelemetry
Per-agent safety pipeline results
Per-agent tool invocation history with full parameter capture

For just-in-time privileges, TBAC supports conditional, scoped authorization. High-risk tool invocations can require human approval. Time-windowed permissions can restrict agent capabilities to specific operational hours. Parameter-level constraints can limit the scope of approved actions.

Authentication is continuous. JWT validation occurs on every request at Gate 1. The validated identity propagates through Gates 2 and 3, where it is re-evaluated against the relevant TBAC policies and token budgets. There is no session-based trust. Every interaction is independently validated.

Domain 3: Data Security and Privacy

AEGIS's Data Security domain covers data classification for agent contexts, controls for agent memory, data enclaves, anonymization, privacy-preserving AI operations, and data provenance.

Coverage: Strong. The composable safety pipeline is Traefik Hub's implementation of these controls. It operates bidirectionally (on both requests and responses) across four tiers, each with different detection capabilities.

Here's a configuration that implements multiple AEGIS data security controls in a single middleware:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: aegis-data-protection
  namespace: ai-services
spec:
  plugin:
    content-guard:
      engine:
        regex: {}

      request:
        # Prevent data extraction via prompt manipulation
        - jsonQueries:
            - ".messages[].content"
          block: true
          reason: "data_extraction_detected"
          entities:
            - "(?i)repeat.*(training|system|original).*(data|prompt|instructions)"
            - "(?i)output.*(everything|all).*(you know|in your|you were)"

        # Mask Social Security numbers before they reach the LLM
        - jsonQueries:
            - ".messages[].content"
            - ".context"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{3}-\\d{2}-\\d{4}"

        # Mask credit card numbers
        - jsonQueries:
            - ".messages[].content"
            - ".customer.payment"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}"

      response:
        # Prevent infrastructure data leakage in LLM responses
        - jsonQueries:
            - ".choices[].message.content"
          mask:
            char: "X"
            unmaskFromLeft: 0
            unmaskFromRight: 0
          entities:
            - "192\\.168\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "10\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "(?i)(postgres|mysql|redis)://[^\\s]+"

This configuration covers three AEGIS data security requirements:

Data loss prevention (request-side): The first rule blocks prompts that attempt to extract training data or system instructions. This is deterministic, sub-millisecond, and catches the most common data extraction patterns.

PII anonymization (request-side): The second and third rules mask SSNs and credit card numbers before they reach the LLM. The data is redacted in-place: the request continues with ***-**-1234 instead of the full SSN. The LLM never sees the original value.

Infrastructure data protection (response-side): The fourth rule prevents the LLM from leaking internal IP addresses and database connection strings in its responses. This catches a common failure mode where models trained on internal documentation inadvertently expose infrastructure details.

This is Tier 1 (Regex Guard) only. For PII that doesn't follow strict patterns (misspelled names, partial phone numbers, context-dependent personal data), add Tier 2 (Microsoft Presidio). For semantic threats (prompt injection attempts that don't use obvious keywords, off-topic drift, harmful content), add Tier 3 (NVIDIA NIMs) and Tier 4 (IBM Granite Guardian). All tiers run in parallel. For a detailed walkthrough of the full four-tier pipeline, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

A note on deployment model: Traefik Hub is self-hosted. All safety pipeline processing runs on the customer's infrastructure. No data leaves the customer's environment for governance evaluation. For organizations subject to data residency requirements (EU AI Act, GDPR, NIS2), this is not a feature but a requirement. Air-gapped deployment is fully supported.

Domain 4: Application Security and DevSecOps

AEGIS's Application Security domain covers the agent software lifecycle: risk evaluation, DevSecOps practices, supply chain assessment, adversarial testing, embedded telemetry, sandbox interactions, input sanitization, and runtime policy enforcement.

Coverage: Strong. Two aspects of this domain are particularly relevant at the infrastructure layer: runtime policy enforcement and supply chain security.

Runtime policy enforcement: TBAC policies are expressed as Kubernetes CRDs and enforced on every MCP tool invocation. This is runtime enforcement that operates independently of the agent's application code. The agent framework does not evaluate TBAC policies, cannot modify them, and has no mechanism to bypass them.

AEGIS also requires that governance controls work with agent architectures, not against them. Traditional gateways return an HTTP 403 Forbidden when a policy blocks a request. For autonomous agents executing multi-step workflows, a 403 breaks the agent's control flow. The agent can't distinguish between "the guardrail blocked this specific request" and "something is fundamentally wrong with the system." Most agent frameworks treat non-2xx responses as system errors.

Traefik Hub's onDenyResponse feature addresses this. When a guard blocks a request, the response can be configured to return a structured, schema-compliant refusal (HTTP 200 with a conversational message) instead of a raw 403:

blockConditions:
  - reason: harmful_content
    condition: Contains("yes")
    onDenyResponse:
      statusCode: 200
      message: >
        This request was blocked by organizational security policy.
        Please reformulate your request within acceptable use guidelines.

An agent receiving this structured refusal can parse it as a valid conversational turn, understand that the request was refused (not that the system failed), and decide what to do next. The middleware chain downstream continues to function normally. This is what makes infrastructure-layer governance agent-aware: enforcement that works with autonomous workflows rather than breaking them.

Supply chain security for agent tools: AEGIS calls for software supply chain assessment, which in the agentic AI context means securing the MCP tool supply chain. The risk is real: malicious MCP servers have already been documented in the wild (the Postmark MCP backdoor in September 2025 shipped 15 clean versions before injecting a backdoor in v1.0.16, affecting approximately 500 organizations).

The MCP Gateway addresses this at the infrastructure layer:

Server allowlisting: TBAC policies explicitly define which MCP servers each agent identity can connect to. Connections to unlisted servers are denied at the gateway.
Tool-level authorization: Even within an allowed MCP server, TBAC controls which specific tools the agent can invoke.
Parameter constraints: TBAC can limit the parameters an agent passes to authorized tools (read-only database queries, restricted file paths, capped transaction values).
Outbound inspection: The MCP Gateway can inspect outgoing API calls from agent tool invocations, preventing data exfiltration through tool parameters.

An agent that gets prompt-injected into connecting to a malicious MCP server is blocked by the TBAC allowlist. These are infrastructure-layer controls that the agent runtime cannot override.

Domain 5: Threat Management and Security Operations

AEGIS's Threat Management domain covers real-time monitoring, anomaly detection, comprehensive logging, rapid containment, hallucination detection, and incident response readiness.

Coverage: Partial (detection and containment). Traefik Hub covers the detection and containment aspects of this domain through the composable safety pipeline and operational controls.

Hallucination detection deserves specific attention because AEGIS calls it out as a required capability and because it is architecturally distinct from other threat types. IBM Granite Guardian provides hallucination detection and RAG quality assessment through the LLM Guard middleware. The useRequestHistory: true parameter provides the request context so the model can evaluate whether the response is factually supported. Regex patterns and NLP-based entity detection cannot detect hallucinations because they match surface patterns, not factual accuracy. This requires a semantic model.

Parallel threat detection across multiple guard providers (NVIDIA NIMs for 22+ harm categories and jailbreak detection, IBM Granite Guardian for hallucination and harm) runs concurrently. Total enforcement time equals the slowest guard, not the sum.

Rapid containment via the Failover Router implements circuit-breaker chains. When a provider returns error status codes (429, 500-504), traffic is automatically rerouted to fallback providers. All safety policies remain enforced regardless of which provider is serving the request. The governance doesn't degrade when the model does.

Comprehensive logging across every gate emits structured telemetry: API authentication results (Gate 1), safety pipeline guard decisions with reasons (Gate 2), and TBAC authorization decisions with tool and parameter details (Gate 3). All telemetry includes the agent identity for cross-gate correlation and is exported via OpenTelemetry to the observability platform of your choice.

Request mirroring provides a distinct and complementary capability worth calling out specifically. Traefik Hub can forward a live copy of every request and response to an external destination in parallel with the primary traffic flow, with no impact on latency for the agent workload. For SIEM and UEBA integration, this means security tooling receives the actual payloads in real time, not just metadata after the fact. Threat detection models can analyze live traffic, identify anomalous patterns, and feed updated policy signals back to the gateway. This is the integration point where infrastructure-layer enforcement connects to SOC workflows.

A note on scope: Traefik Hub provides detection and containment at the traffic layer. Behavioral anomaly detection (establishing baseline invocation patterns and alerting on deviation) and incident response workflows are organizational capabilities that require SIEM/UEBA integration and SOC processes. AEGIS expects both. The gateway provides the telemetry and the live traffic mirror that those processes require; the analytical and response layers sit above it.

Domain 6: Zero Trust Architecture

AEGIS's Zero Trust domain is the most direct validation of infrastructure-layer enforcement. The AEGIS framework requirements read as a description of a policy gateway architecture:

"All access to APIs or data [must] flow through Zero Trust enforcement layers such as policy gateways or conditional access controls."
Network access layer controls for agent workloads.
Continuous validation of agent runtime environment.
Agent-to-agent communication monitoring.
Ephemeral identities for autonomous systems.

Coverage: Strong. The Triple Gate architecture is a policy gateway. Every API call (Gate 1), every LLM interaction (Gate 2), and every MCP tool invocation (Gate 3) flows through the gateway with policy enforcement. There is no bypass path.

TBAC enforces least agency at the infrastructure layer. Agents receive only the tools, tasks, and transaction authority explicitly granted by their TBAC policy. No implicit permissions. No inheritance of elevated privileges from the agent runtime.

When agents communicate via API calls or MCP tool invocations, both sides of the interaction pass through the Triple Gate. The initiating agent's request is authenticated and authorized via TBAC. The receiving agent's response passes through the safety pipeline. The full interaction is logged with both agent identities.

JWT-based identity supports scoped tokens with configurable expiration. Agents can be issued tokens that are task-scoped, with a short time-to-live set at issuance time. Expired tokens and tokens with incorrect scopes are rejected.

For organizations in regulated industries where SaaS governance tools are not permitted, Traefik Hub is self-hosted on the customer's infrastructure. Air-gapped deployment is fully supported. Traefik Hub is FIPS 140-2 validated and FIPS 140-3 ready (native Go BoringCrypto). No governance data, telemetry, or policy configurations leave the customer's environment.

Putting It Together: One Request, Five Domains

Here's how AEGIS domains intersect in a single request through the Triple Gate:

Request arrives. Gate 1 validates the agent's JWT and enforces API rate limits. → Domain 2 (IAM), Domain 6 (ZTA)
Token budget check. If the estimated input tokens would exceed the agent's remaining budget, the request is blocked before any guard runs. → Domain 6 (ZTA: least agency applied to resource consumption)
Safety pipeline executes. Regex Guard masks SSNs and credit cards. Presidio catches context-dependent PII. NVIDIA NIMs scan for prompt injection and harmful content. All tiers run in parallel. → Domain 3 (Data Security), Domain 5 (Threat Management)
LLM responds. The response passes back through the pipeline. Granite Guardian checks for hallucination against the request context. Regex Guard masks any internal IP addresses in the output. → Domain 3 (Data Security), Domain 5 (Threat Management)
Agent invokes an MCP tool. TBAC validates the tool invocation against the agent's authorized tools, permitted parameters, and usage limits. → Domain 4 (Application Security), Domain 6 (ZTA: least agency)
Telemetry emitted. Every gate logs the agent identity, governance decisions, guard results, tool invocations, and token consumption. → Domain 1 (GRC: evidence for governance reporting), Domain 5 (Threat Management: SOC monitoring)

Five of six AEGIS domains are enforced in a single request, through one platform, with one identity context.

Coverage Summary

Not every AEGIS domain is a product-layer control. AEGIS is a comprehensive organizational framework, and some domains require processes, teams, and governance structures that no infrastructure product can provide. Here's our assessment of where Traefik Hub's Triple Gate provides strong coverage and where complementary solutions are required:

AEGIS Domain	Triple Gate Coverage	What's Covered	What Requires Complement
Domain 1: GRC	Partial (enforcement)	Machine-executable policy enforcement, governance evidence via telemetry	Governance oversight function, AI system inventory, risk classification
Domain 2: IAM	Strong	JWT identity continuity, per-agent tracking, TBAC-scoped authorization, continuous authentication	Agent onboarding/offboarding workflows (requires IdP integration)
Domain 3: Data Security	Strong	4-tier PII detection and masking, bidirectional enforcement, DLP, self-hosted data processing	Data classification taxonomy, agent memory controls
Domain 4: Application Security	Strong	TBAC runtime policy enforcement, MCP supply chain security, agent-aware error handling, input sanitization	SDLC integration, adversarial red teaming
Domain 5: Threat Management	Partial (detection + containment)	Hallucination detection, content threat detection, prompt injection defense, circuit breakers, comprehensive logging	Behavioral anomaly detection, forensic investigation workflows, incident response playbooks
Domain 6: Zero Trust	Strong	Policy gateway enforcement, least agency via TBAC, continuous validation, agent-to-agent monitoring, air-gapped deployment	Network segmentation (requires CNI integration)

Four of six domains with strong, product-native implementation. Two domains with partial coverage that requires organizational processes or complementary solutions. This is an expected distribution for an infrastructure product: GRC and SecOps are organizational capabilities, while IAM, Data Security, Application Security, and Zero Trust are enforceable at the infrastructure layer.

What's Next

AEGIS is gaining traction as the evaluation standard for agentic AI governance. As CISOs adopt it, the question shifts from "do we need agent governance?" to "how do we implement these 39 controls?" For the infrastructure-layer controls, the answer is a policy gateway that enforces identity, data security, application security, and zero trust on every request, every response, and every tool invocation.

For a complementary perspective on infrastructure-layer enforcement, see our mapping of the OWASP Top 10 for Agentic Applications to the gateway layer. And for a deep technical walkthrough of the composable safety pipeline referenced throughout this post, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Traefik Hub v3.20 is now available as an Early Access release, with general availability planned for late April 2026. To try the capabilities described in this post, sign up for Early Access.

Frequently Asked Questions

What is Forrester's AEGIS framework?

AEGIS (Agentic AI Enterprise Guardrails For Information Security) is a security framework published by Forrester Research that defines 39 controls across six domains for securing AI agents in the enterprise. The six domains are: Governance, Risk, and Compliance (GRC); Identity and Access Management (IAM); Data Security and Privacy; Application Security and DevSecOps; Threat Management and Security Operations; and Zero Trust Architecture. The framework includes regulatory cross-mapping to NIST AI RMF, ISO/IEC 42001, OWASP Top 10 for LLMs, the EU AI Act, and MITRE ATLAS.

What is "least agency" in the AEGIS framework?

Least agency is AEGIS's foundational security principle. It extends the concept of least privilege beyond access control to cover decisions and actions. Under least agency, AI agents should receive only the minimum permissions, capabilities, tools, and decision-making authority necessary for a specific task. It is the agentic equivalent of the principle of least privilege, applied to autonomous systems that make decisions, not just access resources.

How do you implement AEGIS controls at the infrastructure layer?

AEGIS controls can be implemented at the infrastructure layer through a policy gateway that sits between agent workloads and the services they consume. Traefik Hub's Triple Gate architecture (API Gateway, AI Gateway, MCP Gateway) enforces AEGIS controls on every API call, LLM interaction, and MCP tool invocation. Identity controls are enforced via JWT-based authentication. Data security controls are enforced via a composable safety pipeline. Application security controls are enforced via TBAC (Tools/Tasks/Transactions-Based Access Control). Zero trust controls are enforced through continuous per-request validation.

How many AEGIS domains can be enforced at the infrastructure layer?

Four of six AEGIS domains (IAM, Data Security, Application Security, Zero Trust) can be strongly enforced at the infrastructure layer through a policy gateway. The remaining two domains (GRC and Threat Management/SecOps) are primarily organizational capabilities that require governance processes, SIEM integration, and SOC operations, though infrastructure-layer telemetry provides the evidence these processes need.

What is the difference between AEGIS and the OWASP Top 10 for Agentic Applications?

AEGIS is a comprehensive organizational security framework with 39 controls across six governance domains, focused on how enterprises should structure their AI agent security programs. The OWASP Top 10 for Agentic Applications is a risk taxonomy that identifies the ten most critical security risks for autonomous AI agents (such as goal hijacking, tool misuse, and identity abuse). AEGIS tells you what controls to implement; OWASP tells you what risks to prioritize. Both are complementary and map to infrastructure-layer enforcement. See our OWASP Agentic Top 10 mapping for the risk-level view.

]]>

The Ingress NGINX Retirement Created Three Risks. Most Teams Are Only Managing One.

Sudeep Goswami — Thu, 26 Mar 2026 12:59:48 GMT

The retirement of Ingress NGINX is official. As of March 24, 2026, there are no more releases, no more bug fixes, and no more security patches. Roughly half of all cloud-native environments are running a controller that will never be updated again. The urgency of the situation is not in question.

What is in question is whether the response to that urgency is complete.

The Ingress NGINX retirement did not create one risk; it created three. The first is migration risk: the operational challenge of moving hundreds of clusters off a retired controller without breaking production. The second is security risk: the architectural question of whether the new controller eliminates the vulnerability class that made Ingress NGINX untenable, or inherits it. The third is stagnation risk: the strategic question of whether the foundation being laid today can support the governance layers every organization will need to build on top of it over the next several years.

The migration conversation happening across the Kubernetes community right now is almost entirely focused on the first risk. The other two deserve equal attention. The decisions being made now will compound, for better or worse, across all three.

Migration Risk

Migration risk is the risk most teams understand best, because it is the most immediately visible. Clusters running Ingress NGINX today are running unmaintained software. Every day that passes is a day closer to an unpatched CVE landing in a production environment with no fix available. The operational pressure to move is real and justified.

But migration risk is not just about urgency. It is about what the migration actually requires in practice, and that is where most of the market response has fallen short.

When the Ingress NGINX retirement was announced, vendors responded by positioning their controllers as the destination. Compatibility tables appeared. Blog posts explained why each vendor's product was the natural landing spot. The conversation became: which controller should we migrate to?

That framing misses the harder problem. Ingress NGINX users have built configurations over years using a specific annotation vocabulary. Some of those annotations have direct equivalents in other controllers. Some require behavioral substitutes. Some require rethinking routing logic entirely. For a team running a handful of clusters, that is a manageable audit. For a team running hundreds, it is a multi-quarter engineering project, unless someone has already built the bridge.

Traefik built the bridge. The open-source Ingress NGINX migration tool, available at github.com/traefik/ingress-nginx-migration, analyzes existing cluster configurations, maps annotations against Traefik's Ingress NGINX Provider, identifies gaps, and generates a migration report before any commitment is made. Traefik's Ingress NGINX Provider covers over 90% of Ingress NGINX annotations, meaning the vast majority of existing configurations migrate without manual rewriting.

That covers the immediate operational problem. But migration risk also has a longer dimension that most teams are not factoring in: the configuration model chosen today becomes the configuration model maintained for years.

The Kubernetes community has been clear on the direction. The Ingress spec is feature-frozen. Kubernetes Gateway API is the designated successor, now a CNCF-standard specification with broad ecosystem backing, and the routing model that future tooling, documentation, and community investment will be built around. Teams treating this migration purely as a like-for-like swap are solving the urgency problem without capturing the modernization opportunity.

Traefik supports both paths simultaneously. The same binary, the same Helm chart, the same Go-native data plane handles Ingress resources, Kubernetes Gateway API, and Traefik's native IngressRoute CRD concurrently. A team that needs to move fast can drop in Traefik against existing Ingress configurations today, validate the migration, and plan the Gateway API transition on its own schedule, without a future re-architecture. The migration decision and the modernization decision can be made independently. No second migration. No second vendor.

This reflects a long-view approach to winning in this market: solve the actual engineering problem teams are facing, and earn the adoption that follows. Not a destination to land on. A bridge to get there without operational disruption.

Migration risk is real and solvable. It is also the least consequential of the three risks if the other two go unaddressed.

Security Risk

Security risk is the risk the migration conversation has most conspicuously skipped. It is also the one with the most structural teeth.

Ingress controller migrations do not happen often. For most organizations, this is the first one in years and will likely be the last one for years to come. The operational cost of replacing a controller at scale is high enough that once the decision is made, it stays made. Which means the question being answered right now is not just "which controller do we migrate to." It is "which security architecture do we commit to for the next several years."

Most of the migration conversation has been focused on whether the destination controller is actively maintained. That is a necessary condition but not a sufficient one. A controller can be actively maintained and still carry a vulnerability class that no amount of maintenance can eliminate. The question worth asking is what the controller is built from, not just who is building it.

C and C++ put memory management responsibility on the developer. When that responsibility is handled imperfectly, the result can be a buffer overflow, a use-after-free, or a heap corruption bug. The critical word is "when," not "if." Two-thirds of reported vulnerabilities in memory-unsafe programming languages still relate to memory issues (CISA), despite decades of investment in tooling, training, and coding standards. Google Project Zero's review of real-world exploits found that 75% of CVEs used in active attacks were memory safety flaws (Industrial Cyber).

The pattern is consistent enough that the conclusion is unavoidable: this is not a discipline problem. It is a structural one. These vulnerabilities can be found and patched in any given version. They cannot be designed out of a C or C++ codebase.

The NSA, CISA, the FBI, and allied cybersecurity agencies across the UK, Australia, Canada, and New Zealand looked at that evidence and reached a formal position: the only durable fix is a language-level one. In October 2024, CISA and the FBI attached a deadline: publish a memory safety roadmap by the end of 2025 or be in a position described as "dangerous and significantly elevating risk to national security."

The deadline has since passed.

Among every C and C++ ingress controller on the official Kubernetes documentation page, not one published a roadmap.

Alarmingly, 77% of the self-hosted ingress controllers listed on the official Kubernetes documentation page currently carry C or C++ memory safety exposure with no published remediation roadmap.

Traefik Proxy is written in Go, which is on the NSA and CISA approved memory-safe language list. Go's garbage collector makes use-after-free structurally impossible. Go's bounds checking makes buffer overflow structurally impossible. These are not mitigations. They are language-level guarantees that remove the vulnerability class from the possibility space entirely. Traefik Proxy has no memory safety debt, and therefore no roadmap to publish.

A migration that lands on a Go-native controller addresses migration risk and security risk in a single motion. A migration that lands on a C or C++ controller defers the memory safety question to the next CVE cycle.

Stagnation Risk

Stagnation risk is the risk almost nobody is talking about. It is also the one with the longest tail.

Ingress NGINX retirement is forcing a migration decision today. But the ingress controller is not the destination. It is the first step.

Every organization running Kubernetes at production scale follows a predictable architectural evolution, and the controller chosen today will either support that evolution or constrain it.

Traffic enters through the ingress layer. As application complexity grows, an API gateway becomes necessary to centralize authentication, rate limiting, routing policy, and API lifecycle management. As security requirements mature, WAF capability provides protection against application-layer attacks. As AI workloads arrive, an AI gateway governs token costs, content policy, model routing, and inference guardrails. As agents and agentic systems move into production, an MCP gateway governs which tools agents can call, under what permissions, and according to what policy constraints.

Each of these is a layer of runtime governance. Each one needs to be deployed, configured, and maintained. And here is where stagnation risk becomes concrete: if each layer requires a separate product from a separate vendor on a separate control plane, the operational overhead compounds with every gateway added. If adding WAF capability means ripping out the ingress layer and replacing it with something else, the organization paid the migration cost twice. If the AI gateway requires a different data plane than the API gateway, the security posture of the two layers cannot be governed consistently.

Stagnation risk is the risk of building the governance architecture on a foundation that cannot grow with it.

Traefik addresses stagnation risk through an architecture built specifically for this evolution. Traefik Proxy is the ingress layer. Traefik Hub extends it, through Helm chart upgrades, to API gateway with integrated Coraza WAF, to AI gateway with NVIDIA safety guardrails and token governance, to MCP gateway with TBAC-based agent access control, and to full API management with GitOps-driven lifecycle management. A single binary. A unified control plane. The Triple Gate Pattern: API Gateway as Gate 1, AI Gateway as Gate 2, MCP Gateway as Gate 3, each adding a layer of runtime governance on the same Go-native, memory-safe foundation.

This means the organization that migrates to Traefik today does not pay a migration cost again when it adds WAF. It does not adopt a new vendor when it needs AI governance. It does not rearchitect the data plane when agents arrive. Each capability is a Helm upgrade on the same foundation. The stagnation risk collapses to near zero because the journey was planned for from the beginning.

The controller chosen today is not just an answer to the migration question. It is the foundation every subsequent decision gets built on. Choosing wrong means paying twice.

A Further Dimension for European Organizations

For organizations operating in European markets, all three risks carry an additional regulatory layer with hard dates. The EU Cyber Resilience Act, which entered into force in December 2024, requires that products with digital elements be designed with security from the outset, with secure practices as a design-phase requirement. Vulnerability reporting obligations become mandatory on September 11, 2026. Full compliance, including the requirement that non-compliant products cannot be sold in the EU market, takes effect on December 11, 2027. An ingress controller sits directly in the path of traffic to products subject to CRA compliance. The language-level security posture of that controller is inside the compliance perimeter. The end of 2025 CISA deadline has passed. The September 2026 CRA reporting deadline is six months away.

All Three Risks Are Addressable. Now Is the Time.

The Ingress NGINX retirement created a forcing function. Teams are doing the work regardless: auditing clusters, mapping configurations, planning cutovers. The operational investment is already in motion.

That investment is the reason now is exactly the right time to address all three risks, not just one. The incremental cost of choosing a Go-native, journey-ready controller over a C or C++ alternative is close to zero at the point where the migration is already running. The long-term cost of not making that choice accumulates with every security advisory, every compliance review, and every governance layer that has to be rebuilt on a foundation that was not designed for it.

Migration risk: addressed by the Traefik Ingress NGINX migration tool, the 90%+ annotation coverage of the Ingress NGINX Provider, and the step-by-step migration guide at ingressnginxmigration.org.

Security risk: addressed by Traefik Proxy's Go-native architecture, which eliminates the memory safety vulnerability class by design, with no debt and no roadmap required.

Stagnation risk: addressed by Traefik Hub's unified platform, which extends from ingress to API gateway, WAF, AI gateway, MCP governance, and API management through Helm upgrades on a single binary and a single control plane.

Three risks. One foundation. Now is the right time to address all of them.

Ready to start? Run the open-source migration tool at github.com/traefik/ingress-nginx-migration and generate your compatibility report in minutes. For enterprise migrations at scale, Traefik Labs provides dedicated migration support at traefik.io/solutions/oss-support.

]]>

OWASP Agentic Top 10: Where Should These Controls Be Enforced?

Zaid Albirawi, Immánuel Fodor — Wed, 25 Mar 2026 13:55:17 GMT

The OWASP Top 10 for Agentic Applications, published in December 2025, defines the ten most critical security risks for autonomous AI agents. It is quickly becoming the standard framework enterprise security teams use to evaluate agent governance solutions, and multiple vendors are publishing their coverage maps.

This is useful. The agentic AI space needed a shared vocabulary for risk, and OWASP provided one.

But the framework raises an architectural question that most coverage announcements skip over: where should these controls actually be enforced?

Key Takeaways

8 of 10 OWASP Agentic risks describe scenarios where the agent itself is the compromised component. Governance running inside that same process is operating in a compromised environment.
Infrastructure-layer enforcement (at the gateway) works even when the application layer is compromised, because it sits outside the blast radius.
Traefik Hub's Triple Gate covers 6 of 10 risks with strong gateway-layer enforcement, 2 with moderate coverage, and 2 with partial coverage. We'd rather publish an honest map than claim 10/10 coverage that doesn't survive architectural scrutiny.

The Enforcement Boundary Problem

Eight of the ten OWASP Agentic risks describe scenarios where the agent itself is the compromised or malfunctioning component. The agent's goals get hijacked (ASI-01). The agent misuses its tools (ASI-02). The agent escalates its own privileges (ASI-03). The agent's memory gets poisoned (ASI-06). The agent goes rogue (ASI-10).

This creates a problem for governance solutions that run inside the agent process. If the agent is hijacked, the governance middleware running in that same process is operating in a compromised environment. If the agent is rogue, application-layer kill switches depend on the rogue process honoring them. A policy engine that runs as an SDK import in the agent's code can be bypassed by the same exploit that compromised the agent.

The pattern is familiar. It's the same reason network firewalls exist alongside host-based security. It's also the same reason API gateways enforce authentication rather than trusting every microservice to check its own tokens. Enforcement at the infrastructure layer works even when the application layer is compromised, because it sits outside the blast radius.

Traefik Hub's Triple Gate Architecture (API Gateway, AI Gateway, MCP Gateway with TBAC) operates at the gateway layer. Every agent request, model interaction, and tool invocation passes through these gates regardless of the agent framework, language, or deployment model. The agent cannot modify, disable, or bypass gateway enforcement because the gateway runs in a separate process, in a separate trust boundary, with separate credentials.

This post maps Traefik Hub's capabilities against each of the ten OWASP Agentic risks, with an honest assessment of where gateway-layer enforcement is strong, where it contributes but doesn't fully own the risk, and where other layers need to fill the gap.

ASI-01: Agent Goal Hijack

Risk: Adversary manipulates agent objectives through prompt injection, context manipulation, or goal drift.

This is where the composable safety pipeline earns its keep. The AI Gateway inspects every prompt on the wire, both inbound (user requests) and outbound (model responses), through four guard tiers running in parallel.

Tier 1 (Regex Guard) catches known prompt injection signatures deterministically:

request:
  - jsonQueries:
      - ".messages[].content"
      - ".prompt"
    block: true
    reason: "prompt_injection_detected"
    entities:
      - "(?i)ignore.*(previous|above|prior).*(instructions|prompt)"
      - "(?i)you are now.*(new|different|unrestricted)"
      - "(?i)disregard.*(all|any).*(rules|guidelines|policies)"

Sub-millisecond, zero external dependencies, deterministic. This catches the common patterns.

Tier 3 (NVIDIA Safety NIMs) catches the uncommon ones. Jailbreak Detection NIM identifies prompt manipulation attempts that don't use obvious keywords. Topic Control NIM enforces conversation boundaries based on guideline prompts. These run on GPU and take 30-200ms per NIM, but with parallel execution, the total time equals the slowest guard, not the sum. For the full four-tier pipeline architecture, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Why gateway-layer matters here: A compromised agent cannot disable the gateway's prompt inspection. Application-layer policy checks run inside the same process the attacker is trying to hijack. The gateway inspects the actual payload on the wire, not a representation the agent chooses to expose.

Gap: No behavioral drift detection over time (detecting gradual goal shift across a multi-turn conversation). Each request is inspected independently.

ASI-02: Tool Misuse & Exploitation

Risk: Agent invokes tools beyond intended scope, with unauthorized parameters, or for unintended purposes.

This is where TBAC (Tools/Tasks/Transactions-Based Access Control) operates. The MCP Gateway enforces tool authorization before the request reaches the tool server.

TBAC provides three dimensions of control:

Tools: Which specific MCP tools each agent can invoke based on their identity.
Tasks: Which workflows or sequence of tool calls are allowed.
Transactions: Parameter-level constraints and validation per tool call.

Rate limiting per tool prevents abuse through rapid repeated invocations. Human-in-the-loop approval flows gate high-risk tool calls on explicit human confirmation before execution. Every tool invocation is logged with full identity, parameters, outcome, and timestamp.

Why gateway-layer matters here: TBAC enforces tool authorization at the infrastructure layer, not inside the agent's application code. A capability model defined in application code can be bypassed if the agent is compromised or if a developer misconfigures the SDK. TBAC cannot be bypassed because the MCP Gateway terminates the connection before the tool server receives it. The agent never talks directly to the tool; it talks to the gateway, which decides whether to forward the request.

This is the control we think matters most in the agentic era, and it only works at the infrastructure layer.

ASI-03: Identity & Privilege Abuse

Risk: Agents operate with excessive permissions, impersonate users, or escalate privileges.

JWT identity propagation flows through all three gates. The same identity context that governs API authentication (Gate 1) also governs token budgets (Gate 2) and tool permissions (Gate 3). There are no identity gaps between layers.

Identity can be managed per-user, per-team, and per-API-key enforcement via JWT claims. For fine-grained per-user rate/token/tool limiting, a JWT middleware extracts claims into headers via forwardHeaders, and Traefik middlewares will enforce limiting based on those headers.

TBAC restricts each agent identity to only the tools required for its task. No ambient authority. An agent authorized for read_database cannot invoke write_database even if the MCP server exposes both tools.

Why gateway-layer matters here: Identity verification at the gateway happens before the agent processes the request. Application-layer identity checks depend on the agent framework correctly extracting and honoring identity, which means the agent must be trusted to enforce its own permissions. The gateway enforces identity regardless of whether the agent is trustworthy.

ASI-04: Supply Chain Vulnerabilities

Risk: Compromised agent components, untrusted plugins, tampered models or tool definitions.

Honest assessment: partial coverage.

What the gateway provides today:

Air-gapped deployment. The full Traefik Hub stack runs without external dependencies, eliminating runtime supply chain exposure.
FIPS 140-2 validated, FIPS 140-3 ready. Cryptographic supply chain validated via native Go BoringCrypto module. No external crypto libraries.
Multi-vendor safety pipeline. Each guard tier comes from an independent vendor (custom regex, Microsoft Presidio, NVIDIA, IBM). Compromising one vendor's component doesn't compromise the pipeline.

What the gateway does not provide today: MCP server integrity verification (hash validation of tool definitions), plugin signing, or tamper detection at the gateway layer.

Architectural note: Supply chain integrity for agent internals (application modules, model weights) is inherently a build-pipeline concern, not a gateway concern. The gateway can verify what passes through it, but it cannot verify the internal integrity of components it doesn't serve. MCP server manifest verification (SHA-256 hashing of tool definitions at the gateway) is on our roadmap for Q2 2026. Build-pipeline integrity (Sigstore, SBOM generation, container image scanning) is complementary and should be addressed at the CI/CD layer.

ASI-05: Unexpected Code Execution

Risk: Agents generate and execute code without proper sandboxing or validation.

Assessment: partial coverage.

The gateway provides prevention, not containment. TBAC can prevent an agent from accessing code execution tools entirely. If an agent identity isn't authorized to invoke a run_code tool, the MCP Gateway blocks the request before it reaches the tool server. Parameter validation can constrain what the agent passes to tools that do execute code.

The gateway cannot sandbox execution that happens inside a tool's runtime. If an agent has authorized access to a code execution tool, the gateway doesn't inspect or contain what happens inside that tool after the request is forwarded.

Prevention is the stronger control for most deployments: a tool that cannot be invoked cannot produce unexpected execution. But containment matters too, and it's better addressed by container security contexts, language-level sandboxes (like LangChain's sandboxed code execution), or Kubernetes security policies.

ASI-06: Memory & Context Poisoning

Risk: Adversary corrupts agent memory, conversation history, or RAG context to influence behavior.

Tier 4 (IBM Granite Guardian) provides two capabilities that no other guard tier offers: hallucination detection and RAG quality assessment. The hallucination detection configuration inspects responses using the request history as context:

response:
  systemPrompt: "hallucination"
  useRequestHistory: true
  blockConditions:
    - reason: hallucination_detected
      condition: Contains("yes")

When useRequestHistory is set, the guard evaluates whether the model's response is grounded in the context that was provided. If the model generates claims not supported by the retrieved documents, the guard blocks the response before it enters the agent's memory.

Regex Guard (Tier 1: deterministic pattern matching against known injection signatures) catches known poisoning patterns in both requests and responses. NVIDIA NIMs (Tier 3: GPU-based semantic analysis) catch semantically poisoned content that doesn't match pattern rules. For a full walkthrough of all four tiers and how they compose, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Why gateway-layer matters here: The gateway inspects the model's actual response before the agent processes it. Application-layer memory integrity checks run after the potentially poisoned content has already entered the agent's context window. The gateway catches poisoning at the wire, not after ingestion.

ASI-07: Insecure Inter-Agent Communication

Risk: Agent-to-agent messages lack authentication, encryption, or trust verification.

The MCP Gateway is the communication channel. Every agent-to-tool call passes through it. The gateway terminates the inbound connection, verifies the agent's identity via JWT, applies TBAC policies, and opens a new authenticated connection to the tool server. TLS secures both legs.

You cannot have insecure inter-agent communication when every message passes through an authenticated, encrypted, audited gateway. Application-layer encrypted channels between agents require each agent to correctly implement and maintain encryption, which is a much larger attack surface.

The complete audit trail captures every inter-agent interaction with full identity, tool, parameter, and outcome detail. Traefik also supports request mirroring, which forwards live traffic to an external tool or analysis service in parallel with the primary flow. This enables real-time threat detection and behavioral monitoring alongside the forensic record. Together, these address both prevention (authentication, encryption) and detection (live analysis and forensic audit).

ASI-08: Cascading Failures

Risk: One agent's failure propagates through multi-agent systems causing systemic breakdown.

The gateway is the natural circuit breaker between systems. Three capabilities address this:

Failover Router builds circuit breaker chains across LLM providers; When a primary backend responds with a configured error (429, 500-504), the request replays to a fallback with independent model selection, API keys, and metrics. All safety policies continue to apply on every fallback. Governance doesn't degrade when the model does.

Graceful error handling via onDenyResponse returns structured HTTP 200 refusals instead of HTTP 403 errors. Agents parse refusals as valid conversation turns and continue operating. Without this, a single guardrail block crashes an entire multi-step workflow.

Token rate limiting with proactive estimation blocks over-budget requests before they reach the LLM. This prevents resource exhaustion cascades where one agent's runaway consumption starves others of capacity.

Why gateway-layer matters here: Application-layer circuit breakers fail when the application itself is failing. An agent that is crashing cannot reliably execute its own failure handling logic. The gateway operates outside the failing system.

ASI-09: Human-Agent Trust Exploitation

Risk: Agent manipulates human operators through social engineering or trust exploitation.

Content safety filtering through NVIDIA NIMs (22+ safety categories) filters manipulative, deceptive, or harmful content in model responses before they reach users. IBM Granite Guardian catches factually incorrect claims (hallucinations) the agent might use to mislead operators. Human-in-the-loop approval flows via TBAC require explicit human confirmation for sensitive operations.

Assessment: moderate coverage. The gateway filters manipulative content and gates sensitive actions, but it cannot fully prevent subtle social engineering within otherwise legitimate conversation. This risk also requires organizational controls (training, escalation procedures) and application-layer UX design (clear agent identity disclosure, confidence indicators).

ASI-10: Rogue Agents

Risk: Agents behave in unintended, uncontrollable, or malicious ways outside governance.

Token rate limiting and quota enforcement set hard ceilings on agent resource consumption. Proactive estimation blocks over-budget requests before GPU cost is incurred. TBAC restricts the blast radius to authorized tools only, so even a rogue agent can only reach the tools it was explicitly granted access to. Per-agent rate limiting prevents runaway request loops.

Assessment: moderate coverage. The gateway provides containment ceilings but not real-time behavioral detection. There is no behavioral anomaly detection today: no baseline modeling of normal invocation patterns, no deviation alerting, no agent kill switch. The data for anomaly detection is already being collected in the audit trail; the analysis layer is the gap. Whether that analysis belongs at the gateway layer or in a dedicated observability platform is an open architectural question. What the gateway provides today is the data collection infrastructure that either approach would require.

Coverage Breadth vs. Enforcement Depth

The OWASP Agentic Top 10 is a useful framework. But evaluating governance solutions purely on how many of the ten risks they "cover" misses the more important question: how deeply and reliably is that coverage enforced?

A governance system that checks all ten boxes from inside the agent process provides broad coverage in a narrow trust boundary. A governance system that enforces eight of ten risks from outside the agent's trust boundary provides deep, bypass-resistant coverage on the risks that matter most.

The right architecture uses both layers:

Layer	What It Governs	Examples
Infrastructure (Gateway)	Identity, tool authorization, content safety, traffic resilience, audit	Traefik Hub (API + AI + MCP Gateway)
Application (Agent Framework)	Agent logic, memory management, code sandboxing, behavioral policies	LangChain, CrewAI, Semantic Kernel
Platform (Kubernetes/OS)	Container isolation, network policies, resource limits	Security contexts, gVisor, network policies
Pipeline (CI/CD)	Supply chain integrity, model provenance, image scanning	Sigstore, SBOM, container scanning

No single layer covers all ten risks alone. But the infrastructure layer is the one that works even when the application layer is compromised. For organizations also adopting Forrester's AEGIS framework, we published a companion post mapping AEGIS's six domains to infrastructure-layer enforcement.

Our Coverage Map

OWASP Risk	Gateway Coverage	What Gateway Provides	What Other Layers Add
ASI-01: Goal Hijack	Strong	4-tier safety pipeline, parallel execution	Behavioral drift detection (app-layer)
ASI-02: Tool Misuse	Best-in-class	TBAC: tool auth, parameters, rate limits, human-in-loop	Framework-level tool selection (app-layer)
ASI-03: Identity Abuse	Strong	JWT propagation, per-user/team enforcement, least-privilege	IAM integration (platform-layer)
ASI-04: Supply Chain	Partial	Air-gapped deployment, FIPS crypto, multi-vendor guards	SBOM, image scanning, model provenance (pipeline-layer)
ASI-05: Code Execution	Partial	Tool access prevention via TBAC	Execution sandboxing (platform-layer)
ASI-06: Memory Poisoning	Strong	Hallucination detection, RAG quality, bidirectional inspection	Memory management policies (app-layer)
ASI-07: Inter-Agent Comms	Strong	MCP Gateway, mTLS, authenticated channels, audit	N/A (gateway is the channel)
ASI-08: Cascading Failures	Strong	Failover Router, circuit breakers, graceful errors, token limits	Application-layer retry policies
ASI-09: Trust Exploitation	Moderate	Content safety filtering, human-in-loop, audit	UX design, organizational controls
ASI-10: Rogue Agents	Moderate	Token ceilings, tool blast radius limits, rate limiting	Behavioral anomaly detection (roadmap)

Six strong, two moderate, two partial. We'd rather publish an honest map than claim ten out of ten coverage that doesn't survive architectural scrutiny.

What's Next

The OWASP Agentic Top 10 mapping detailed in this post reflects Traefik Hub v3.20 capabilities as of March 2026. The composable safety pipeline, TBAC, Failover Router, token rate limiting, and graceful error handling are all available in Early Access today, with general availability planned for late April 2026.

We encourage organizations evaluating agent governance to ask every vendor not just which risks they cover, but where their controls are enforced and what happens when the agent itself is the thing that's compromised.

For a complementary framework-level perspective, see our mapping of Forrester's AEGIS framework to infrastructure-layer enforcement. And for a deep technical walkthrough of the composable safety pipeline referenced throughout this post, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Traefik Hub v3.20 is available as an Early Access release. To try the composable safety pipeline and the other features covered in this post, sign up for Early Access.

Frequently Asked Questions

What is the OWASP Top 10 for Agentic Applications?

The OWASP Top 10 for Agentic Applications is a risk taxonomy published in December 2025 that identifies the ten most critical security risks for autonomous AI agents. The risks range from agent goal hijacking (ASI-01) and tool misuse (ASI-02) to cascading failures (ASI-08) and rogue agents (ASI-10). It is maintained by the OWASP GenAI Security Project and is becoming the standard framework for evaluating AI agent governance solutions.

What is the difference between application-layer and infrastructure-layer AI agent governance?

Application-layer governance runs inside the agent process (as an SDK, middleware library, or framework plugin) and governs the agent's logic. Infrastructure-layer governance runs at the network gateway, outside the agent's trust boundary, and governs the traffic. The key distinction: if the agent is compromised through prompt injection, jailbreak, or privilege escalation, application-layer governance is compromised with it. Infrastructure-layer governance continues to enforce because it runs in a separate process with separate credentials.

What is TBAC (Tools/Tasks/Transactions-Based Access Control)?

TBAC is Traefik Hub's access control model for AI agents, enforced at the MCP Gateway. It provides three dimensions of control: which tools an agent can invoke, which tasks (workflows) it can perform, and what transaction parameters are permitted. TBAC operates at the infrastructure layer, meaning the agent runtime cannot modify, bypass, or disable these controls.

How does the OWASP Agentic Top 10 relate to Forrester's AEGIS framework?

The OWASP Agentic Top 10 is a risk taxonomy (what can go wrong). Forrester's AEGIS framework is a governance framework (what controls to implement). They are complementary: OWASP identifies the risks, AEGIS defines the organizational and technical controls to mitigate them. Both map to infrastructure-layer enforcement. See our AEGIS framework mapping for the governance-framework perspective.

Can any single product cover all 10 OWASP Agentic risks?

No. The ten risks span multiple enforcement layers: infrastructure (gateway), application (agent framework), platform (Kubernetes/OS), and pipeline (CI/CD). A governance system that claims 10/10 coverage from a single layer is either defining "coverage" loosely or operating within a narrow trust boundary. The right architecture uses defense-in-depth, with the infrastructure layer providing the controls that work even when the application layer is compromised.

]]>

When Every File is an Attack Vector: Why Agent Governance Must Live Outside the Runtime

Immánuel Fodor — Wed, 25 Mar 2026 12:25:09 GMT

The LiteLLM supply chain compromise on March 24 was a pip package that stole credentials, such as SSH keys, cloud provider credentials, Kubernetes configs, database passwords, and API keys. That is a serious incident, and the architectural lessons it exposed are significant. But the harder conversation is about what comes next.

AI agents in 2026 do not just call APIs. They read your filesystem. They process your documents, configs, and code. They maintain persistent memory across sessions. They spawn sub-agents. They act on your behalf at machine speed, often without a human reviewing each step.

The LiteLLM attack exploited the Python package supply chain. The next class of attacks will exploit something far larger: the entire input surface of the agent itself.

Your Filesystem Is the New Codebase

When a coding agent operates with filesystem access, every file it reads becomes a potential instruction. A poisoned PDF in a shared folder can carry hidden prompt injections. A modified .env file can redirect agent behavior. A malicious markdown document in a project directory can instruct an agent to exfiltrate data, modify code, or propagate itself to other files the agent touches.

This is not theoretical. Microsoft's security team has documented how indirect prompt injection works in practice: an attacker never talks to the agent directly. Instead, they poison the data sources the agent reads. When the agent retrieves that content through tool calls (emails, documents, support tickets, web pages, database entries), the malicious instructions ride along, invisible to human reviewers, fully legible to the model.

The structural problem is that LLMs process instructions and data as tokens in the same context window. The agent cannot reliably distinguish between a system prompt from its operator and an instruction embedded in a document it just ingested. As researchers in the "Agents of Chaos" study (MIT, Harvard, Stanford, CMU) put it: prompt injection is a structural feature, not a fixable bug.

In a vibe coding workflow, where developers describe intent in natural language and agents generate entire applications, this surface expands further. The agent reads project files, configs, dependencies, and documentation as context for its work. Every one of those files is now part of the instruction set. Every text file can carry a payload. The attack surface is not the chat interface. It is the entire data environment.

Vibe Coding Accelerates the Problem

The rise of vibe coding has made this structural vulnerability operationally urgent.

A CodeRabbit analysis of 470 open-source GitHub pull requests found that AI co-authored code contained 2.74x more security vulnerabilities than human-written code, along with 75% more misconfigurations. Veracode's GenAI Code Security Report found that 45% of AI-generated code introduces security vulnerabilities. These are not edge cases. They are base rates.

The Moltbook incident in February 2026 demonstrated the consequences. The AI-agent social network was built entirely through vibe coding, with no human-written code. Wiz discovered a misconfigured Supabase database exposing 1.5 million authentication tokens and 35,000 email addresses to the public internet. The root cause was not a sophisticated attack. It was the absence of a security review in a process optimized for speed.

There is a pattern here. Coding agents optimize for making code run, not making code safe. Research from Columbia University documented how agents routinely remove validation checks, relax database policies, and disable authentication flows to resolve runtime errors. The constraint causing the error is sometimes the security guard, and the agent removes it to satisfy the prompt.

When agents generate code at this velocity and with these failure modes, treating the agent's runtime as a trusted environment becomes untenable. The code it writes may be insecure. The files it reads may be poisoned. The dependencies it pulls may be compromised (as LiteLLM demonstrated). Every layer of the agent's input and output is potentially adversarial.

The Impersonation Problem

There is a dimension to this that goes beyond credential theft.

An AI agent with access to your filesystem, communication history, code repositories, and working documents has enough context to impersonate you. Not in the crude phishing sense of faking an email header, but in the deeper sense of replicating your communication patterns, decision-making style, and domain knowledge across every platform you use.

A compromised agent does not just steal your credentials. It becomes a distorted version of your digital identity, capable of acting on your accounts, writing in your voice, and making decisions that appear to be yours. The persistent memory that makes agents useful (remembering your preferences, your projects, your colleagues) is the same memory that makes compromise catastrophic. The agent knows enough to be you, and it operates faster than you can intervene.

This is why agent authorization cannot be implemented in the agent's runtime. If the runtime is compromised, every control embedded in it, including identity verification, access policies, and behavioral guardrails, is compromised with it.

Claws Need Shells

The security community is converging on a principle that infrastructure engineers have understood for decades: enforcement must be independent of the thing being enforced.

A firewall does not run inside the application it protects. A network policy does not depend on the container it restricts. In the same way, agent governance cannot live inside the agent runtime. The runtime is what gets compromised, whether through supply-chain attacks (LiteLLM), poisoned context (indirect prompt injection), or insecure generated code (vibe coding failures). Controls that depend on the integrity of that runtime are not controls. They are assumptions.

Defense in depth for AI agents requires multiple independent enforcement layers, each operating in its own trust boundary:

At the API layer: Rate limiting, authentication, and data loss prevention detect anomalous behavior at the network edge. An agent suddenly exfiltrating credential files to an unfamiliar domain triggers enforcement that the agent itself cannot disable.

At the AI interaction layer: Content safety, prompt defense, and PII filtering govern what goes into and comes out of the LLM. These controls run at the infrastructure layer, outside the application process, so they remain intact even when the runtime is compromised.

At the tool access layer: Task-scoped, tool-level authorization ensures that a compromised agent cannot pivot laterally to systems beyond its permitted scope. Authorization is cryptographically attested by an identity provider and enforced at the gateway, not granted by application code that the agent controls.

The critical property is independence. Each layer operates in its own trust boundary. Compromising the agent's runtime (through a poisoned file, a malicious dependency, or a prompt injection) does not disable enforcement at the other layers. This is what separates architecture from aspiration.

The Governance Gap Will Not Close Itself

The data on this is stark. According to Okta's AI at Work report, 91% of organizations are already deploying AI agents, yet only 10% have a well-developed strategy for managing non-human identities. The CrowdStrike 2026 Global Threat Report documented an 89% year-over-year increase in AI-enabled adversary attacks, with average breakout time falling to just 29 minutes and the fastest observed breakout occurring in 27 seconds. In one intrusion, data exfiltration began within four minutes of initial access.

A Dark Reading poll found that 48% of cybersecurity professionals now identify agentic AI as the top attack vector heading into 2026, outranking deepfakes, board-level cyber recognition, and passwordless adoption.

Application-level controls are necessary but insufficient. When the agent's runtime is the attack surface (not just the network or the endpoint, but the code, the context window, and the filesystem), governance must operate at a layer the agent cannot reach. That means infrastructure-layer enforcement: independent of the agent framework, below the application, governing every API call, LLM interaction, and tool invocation as it crosses the network.

Three Questions Every Team Should Ask

If a poisoned file entered your agent's context window, which controls would survive? If every guardrail runs inside the same runtime that processes the file, the answer may be none.
Can your agent disable its own governance? If access control, content safety, and tool authorization are implemented as libraries in the agent's process, a prompt injection or compromised dependency can override them. Infrastructure-layer enforcement operates in a separate trust boundary that the agent cannot modify.
How many independent enforcement layers exist between a compromised agent and your most sensitive systems? The LiteLLM breach showed what happens when the answer is zero. The vibe coding era, where agents generate code, read untrusted files, and spawn sub-agents at machine speed, makes this question existential.

The LiteLLM attack compromised a library. The next wave of attacks will compromise the files agents read, the code agents write, and the memory agents accumulate. The question is not whether your agent will encounter adversarial input. It is whether your architecture survives when it does.

Traefik Hub's Triple Gate Pattern (API Gateway, AI Gateway, MCP Gateway with TBAC) implements the independent, infrastructure-layer enforcement described in this post. For a technical walkthrough of how the composable safety pipeline works across these layers, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

]]>

The Agent Framework Wars Are Over. Everyone Won. That's the Problem.

Sudeep Goswami — Tue, 24 Mar 2026 13:34:20 GMT

Ask a platform architect how many cloud providers their organization runs. Watch their expressions. The answer is almost never one, and the reasons are always the same: different teams, different timing, different commitments made before the current strategy existed. The cloud was supposed to consolidate. It has been consolidating for 15 years, and most enterprises are still running more than one.

CI/CD was supposed to standardize. Jenkins still runs in the basement alongside GitHub Actions and whatever the platform team mandated two years ago. The service mesh question, Istio or Linkerd or Consul, was asked in 2019 and is still open at organizations that considered themselves decisive.

This is not a failure of enterprise decision-making. It is how enterprise infrastructure actually works. Commitments compound. Migration costs exceed the value of consistency. New teams make new bets. And the platforms themselves have no incentive to make switching easy, because lock-in is the business model.

Every major agent framework launched in the last twelve months is built on exactly this logic. Which means the governance problem coming at every CISO, Chief AI Officer, and platform architecture leader is not which framework to choose. It is what to do when the answer, inevitably, is all of them.

Twelve Months, Five Frameworks

The agent framework landscape consolidated from "many experiments" to "a few serious platforms" in roughly one year. The platforms that emerged are not startups or community projects. They are flagship products from the five largest technology companies in the world.

OpenAI Agents SDK, released March 2025, is a lightweight Python and TypeScript framework built around a deliberately minimal set of primitives: Agents, Handoffs, Guardrails, and Tracing. Provider-agnostic, supporting over 100 LLMs. The spiritual successor to Swarm, optimized for simplicity over configurability.

Google ADK, introduced at Google Cloud Next 2025, is an open-source framework designed for full-stack development of agents and multi-agent systems. Model-agnostic and deployment-agnostic in principle, optimized for Gemini and Vertex AI in practice. The same framework powering Google's own Agentspace and Customer Engagement Suite.

Microsoft Agent Framework, announced in October 2025, is the convergence of Semantic Kernel and AutoGen into a single, unified runtime that combines Semantic Kernel's enterprise foundations with AutoGen's multi-agent orchestration patterns. Open standards across MCP, A2A, and OpenAPI. Python and .NET. Routes toward Azure AI Foundry.

NVIDIA NemoClaw, unveiled at GTC 2026 last week, is the enterprise extension of OpenClaw, a full agent runtime with hardware-optimized inference, enterprise RBAC, signed skill registries, and the Nemotron model stack underneath. Jensen Huang called it the operating system of agentic computers.

LangChain ecosystem (LangGraph + LangSmith) holds a deep enterprise install base, highly recommended framework for production agents, with over 70 million monthly downloads across the LangChain ecosystem and deployments in production at LinkedIn, Uber, Cisco, BlackRock, and JPMorgan. LangSmith provides observability.

And behind these five: CrewAI, PydanticAI, LlamaIndex, Amazon Bedrock Agents, IBM Watsonx Orchestrate, and a dozen more in active enterprise deployment.

The Consolidation That Will Not Come

The reasonable expectation, looking at previous infrastructure categories, is that the market consolidates. Two or three frameworks win. Enterprises standardize. The governance story simplifies.

That is not what is happening here, and the reason is structural. These frameworks are not neutral tools. They are distribution mechanisms for model consumption, cloud compute, and inference revenue. No platform company has an incentive to make their framework interchangeable with a competitor's. The switching costs are a feature, not a bug.

Enterprises are not going to pick one. A financial services firm running on Azure will use Microsoft Agent Framework for its core workflows. Its data science team, hired from Google, will prototype with ADK. Its NVIDIA DGX cluster will run NemoClaw for latency-sensitive workloads. Its compliance team will insist on LangSmith observability because that is what they already have. Its newest engineering team will ship with OpenAI Agents SDK because it was the fastest to working prototype. The frameworks are too useful, the switching costs are too high, and the organizational inertia is too real. Multi-framework is the destination, not a temporary state.

The Governance Problem Nobody Has Solved

Every one of these frameworks ships with application-layer governance. Guardrails, tracing, access controls, audit logs, safety checks. Each vendor has invested seriously in making their governance story credible.

And every one of those governance implementations is specific to that framework.

OpenAI's guardrails govern OpenAI Agents SDK workflows. They do not govern NemoClaw agents. Google ADK's safety patterns apply inside ADK runtimes. They do not apply to Microsoft Agent Framework deployments. NemoClaw's signed skill registry and RBAC operate inside the NemoClaw runtime. They are unaware of the LangChain agents running alongside them on the same Kubernetes cluster.

As the framework landscape fragments, application-layer governance fragments with it. Each new framework an enterprise adopts adds a new governance silo: new policies to configure, new audit logs to aggregate, new access control models to reconcile with enterprise identity, and new safety configurations to maintain. The governance overhead scales with the number of frameworks, and the coverage is always incomplete at the edges where frameworks interact.

There is a more serious problem underneath this. Application-layer governance, by definition, lives inside the process being governed. When a NemoClaw agent calls an OpenAI endpoint through a LangChain routing layer inside a Microsoft Agent Framework orchestration workflow, a configuration that will exist in production before this year is out, and none of the application-layer governance from any of those frameworks can see the full picture. Each sees only its own slice of the execution. The inference call crosses network boundaries that each framework's internal controls cannot follow.

This is not an edge case. It is the normal operating condition of enterprise multi-agent deployments at scale.

The One Layer That Does Not Fragment

Infrastructure-layer governance does not know which framework built the agent. It governs the traffic.

An inference call from a NemoClaw agent and an inference call from a Google ADK agent are, at the network layer, identical: an HTTP request carrying a prompt to an LLM endpoint. The AI Gateway's safety pipeline, covering pattern matching, PII redaction, NVIDIA Safety NIMs, and hallucination detection, applies to both independently, without modifying either framework.

An MCP tool invocation from an OpenAI Agents SDK workflow and an MCP tool invocation from a Microsoft Agent Framework orchestration are, at the network layer, the same protocol: a tool call with an identity, a method, and parameters. Infrastructure-layer authorization validates both against the same policy, regardless of which runtime initiated the call.

A policy enforced at the network layer applies across the entire multi-framework agent estate. One set of policies. One audit log. One enforcement boundary. The governance overhead does not scale with the number of frameworks because it is not within any of them.

This is what Traefik Hub's Triple Gate architecture was designed to address. Three enforcement points, each targeting a distinct traffic pattern every agent generates regardless of runtime:

The AI Gateway intercepts every inference call: the safety pipeline, token rate limiting, semantic caching, and model failover, which apply universally to every LLM request, whether it originates from NemoClaw, ADK, OpenAI SDK, or LangGraph.
The MCP Gateway intercepts every tool invocation: infrastructure-layer authorization on the specific parameters of each tool call, independent of the role-based controls inside any particular framework's runtime.
The API Gateway governs the management plane: the configuration endpoints, monitoring interfaces, and orchestration APIs of the agent infrastructure itself, enforcing authentication, schema validation, and threat detection regardless of which framework generated the traffic.

Three patterns. Three enforcement points. Framework-agnostic, by design.

Why This Argument Gets Stronger Over Time

In a world with five competing frameworks running within the same enterprise, application-layer governance requires five separate configurations, five separate audit log streams, five separate safety policy maintenance cycles, and still provides no visibility at the boundaries where frameworks interact.

Infrastructure governance provides one. The multi-framework future makes that one more valuable, not less.

The pattern is already established in adjacent infrastructure categories. Enterprises do not maintain separate network security policies for each application stack. They do not run separate identity providers for each platform vendor's tooling. The governance lives at the layer that does not change when the application layer changes.

Agent infrastructure is arriving at the same conclusion. The application layer will continue to fragment: new frameworks, new runtimes, new vendor platforms, new enterprise bets. The infrastructure layer is where governance stabilizes.

The Strategic Question for Every Platform Leader

The question facing every CISO, Chief AI Officer, and platform architecture leader right now is not which agent framework to standardize on. That question has no good answer, because the organization is already running multiple frameworks and will continue to do so.

The question is: where does governance live in a multi-framework agent estate, and how does it scale as the number of frameworks grows?

If the answer is "inside each framework," the honest follow-up is: who maintains five separate governance configurations, how are they kept consistent, and what happens at the boundaries where frameworks interact?

If the answer is "at the infrastructure layer," the follow-up is: which traffic governance architecture covers all three patterns, inference calls, tool invocations, and management APIs, across every framework without requiring modification to any of them?

That second answer is the one that scales. It is also the answer that matches how enterprise security has worked in every infrastructure category that came before.

The agent framework wars are over. Everyone won. The governance story for a world where everyone won is an infrastructure-layer story, and that story is only beginning to be written.

]]>

Ingress NGINX is Out and There’s Only One Realistic Alternative

Emile Vauge — Tue, 24 Mar 2026 07:40:59 GMT

Traefik v3.7 brings 80+ NGINX Ingress annotations, snippet support, and a zero-downtime migration path.

Last July, when we first introduced Ingress NGINX native support to Traefik, we knew something was happening, but we didn't yet know how soon the situation would escalate. Then came November: the Kubernetes SIG Network and Security Response Committee officially announced that the Ingress NGINX Controller would be retired in March 2026. No new releases. No bug fixes. No security patches.

As of this month, that deadline has arrived. If you're running Ingress NGINX in production—and millions of clusters still are—you're now running unmaintained software with known architectural vulnerabilities.

Over the past months, we've been on a singular mission: make the migration from Ingress NGINX a non-event. Not a multi-quarter project. Not a forced rearchitecture. A non-event. Today, I want to share where we've arrived, what's new, and why we believe Traefik is the only path that doesn't ask you to choose between safety and disruption.

And the path we've taken has been validated by the industry. This morning, we're proud to announce that IBM Cloud, Nutanix, OVHcloud, SUSE, TIBCO, and others have each independently chosen Traefik as their strategic ingress controller and Gateway API solution. These aren't experiments or evaluations. These are production commitments from organizations that evaluated every option on the table and concluded that Traefik's approach, native Ingress NGINX compatibility combined with full Gateway API support, is the right foundation for their Kubernetes networking stack going forward.

The Alternatives All Have the Same Problem

Since the retirement announcement, the ecosystem has responded along two lines. Neither solves the fundamental issue.

The First Approach: Migrate to a Different Configuration Format

Every major proxy vendor (e.g., Kong, HAProxy, NGINX F5, cloud-native providers) has published migration guides. We've read them all. They all share the same fundamental limitation: they require you to rewrite your configurations immediately.

Either you convert your NGINX annotations to a completely different set of vendor-specific annotations, or you jump straight to Gateway API resources. In both cases, every annotation, every routing rule, every customization must be manually converted to a different syntax, a different mental model, a different set of abstractions to land you on a new controller UX that you do not master yet. Under deadline pressure, this is a recipe for outages.

The Second Approach: Fork Ingress NGINX and Keep Running It

Following the retirement announcement, a maintenance fork emerged to keep the project alive by providing security patches. It's a tempting narrative; just switch the image and stay safe. But as we explained in detail in our analysis, a fork doesn't fix the fundamental architectural flaws. The configuration injection surface that enabled #IngressNightmare isn't a bug to patch. It's a consequence of how NGINX-based controllers work—template rendering, C/C++ memory safety risks, etc.

A fork inherits all of these structural vulnerabilities. And it anchors you to a legacy technology with no path forward. Finally, if the original maintainers gave up, how could a third-party vendor make this project safe to use?

Meanwhile, InGate, the proposed from-scratch successor, has also been announced as end-of-life in March as well. The official Kubernetes path forward for ingress is Gateway API, but Do It Yourself.

Traefik’s Approach: The Only Drop-In Replacement

We took a fundamentally different approach. Traefik's NGINX Ingress Provider reads your existing Ingress resources and their nginx.ingress.kubernetes.io annotations natively. No conversion scripts. No YAML rewriting. No dual-maintenance period where you're running two sets of configurations side by side.

This isn't a compatibility shim or a best-effort translation layer. It's a full provider that understands the Ingress NGINX annotation contract and translates it into Traefik's internal object model securely, without the configuration-injection vulnerabilities that plague template-based approaches.

This makes Traefik the only drop-in replacement for Ingress NGINX in the industry. And critically, Traefik is also the only solution that supports both Ingress NGINX annotations and Gateway API on the same cluster, on the same instance, at the same time. This is what makes a truly seamless migration possible, and this is exactly why IBM Cloud, Nutanix, OVHcloud, SUSE, TIBCO, and others chose Traefik as their path forward.

What's New: A Data-Driven Expansion of Annotation Support

From the beginning, we took a data-driven approach to deciding which annotations to support and in what order. Our open-source ingress-nginx-migration tool doesn't just analyze your cluster, when users opt in, it shares anonymized, aggregated statistics with us. No resource names, no namespaces, no configurations. Just annotation counts and usage patterns.

The result: we now have an excellent overview of real-world annotation usage across hundreds of diverse deployments, from small startups running a handful of Ingresses to large enterprises with thousands. This data has directly shaped our implementation priorities, ensuring that every annotation we add addresses actual production needs, not hypothetical edge cases.

Introducing Traefik v3.7: 80+ NGINX Ingress Annotations, 90% of Real-World Usage, Available Now

Today we are releasing Traefik v3.7 early access, and it's the biggest leap forward yet for Ingress NGINX migration. This release brings support for over 80 NGINX Ingress annotations natively, covering more than 90% of real-world usage patterns we've observed across hundreds of production clusters. Authentication, TLS, session affinity, CORS, routing, proxy settings, rate limiting, canary deployments, snippets, and more: they all work out of the box, with your existing Ingress manifests, unchanged.

This is the release that turns the Ingress NGINX sunset from a migration project into a configuration change.

Snippets: A Pragmatic Solution to the Long Tail

One of the most interesting findings from our migration tool data was the prevalence of configuration-snippet and server-snippet annotations. These are the escape hatches of the Ingress NGINX world (i.e., raw NGINX configuration directives injected directly into the generated config). They're also the primary vector behind the IngressNightmare vulnerabilities, and the reason a naive "just support all annotations" approach is a security non-starter.

But we can't ignore them either. Our data shows that snippets are heavily used in production, and a significant portion of "unsupported" Ingresses rely on them.

So we built something new: partial snippet support through a basic interpreter. Rather than blindly injecting raw NGINX configuration, which would recreate the exact security vulnerabilities we're trying to avoid, Traefik now parses snippet content and interprets a curated set of the most commonly used NGINX directives.

This covers the most common snippet use cases without enabling arbitrary configuration injection. It's a pragmatic middle ground: broad enough to unblock real migrations but strict enough to maintain Traefik's security-by-design guarantees.

Variable Interpolation

Alongside snippet support, Traefik now supports variable interpolation, initially focused on the headers manipulation use case. If your snippets use NGINX variables to set or modify request and response headers—referencing client IPs, URI components, or upstream information—Traefik maps them to their equivalent values in its own routing context.

This was another priority directly informed by our migration data. Header manipulation through snippets with variable interpolation is one of the most common patterns we observed, and without this support, even a well-parsed snippet would produce incorrect behavior. The architecture is designed to expand variable coverage to additional use cases based on community feedback.

Annotation Support: The Full Picture

Here's a snapshot of the 80+ NGINX Ingress annotations now supported by Traefik's NGINXIngress Provider, as available in the v3.7 early access.

For the full technical reference with behavioral notes and limitations, see the NGINX Ingress Provider documentation.

Authentication auth-type auth-secret auth-secret-type auth-realm auth-url auth-method auth-response-headers auth-signin auth-snippet auth-tls-secret auth-tls-verify-client auth-tls-verify-depth auth-tls-pass-certificate-to-upstream	SSL / TLS ssl-redirect force-ssl-redirect ssl-passthrough proxy-ssl-secret proxy-ssl-verify proxy-ssl-name proxy-ssl-server-name	Session Affinity affinity affinity-mode affinity-canary-behavior session-cookie-name session-cookie-path session-cookie-domain session-cookie-samesite session-cookie-secure session-cookie-max-age session-cookie-expires
Proxy Settings proxy-body-size client-body-buffer-size proxy-buffer-size proxy-buffers-number proxy-buffering proxy-max-temp-file-size proxy-connect-timeout proxy-read-timeout proxy-send-timeout proxy-request-buffering proxy-http-version proxy-redirect-from proxy-next-upstream proxy-next-upstream-timeout proxy-next-upstream-tries	CORS enable-cors cors-allow-credentials cors-allow-headers cors-allow-methods cors-allow-origin cors-max-age cors-expose-headers	Routing & Redirects use-regex rewrite-target app-root permanent-redirect permanent-redirect-code temporal-redirect temporal-redirect-code from-to-www-redirect server-alias
Load Balancing & Backend load-balance backend-protocol service-upstream upstream-vhost upstream-hash-by default-backend	Rate Limiting & Access Control whitelist-source-range allowlist-source-range limit-rps limit-rpm	Custom Headers custom-header x-forwarded-prefix
Canary Deployments canary canary-by-header canary-by-header-value canary-by-header-pattern canary-by-cookie canary-weight canary-weight-total	Error Handling custom-http-errors	Security / WAF enable-modsecurity modsecurity-snippet
Observability enable-access-log	Snippets configuration-snippet server-snippet

Many of these annotations were implemented by community contributors, and that momentum continues to grow. If you need support for annotations not listed here, or want to help expand coverage further, head to traefik/traefik#12631. Annotations are categorized by complexity (Basic, Medium, Complex), making it easy to find a contribution that matches your experience level.

More than half of Traefik's merged pull requests have historically come from external contributors. The Ingress NGINX Provider is proving to be no exception.

The Migration: Three Phases, Zero Downtime, Zero Risk

We designed the migration from Ingress NGINX to be a simple, three-phase process. All three phases uphold our core principles: zero downtime, progressive transition, and a smooth experience that lets you move at your own pace.

Phase 1: Assess Your Situation

Run the ingress-nginx-migration tool against your cluster. In minutes, you'll get a clear picture of your annotation usage and compatibility status: which Ingresses will migrate seamlessly, and which ones need attention. No manual audits are required.

Phase 2: Ingress NGINX Decommissioning

Deploy Traefik alongside your existing NGINX controller. Both will serve the same Ingress resources simultaneously. Progressively shift traffic to Traefik, one Ingress at a time, validating behavior with each step. Once you're confident, remove Ingress NGINX. There is no big-bang cutover, no downtime, and your production traffic remains secure while you eliminate the unmaintained software from your stack.

Phase 3: Progressive Transition to Gateway API

With Traefik handling your existing Ingress resources and their NGINX annotations, you're no longer under a deadline. When you're ready (not when a crisis forces you), you can begin converting individual Ingress resources to Gateway API. Traefik supports both simultaneously, allowing you to migrate one route at a time, validate behavior, and roll back if needed. This is a seamless, progressive modernization with zero downtime.

This three-phase approach is unique to Traefik. No other solution provides both a drop-in replacement for Ingress NGINX and a production-grade Gateway API implementation that runs side by side, enabling a fully progressive migration from legacy to modern at your own pace.

The retirement of Ingress NGINX doesn't have to be a crisis. We built a migration path for you because we believe no one should be forced to rearchitect under pressure. The Kubernetes ecosystem deserves better than that.

None of this would have been possible without the incredible contributions from the community. Developers from around the world have submitted pull requests adding new annotations, reporting edge cases, improving documentation, and testing the provider against their own production configurations. This effort is as much yours as it is ours, and we're deeply grateful for it.

If you haven't already, we invite you to try the Traefik v3.7 early access release against your own cluster. Run the migration tool, deploy Traefik alongside NGINX, and see for yourself how many of your Ingresses just work. Your feedback on this release will directly shape the final v3.7 GA.

The deadline is here. But for Traefik users, it's just another Tuesday.

Resources:

Migration tool: github.com/traefik/ingress-nginx-migration
Migration guide: doc.traefik.io/traefik/migrate/nginx-to-traefik
Early Access: Traefik v3.7 Early Access: v3.7.0-ea.2
Contribute annotations: traefik/traefik#12631
Full annotation reference: doc.traefik.io/.../ingress-nginx
Migration website: ingressnginxmigration.org
Why a fork won't save you: The Illusion of Safety

]]>

Hardware Has Never Cost More. Is Your Architecture Built for That Reality?

Sudeep Goswami — Thu, 19 Mar 2026 15:55:44 GMT

At NVIDIA GTC this week, the chairman of SK Group, which controls the world's largest supplier of high-bandwidth memory, told reporters that the global chip wafer shortage will likely persist until 2030. Not this year. Not next year. 2030.

The wafer supply deficit is running above 20% industry-wide. New fabrication capacity takes four to five years to come online. The root cause is not a temporary demand spike but a structural reallocation: AI infrastructure requires high-bandwidth memory at a scale that is pulling cleanroom capacity away from the standard DRAM and NAND that conventional enterprise servers depend on.

This is the environment in which enterprise IT leaders are operating for the foreseeable future. Hardware costs more. Hardware takes longer to arrive. And the old economics of traditional procurement, buying more capacity when you need more performance, no longer hold.

None of that changes the fact that compute, storage, and networking remain the foundation of every system. You cannot run workloads without them. But when that foundation costs more and takes longer to arrive, every layer of the stack that consumes hardware inefficiently becomes a liability.

Which raises the question every CIO should be asking: where in your architecture are you still consuming hardware for something software can do better?

The Case for Virtualization Has Never Been Stronger

Virtualization has been extending the life and efficiency of hardware for over two decades. What is new is the urgency. When DRAM prices surge nearly 95% in a single year and wafer supply is projected to stay constrained through the end of the decade, the cost of inefficiency at every layer of the stack compounds rapidly.

The question is no longer whether or not to virtualize. It is whether or not you have pushed virtualization far enough up the stack.

Most enterprises have virtualized compute. Many have virtualized storage. A growing number have virtualized networking at the infrastructure layer. But one layer consistently lags: application delivery.

Load balancers. Web application firewalls. API gateways. These functions still run on dedicated hardware appliances or on virtual appliances that behave like hardware and carry hardware-level cost, in the majority of enterprise environments. In a world where every rack unit needs to justify itself, that gap is no longer defensible.

What a Software-Defined Gateway Actually Delivers

A traditional application delivery architecture places purpose-built appliances at fixed points in the traffic path. One device handles external load balancing. Another manages internal routing. A third handles API authentication. A fourth inspects traffic for threats. Each is a capital expense, a configuration boundary, a failure domain, and a refresh cycle.

A software-defined gateway collapses all of those discrete points into a single programmable control plane. Whether running inside a VM on an existing virtualization platform or natively within a container environment, traffic is handled end-to-end by software: routing, TLS termination, authentication, rate limiting, circuit breaking, and observability. Policy is expressed as code. It is version-controlled, reproducible, and auditable. No firmware update or vendor support call required.

The performance argument for dedicated hardware has been closed for years. Modern software-defined gateways running on standard infrastructure handle enterprise traffic at scale with the observability and programmability that hardware appliances were never designed to provide.

The remaining argument for hardware appliances is inertia, not capability.

Portable vs Proprietary

Not all software-defined gateway solutions are equal, and the distinction matters more today than ever.

Some virtual appliances are software in name only. They run inside a specific hypervisor, require a particular vendor's control plane, or are licensed as part of a broader virtualization platform. These tools reduce hardware costs at the margin, but they substitute one form of lock-in for another. The procurement burden shifts; the architectural constraint does not.

The value of a truly software-defined gateway comes from its portability. A gateway that is environment-agnostic, hypervisor-agnostic, and location-agnostic runs identically whether deployed as a VM on an existing virtualization platform, as a workload inside a container environment, in a private data center, in a colocation facility, or in an air-gapped deployment with no external connectivity. Configuration is portable. Policy is portable. Observability is portable. The platform beneath you can change, and in 2026, platforms do change, without requiring you to re-architect the application delivery layer on top.

The right frame is not hardware versus software. It is portable versus proprietary.

Two Forcing Functions Arriving at the Same Time

The hardware economics argument is compelling on its own. But for much of the enterprise market, it is arriving alongside two platform-level disruptions that make the application delivery decision not just strategic but immediate.

VMware Migrations

The Broadcom acquisition of VMware has triggered one of the most consequential infrastructure re-evaluation cycles in a generation. Enterprises that built their virtualization strategy around vSphere and NSX are now reassessing licensing costs, platform dependencies, and long-term roadmaps. Many are moving workloads to alternative virtualization platforms. Some are accelerating their shift to containerized environments. All of them are asking the same question: which components of our stack do we carry forward, and which do we replace with something better?

VMware-bundled load balancing and gateway capabilities are, by definition, VMware-dependent. An enterprise migrating off VMware cannot lift those components and drop them into a new environment. It needs an application gateway that is platform-independent from the ground up. That is an architectural requirement, not a preference.

Ingress NGINX End-of-Life

For enterprises running containerized workloads, a parallel forcing function is already underway. Ingress NGINX, the most widely deployed Kubernetes ingress controller in production, announced end-of-life. This is not a minor dependency update, it is an architectural decision point.

The ingress layer is where external traffic enters, TLS terminates, routing decisions are made, and API governance begins. Replacing it isn’t a migration task, it means deciding what the next architecture looks like. Organizations that take the opportunity to rearchitect will come out with a gateway layer that is more capable, more portable, and substantially less expensive than what they are replacing.

Both forcing functions are active simultaneously. Each, on its own, is a sufficient reason to revisit the application delivery layer. Together, they represent a generational consolidation opportunity.

Migrate. Modernize. Transform.

Enterprises are currently in all three phases of infrastructure evolution simultaneously:

They are migrating: off VMware, off Ingress NGINX, off legacy appliances that cannot follow workloads to new environments.
They are modernizing: replacing hardware-dependent components with software-defined alternatives that are portable, programmable, and environment-agnostic.
They are transforming: deploying AI-powered applications, agent workflows, and model inference infrastructure that require governance capabilities hardware appliances were never designed to provide.

The risk in navigating all three individually is fragmentation. Each transition, approached in isolation, produces a new policy boundary, a new configuration silo, and a new governance gap. Audit risk compounds. Policy drift accumulates. Operational drag becomes structural.

The answer is a single software-defined gateway that follows workloads through migration, enforces consistent policy through modernization, and extends to govern AI traffic through transformation. One control plane. Consistent policy. No hardware in the critical path.

That control plane also serves as the enforcement point for AI governance: inspecting model requests before they reach an endpoint, enforcing authentication on agent calls, rate-limiting inference traffic by identity, and routing to different model versions based on policy. None of this requires a new appliance. It is a configuration change on the same gateway already managing the rest of your application traffic.

See It in Production

At KubeCon EU in Amsterdam next week, the Traefik Labs team will demonstrate exactly how this architecture operates in practice: software-defined ingress, API governance, and AI traffic management running on a single, environment-agnostic, hypervisor-agnostic, and location-agnostic control plane, with no hardware appliances in the critical path.

If you are working through a VMware migration, an ingress modernization, or the infrastructure layer for AI applications, we would welcome the conversation. Our team will be at Booth #981.

The hardware constraint is not going away before 2030. The enterprises that use this moment to unify their migration, modernization, and transformation onto a single portable gateway will carry a structural advantage that outlasts the current supply cycle.

]]>

OpenClaw is Having Its Enterprise Moment, But Application-Layer Governance with NemoClaw Isn't Enough

Sudeep Goswami — Wed, 18 Mar 2026 16:33:31 GMT

There is a specific kind of moment in enterprise technology that repeats itself with uncomfortable regularity. Something built for individuals gets handed to organizations, and nobody has figured out the governance story yet.

It happened with the cloud. The moment Amazon launched EC2, developers started spinning up servers on personal credit cards. It took years for the enterprise to catch up on IAM policies, cost governance, and audit trails. The capability arrived long before the controls did.

It happened again with SaaS. By the time IT leaders understood what Dropbox was doing on their networks, half the organization had already stored sensitive documents on personal accounts. Shadow IT emerged as a category because the tools moved faster than the institutions.

That moment is here again with AI agents. And the signal worth watching most closely right now is what just happened to OpenClaw.

From Developer Favorite to Enterprise Infrastructure

OpenClaw started as an open-source personal AI agent gateway, a tool that lets individual developers connect chat applications to AI models and local tools. It caught fire. 247k+ GitHub stars. 47k+ forks. The kind of organic adoption that does not happen because of marketing, it happens because engineers found something genuinely useful and started sharing it.

The creator, Peter Steinberger, joined OpenAI in early 2026. And then NVIDIA did something that tells you everything about where agentic AI is headed.

At GTC 2026, Jensen Huang unveiled NemoClaw, describing it as "30 years of NVIDIA computing, distilled into an agent platform." NemoClaw builds on what OpenClaw built for individuals and adds enterprise-grade architecture on top: multi-agent orchestration, hierarchical task delegation, signed skill registries, role-based access controls, audit logging, and deterministic logic for verifiable tool call sequences. It supports the Model Context Protocol natively, connects to the Agent-to-Agent protocol, runs on Nemotron models at 30B, 120B, and 500B parameter scale, and integrates with Salesforce, Cisco, Adobe, Google Cloud, and CrowdStrike out of the box.

This is not a library. Not a framework. It is the operating system of agentic computers — open-source under Apache 2.0, with a paid tier for managed infrastructure and compliance tooling.

This is the enterprise moment. And it changes what every CISO, Chief AI Officer, and platform architecture leader needs to be thinking about right now.

What NVIDIA Got Right

The governance capabilities built into NemoClaw are real and meaningful.

Enterprise role-based access controls on agent identities, so the finance agent and the customer support agent do not share credentials or capability sets. Signed skills, so the tools available to agents cannot be silently modified by a supply chain compromise. Step-level policy checks before every tool invocation, so an agent cannot take an action the platform has not explicitly authorized. Comprehensive audit logging at the orchestration layer.

These are not checkbox compliance features. They represent serious architectural thinking about what makes agentic AI deployable inside organizations with actual security programs.

NemoClaw's launch confirms what practitioners have been arguing for two years: enterprise agent governance is a mainstream requirement, not a niche concern. NVIDIA, with all its platform credibility and hardware distribution power, is now building it into a flagship product. That is a meaningful signal.

But the existence of application-layer governance does not imply complete governance. That distinction is where the real architectural question begins.

One Governance Layer Is Never Enough

The foundational principle of enterprise security is Defense-in-depth: no single control is trusted to stand alone. Every meaningful security architecture deploys independent enforcement at multiple layers, so that a failure, bypass, or compromise at one layer does not collapse the entire posture.

This principle predates AI agents by decades. It is why organizations run both endpoint detection and network monitoring. It is why zero-trust architectures do not simply trust identity at login and then grant full access. It is why the firewall analogy holds so cleanly: application firewalls understand business logic and enforce policy with context, but network firewalls enforce at a layer the application cannot reach. The answer has never been one or the other. It has always been both.

NemoClaw's governance is application-layer governance. Its guards operate inside the agent runtime, coupled with the agent itself. The access control policies, the signed skill checks, the deterministic logic engine, the audit logs, all of these run within the same process that is being governed. A compromise to the agent also compromises the guards that are governing it. This is not a design flaw. It is the nature and limitation of tightly coupled application-layer controls, and it is exactly what Defense-in-depth says is necessary but not sufficient.

Consider what happens when a NemoClaw agent makes an inference call to a Nemotron model. The request leaves the agent runtime and travels across the network to an endpoint. At that moment, one cannot be sure whether NemoClaw's governance allowed it because it's a legitimate request or because its governance was also compromised along with the agent. Its governance might allow the payload to contain PII from a customer database. It might be a prompt injection payload embedded in a tool response. Also, application-layer controls cannot enforce a token budget at the network layer before GPU cycles are consumed. It cannot verify that the specific parameters in an MCP tool call, not just the tool class, but the actual values, conform to the organization's security policy.

This is not a gap unique to NemoClaw. It is inherent to any system that governs from the inside out. And it is precisely the gap that the defense-in-depth principle exists to close.

The Triple Gate: Defense-in-Depth Made Concrete

Applying Defense-in-depth to NemoClaw deployments means identifying the distinct traffic vectors generated by agents and placing independent enforcement at each.

NemoClaw agents generate three kinds. Every inference call to a language model. Every MCP tool invocation. Every request to the agent management and orchestration APIs. Three vectors, each carrying different risks, each requiring its own enforcement point.

This is the architecture Traefik Hub's Triple Gate Pattern addresses directly.

The AI Gateway sits in front of every inference call a NemoClaw agent makes, to Nemotron, to OpenAI, to Anthropic, to any model endpoint. It runs a four-tier safety pipeline: pattern-matching for injection signatures and PII formats, real-time PII redaction, NVIDIA Safety NIMs for topic control and jailbreak detection, and response-side hallucination scoring. This pipeline runs at the traffic layer, independently of NemoClaw's internal guardrails. If the agent runtime is compromised, the AI Gateway continues to run. It also enforces token rate limits per agent, per team, and per API key, a hard network-layer stop that operates before GPU resources are allocated.

The MCP Gateway sits in front of every tool invocation. NemoClaw's role-based controls determine which tools an agent role is permitted to use. The MCP Gateway enforces at the parameter level, the specific values being passed, the specific data being accessed, the specific operations being requested, against infrastructure-layer policy that exists independently of the agent runtime. This is the enforcement layer that catches tool-call escalations even when the tool call itself is legitimately authorized. Refusals are returned as structured responses that NemoClaw agents handle gracefully, without crashing workflows.

The API Gateway governs the NemoClaw management plane itself, the agent configuration endpoints, monitoring interfaces, and orchestration APIs that constitute the control surface of any NemoClaw deployment. These endpoints are attack surfaces in their own right. The API Gateway enforces authentication, rate limiting, schema validation, and WAF rules against the OWASP threat model, independent of anything running inside the NemoClaw runtime.

Three gateways. Three enforcement points. Each layer is independent. Each layer covers attack surfaces that the others do not reach. This is Defense-in-depth applied to the specific topology of enterprise agent deployments.

Four Scenarios That Illustrate the Need for Defense-in-Depth

Defense-in-depth only has value if it addresses real attack vectors. These four scenarios are representative of what enterprise deployments will face.

1. Prompt Injection via Tool Response

An agent calls an external tool and receives a response. Embedded in that response is a carefully crafted instruction designed to redirect the agent's next action. NemoClaw processes this at the application layer after it has already been ingested by the runtime. The AI Gateway intercepts the tool response before the agent sees it, running it through the safety pipeline at the traffic layer. The injection never reaches the runtime.

2. Token Drain via Recursive Calls

A misbehaving agent triggers recursive inference loops. NemoClaw's resource budgets are soft limits enforced inside the runtime. If the runtime is in a problematic state, those budgets may not hold. The AI Gateway's token rate limiting is a hard stop at the network layer, applied before inference requests reach the GPU cluster. No compute is consumed, regardless of the runtime's state.

3. MCP Tool Escalation via Parameter Crafting

An agent constructs a tool call in which the tool itself is authorized under its role, but the parameters are crafted to exceed the intended scope. NemoClaw's role-based controls authorize the tool class. The MCP Gateway validates the specific parameters against infrastructure-layer policy at the network layer, independently of the runtime. The escalation attempt fails at the enforcement point closest to the tool.

4. PII Leakage in Inference Responses

A model response contains customer personally identifiable information that was present in the context window but was not intentionally requested. NemoClaw logs the interaction after the fact. The AI Gateway operates bidirectionally on the traffic itself, redacting PII in both requests and responses before they move further in the chain. Whether the agent requested the data or received it incidentally, anything that crosses the network gets inspected and sanitized before it travels further.

In each scenario, the enforcement point that stops the incident is independent of the agent runtime. That independence is exactly what Defense-in-depth requires.

The Architecture Choice That Only Gets Made Once

When enterprise leaders evaluate NemoClaw deployments, the real decision is not which agent platform to use. NemoClaw is genuinely compelling and will see serious adoption. The decision is whether to deploy it with only application-layer governance or with the full defense-in-depth posture demanded by the threat landscape.

That choice compounds over time. Teams that build governed agent infrastructure now, with the agent runtime owning application-layer governance and the Triple Gate owning traffic governance, will move faster as the agent landscape expands. Their security posture does not depend on any single component being uncompromised. Their compliance story holds up to auditors who understand Defense-in-depth. Their cost controls operate at the network layer, where they cannot be bypassed by a misbehaving agent.

Teams that deploy with only application-layer governance will eventually encounter an incident that exposes the gap. Retrofitting infrastructure controls into a production system under pressure is the most expensive way to build them.

What This Means for Platform Decisions

For security leaders evaluating agent deployments, the question to put to architecture teams is direct: does our governance posture reflect Defense-in-depth, or does it rely on a single enforcement layer inside the agent runtime? If the runtime is degraded or compromised, what remains?

For Chief AI Officers enabling agentic workflows at scale, the infrastructure layer is what enables scalable governance without becoming a bottleneck. A policy enforced at the network layer applies to every agent, regardless of which framework built it, which runtime runs it, or which vendor's model it calls. That universality is the difference between governance that scales and governance that requires a new exception for every new deployment.

For platform architecture leaders, the Triple Gate model maps cleanly to a separation of concerns that infrastructure teams already understand. The agent platform team owns the runtime governance. The platform infrastructure team owns the traffic governance. Neither depends on the other to do its job, and a compromise of one does not take down both.

The Broader Signal

NVIDIA building enterprise governance into NemoClaw confirms that the demand for governed agents is now mainstream enough to justify a platform-level investment from one of the most capital-efficient organizations in technology history. Jensen Huang does not build products for markets that do not exist.

What NemoClaw does not include, the API gateway in front of agent endpoints, the safety pipeline on inference traffic, the infrastructure-level MCP authorization, the token rate limiting at the network layer, is not an oversight. It is a scope decision. NVIDIA's business is silicon and model infrastructure. Agent governance at the traffic layer is a different problem, one that requires deep integration with Kubernetes ingress, API gateway protocols, and network-layer traffic management.

That scope decision is not a gap NVIDIA needs to fill. It is the natural boundary where the agent runtime ends and the infrastructure governance layer begins — the boundary where defense-in-depth becomes operational.

The organizations that understand this, that see NemoClaw's application-layer governance and Traefik's Triple Gate as two halves of a complete defense-in-depth posture, are the ones that will deploy AI agents at enterprise scale without discovering their assumptions were wrong after the fact.

OpenClaw had its enterprise moment at GTC 2026. Defense-in-depth is what turns that moment into a durable foundation.

To see how Traefik Hub's Triple Gate architecture provides infrastructure-level governance for NemoClaw deployments, see this blog post.

]]>

The McKinsey Breach Was SQL Injection. The Real Threat Was 95 Writable System Prompts.

Zaid Albirawi, Immánuel Fodor — Tue, 17 Mar 2026 17:47:53 GMT

An autonomous AI agent breached one of the world's most prestigious consulting firms in two hours. The vulnerability was 20 years old. The implications are brand new.

The Attack Chain

On March 9, 2026, security startup CodeWall disclosed that its autonomous offensive AI agent had achieved full read-write access to McKinsey's internal AI platform, Lilli, in under two hours. The agent operated with no credentials, no insider access, and no human in the loop.

The attack chain was startlingly simple. Of Lilli's 200+ API endpoints, 22 required no authentication whatsoever. The agent discovered that while user input values were properly parameterized in SQL queries (the standard defense against injection), the JSON field names were concatenated directly into SQL without sanitization. When database error messages began reflecting live production data, the agent recognized a classic error-based SQL injection vector and began extracting data iteratively.

The result: access to 46.5 million plaintext chat messages covering strategy, M&A, and client engagements. 728,000 files, including PDFs, Excel spreadsheets, and PowerPoint decks. 57,000 user accounts. 266,000+ OpenAI vector stores. 3.68 million RAG document chunks. And 95 system prompts across 12 model types.

All writable.

McKinsey patched the vulnerabilities within hours of disclosure, engaged a third-party forensics firm, and stated that no evidence of unauthorized client data access was found. CodeWall operated under McKinsey's public HackerOne responsible disclosure program. The response was swift and professional.

But the structural lessons are too important to let pass quietly.

This Is Not a Data Breach Story

Every headline focused on the data: 46.5 million messages, 728,000 files, the sheer volume of sensitive information exposed. Those numbers are staggering, and they deserve attention. But they are not the most important part of this story.

The most important detail is that 95 system prompts were writable.

Lilli is used by over 40,000 McKinsey consultants, and processes more than 500,000 prompts per month. Consultants rely on it for strategy research, competitive analysis, M&A evaluation, and client recommendations. The system prompts define how Lilli thinks: what it recommends, what it refuses, how it cites sources, and what guardrails it follows.

As CodeWall put it in their report: "No deployment needed. No code change. Just a single UPDATE statement wrapped in a single HTTP call."

A threat actor with write access to those prompts could have silently rewritten how Lilli frames competitive landscapes, evaluates acquisition targets, or assesses risk. The poisoned output would flow directly into deliverables for Fortune 500 clients. No one receiving the advice would know it had been tampered with.

This is not a data breach. It is a supply chain attack vector for corporate decision-making itself.

And it gets worse. The same vulnerability exposed 266,000+ OpenAI vector stores and 3.68 million RAG document chunks. These are the knowledge bases that Lilli retrieves from and synthesizes when answering questions. An attacker with write access to both system prompts and RAG stores could manipulate not just how the AI reasons, but also the source material it draws from. The poisoning would be nearly undetectable.

The Root Cause Is Architectural

It is tempting to reduce this breach to a single attack vector: SQL injection, one of the oldest vulnerability classes in web security. And yes, it is remarkable that a production AI platform serving 40,000 users in 2026 shipped with an OWASP Top 10 vulnerability. But blaming the developers misses the point.

McKinsey's team did the standard thing. They parameterized user input values in their SQL queries. They followed the textbook. What they missed was that JSON field names were also being concatenated into SQL, an unusual injection vector that standard scanners like OWASP ZAP do not typically test for.

The real failure is not that a developer missed an edge case. The real failure is that the architecture had no independent layers of defense between the internet and the production database. No request inspection at the gateway. No authentication on 22 endpoints. No content safety checks on the AI pipeline. No access governance for agents or automated callers.

The application was the only line of defense. When it failed, everything behind it was exposed.

This is the architectural lesson: application-level security is necessary but never sufficient. You cannot rely on every developer getting every edge case right across every endpoint in every release. You need independent enforcement at the infrastructure layer, inspecting and governing traffic before it ever reaches application code.

Defense in Depth for the AI Era: The Triple Gate Pattern

At Traefik Labs, we have been building toward this exact threat model. The Triple Gate Pattern is an architecture for defense in depth across the full AI execution path: from HTTP requests to LLM interactions to agent tool calls. Three independent gates, each operating on different principles, each enforcing policy at the infrastructure layer, independent of the application runtime.

Diagram: The Triple Gate Pattern

Let's walk through the McKinsey attack chain and show how each gate addresses a specific class of failure.

Gate 1: API Gateway—Authentication, WAF, and Runtime API Governance

The first gate governs all HTTP traffic entering your infrastructure. It enforces authentication, authorization, rate limiting, schema validation, and web application firewall (WAF) protection.

This is where the McKinsey breach would have been stopped cold.

Traefik Hub's API Gateway natively integrates the Coraza WAF with OWASP Core Rule Set (CRS) support. The CRS is a community-maintained, battle-tested collection of attack detection rules that has closed over 500 rule bypasses through its own bug bounty program. SQL injection detection is one of its most mature capabilities.

CRS inspects the entire HTTP request payload for SQL injection patterns. It does not care whether the injection point is in a query parameter, a JSON value, or a JSON key. It scans the full request. McKinsey's developers missed the JSON key vector because they were thinking about parameterization at the application layer. A WAF at the gateway layer does not make that distinction. It would have flagged the injection pattern and blocked the request before a single byte reached the application.

And this is not a theoretical claim. SQL injection accounts for roughly 65% of all web application attacks. It is the single most common attack class on the internet. The OWASP CRS exists precisely because you cannot trust application code alone to catch every variant.

But WAF is only one capability within Gate 1. Even if an exotic injection variant somehow bypassed the WAF rules, the API Gateway enforces authentication and authorization on every endpoint. McKinsey's 22 unauthenticated endpoints would never have existed in this architecture. No valid identity token, no access. Zero trust at the API layer means every request is scoped, every call is auditable, and every endpoint is protected regardless of what the application code does or does not validate.

Traefik Hub's native Coraza integration runs at high performance because it is compiled directly into the gateway binary. No sidecar container. No separate appliance. No additional network hop. For organizations currently running ModSecurity as a separate layer in front of their ingress controller, this is a consolidation story: the WAF, the API gateway, and the ingress controller converge into a single platform with a single control plane.

Gate 1 alone would have prevented the McKinsey breach entirely. But defense in depth means you never rely on a single gate.

Gate 2: AI Gateway—Content Safety, Guardrails, and Cost Controls

The second gate governs the AI-specific traffic: LLM prompts, model responses, and the content flowing through the inference pipeline.

Even if an attacker bypasses Gate 1 and reaches the AI layer, Gate 2 provides independent enforcement. The AI Gateway sits between callers and the LLM backend, inspecting both inbound prompts and outbound responses against configurable safety policies.

In the McKinsey scenario, the most dangerous outcome was not data exfiltration. It was the ability to rewrite system prompts. A poisoned system prompt would change how Lilli responds to every query from every user. But if an AI Gateway with content-safety guardrails is inspecting every response, the poisoned outputs would need to survive those checks on every single inference call. A prompt rewritten to inject biased analysis, suppress certain topics, or leak confidential data would trigger content safety, topic control, or jailbreak detection guardrails on the response path.

With Traefik Hub v3.20, announced yesterday at NVIDIA GTC, the AI Gateway introduces a composable, multi-vendor safety pipeline with parallel guard execution. This means organizations can chain multiple high-latency guardrail providers (NVIDIA Safety NIMs, IBM Granite Guardian) and execute them in parallel rather than sequentially. The architectural insight is that safety layers must be composable and multi-vendor. No single guardrail provider catches everything. Parallel execution means you get multi-vendor coverage without the latency penalty.

Token-level cost controls, also new in v3.20, address another dimension of the McKinsey exposure. An attacker with write access to system prompts could inflate token usage massively: longer prompts, more verbose responses, and chain-of-thought reasoning injected into every query. At 500,000+ prompts per month, this becomes a denial-of-wallet attack. Infrastructure-layer token rate limiting and quota management prevent this class of abuse before requests reach the model.

Multi-provider failover routing adds operational resilience. McKinsey was deeply coupled to OpenAI (evidenced by the 266,000+ vector stores). If your primary model provider's integration is compromised, multi-provider failover lets you route traffic to a secondary provider while the primary is under investigation.

Gate 2 ensures that even if the perimeter is breached, the AI pipeline itself has independent safety enforcement that an attacker cannot bypass by modifying application-layer configuration.

Gate 3: MCP Gateway—Agent Governance and Access Control

The third gate governs what AI agents can do: which tools they can invoke, which tasks they are authorized for, which data they can read or write, and which operations they can perform.

This is the gate that addresses the most forward-looking dimension of the McKinsey breach. CodeWall's agent autonomously selected McKinsey as a target, mapped its attack surface, and executed a multi-step attack chain at machine speed. This is precisely the kind of autonomous agent behavior that enterprises are now deploying internally for legitimate purposes and that threat actors will weaponize externally.

The MCP Gateway enforces Task-Based Access Control (TBAC), an authorization model designed specifically for AI agents. Traditional API security (RBAC, OAuth scopes) was not built for autonomous callers that probe, adapt, and chain actions iteratively. TBAC scopes permissions to the actual work being done: which business tasks the agent is authorized to perform, which MCP tools it can access, and which exact operations and data resources it can touch.

In the McKinsey context, even if an autonomous agent somehow bypassed Gate 1 (WAF + API authentication) and Gate 2 (AI content safety), Gate 3 would constrain what the agent could actually do with any access it obtained. A TBAC policy can enforce that no caller, human or autonomous, has write access to system prompt tables through the MCP layer. Tool invocations are scoped, transactions are auditable, and permissions are enforced at the infrastructure layer, below the application runtime.

This last point is critical. Agent platforms are increasingly adding application-level governance: RBAC, audit logging, and signed skills. These controls are valuable, but they operate inside the agent runtime. If the runtime is compromised, as McKinsey's effectively was, so are the guardrails. Infrastructure-layer enforcement through the MCP Gateway operates independently of the agent platform, so even a compromised runtime cannot override the access policies.

Traefik Hub v3.20 also introduces graceful error handling for agent-aware enforcement. When a guardrail blocks a request, traditional gateways return an HTTP 403 response, breaking agent control flow and crashing multi-step workflows. Traefik Hub can now return structured, schema-compliant refusal responses that agents and applications process gracefully. The agent continues operating within its authorized scope. Middleware chains stay intact. Users see conversational refusals instead of technical errors. This is what makes the runtime governance agent-aware: enforcement that works with autonomous workflows rather than breaking them.

The Uncomfortable Question

McKinsey will strengthen its security posture. They responded quickly, they engaged forensics, and they patched within hours. That deserves acknowledgement.

The uncomfortable question is not about McKinsey. It is about the thousands of enterprises that are deploying internal AI platforms right now with the same architectural gaps. No WAF on the AI endpoints. No authentication on development or staging APIs that quietly became production. No content safety guardrails independent of the model provider. No agent governance at the infrastructure layer.

Gartner estimates that 40% of enterprise applications will incorporate AI agents by the end of 2026, up from less than 5% in 2025. The MCP ecosystem has grown to over 10,000 published servers. NVIDIA just unveiled NemoClaw at GTC 2026, bringing enterprise-grade agent orchestration to the NVIDIA stack.

The agents are coming. The attack surface is expanding at machine speed. And the defenses, in most organizations, are still designed for a world where humans typed queries into web forms and waited for responses.

Defense in depth is not a new concept. But the AI era demands a new implementation: one that governs not just HTTP traffic, but LLM content, model interactions, and autonomous agent behavior, all at the infrastructure layer, all enforced independently of the application code that will inevitably have bugs.

That is what the Triple Gate Pattern is for. And it is what we built Traefik Hub to deliver. Organizations already running Traefik Proxy for ingress can add the full API Gateway (with native WAF), AI Gateway, and MCP Gateway capabilities through a single in-place upgrade. No re-architecture. No traffic migration. No additional proxies in the data path. Three gates, one platform, one control plane.

The McKinsey breach is the proof point that this architecture is no longer optional.

Traefik Hub v3.20, including the composable safety pipeline, multi-vendor guardrails with parallel execution, token-level cost controls, graceful agent-aware error handling, IBM Granite Guardian integration, and custom Regex Guards. Sign up for Early Access or read the technical deep dive: From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

For a detailed overview of the Triple Gate Pattern and Traefik Hub's AI and MCP Gateway capabilities, visit traefik.io/solutions/ai-gateway.

]]>

From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline with NVIDIA, IBM, Microsoft, and Custom Pattern Matching

Immánuel Fodor, Zaid Albirawi — Mon, 16 Mar 2026 15:19:57 GMT

AI runtime safety has always been about layers. No single guard catches everything. Deterministic pattern matching is fast, but it can't understand context. NLP-based entity detection catches structured PII but misses semantic threats. GPU-accelerated models understand nuance but add latency and cost. The right approach has always been defense-in-depth: multiple guards, different detection methods, complementary coverage.

The problem is that defense-in-depth has historically meant stacking latency. Three guards running in series can add several seconds of enforcement overhead per request. In production, that's a non-starter. Organizations end up choosing between thorough protection and acceptable performance, deploying one or two guards instead of the four or five they'd actually want.

Traefik Hub v3.20 changes this. The AI Gateway now supports a composable, multi-vendor safety pipeline, where organizations can choose from multiple guardrail providers, combine them based on their requirements, and run them all in parallel. Total enforcement time equals the slowest guard, not the sum. And when a guard blocks a request, the response is structured so agents and middleware chains can handle it gracefully instead of crashing.

This post walks through the architecture, configuration, and practical considerations for building a multi-vendor safety pipeline with Traefik Hub.

The Pipeline: Four Tiers from Deterministic Speed to Semantic Intelligence

The composable safety pipeline spans four tiers. Each tier serves a different purpose, uses a different detection approach, and has different performance and infrastructure characteristics. Organizations deploy the tiers they need based on their threat model, latency budget, and infrastructure.

Tier 1: Regex Guard (Custom Pattern Matching)

Regex Guard is a framework for organizations to write their own content guards. It's not a pre-built guard with fixed rules. Teams define the patterns they want to catch, the fields they want to scan, and whether to block or mask matched content.

Here's how it works: the Regex Guard implements a rule-based pattern matching engine integrated into the Content Guard middleware. Each rule specifies one or more JSONQuery expressions that locate target fields in structured payloads, one or more regex patterns to match against, and an action (block or mask) to take on match.

Here's a practical configuration that blocks prompt injection patterns in chat messages while masking credit card numbers in customer data:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: regex-guard
  namespace: apps
spec:
  plugin:
    content-guard:
      engine:
        regex: {}

      request:
        # Block prompt injection attempts in chat messages
          - jsonQueries:
              - ".messages[].content"
              - ".prompt"
            block: true
            reason: "Potential prompt injection detected"
            entities:
              - "(?i)ignore.*(previous|above|prior).*(instructions|prompt)"
              - "(?i)you are now.*(new|different|unrestricted)"

        # Mask credit card numbers in customer data
        - jsonQueries:
            - ".customer.payment"
            - ".billing.card_number"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}"

        # Mask US Social Security numbers
        - jsonQueries:
            - ".user.ssn"
            - ".applicant.social_security"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{3}-\\d{2}-\\d{4}"

      response:
        # Mask internal IP addresses in error responses
        - jsonQueries:
            - ".error_detail"
            - ".debug_info"
          mask:
            char: "X"
            unmaskFromLeft: 0
            unmaskFromRight: 0
          entities:
            - "192\\.168\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "10\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}"

A few things to note about the configuration:

JSONQuery field targeting lets you scope rules to specific fields rather than scanning entire payloads. .messages[].content targets the content field in a chat completion request. .customer.payment targets a specific nested field.

Block vs. mask serves different purposes. Blocking rejects the request entirely. Masking redacts the matched content in-place and lets the request continue. Use blocking for threats (prompt injection, system prompt leakage). Use masking for PII (credit cards, SSNs, phone numbers) where you want the request to proceed with sensitive data redacted.

Bidirectional rules apply to both requests and responses independently. Request-side rules catch threats before they reach the LLM. Response-side rules prevent information leakage (internal IPs, database connection strings, infrastructure identifiers) in the LLM's output.

Performance: Regex Guard runs entirely in-process with zero external dependencies. Latency is sub-millisecond for typical payloads. There are no network round-trips, no GPU requirements, and no probabilistic variance. The same input always produces the same output.

The cost argument is straightforward. Catching a credit card number or Social Security number doesn't require semantic understanding. These are well-defined patterns with well-known formats. Running them through a GPU-accelerated AI model adds latency and cost with no improvement in detection accuracy for these patterns. Regex Guard handles them at a fraction of the cost and an order of magnitude faster.

Tier 2: Content Guard (Microsoft Presidio)

Content Guard provides PII detection and masking powered by Microsoft's Presidio analyzer. Where Regex Guard relies on patterns you define, Presidio uses statistical NLP-based entity recognition with a library of built-in entity types: email addresses, phone numbers, medical record IDs, credit card numbers, and more.

Presidio requires an external analyzer instance. You deploy the Presidio service and configure Content Guard to point at it. The middleware sends the content to Presidio for analysis, receives entity detection results, and applies blocking or masking based on the matches.

Content Guard supports custom entity patterns through Presidio's custom analyzer endpoints, allowing organizations to extend beyond the built-in entity types for formats specific to their business.

The tradeoff vs. Regex Guard: Presidio's NLP-based detection can catch entities that don't follow strict patterns (misspelled names, partial phone numbers, context-dependent PII). But it requires an external service, adds network latency, and introduces a dependency. For well-defined patterns, Regex Guard is faster. For fuzzy or context-dependent PII, Presidio is more thorough. In a composable pipeline, you can run both.

Tier 3: LLM Guard with NVIDIA NIMs

NVIDIA NIM (NVIDIA Inference Microservices) integration provides GPU-accelerated content safety through three specialized microservices:

Topic Control NIM enforces conversation boundaries. It takes guideline-based prompts that define allowed and prohibited topics, and blocks requests that fall outside the defined scope.

Content Safety NIM detects harmful content across 22+ safety categories, including violence, hate speech, and privacy violations. Analyzes both user requests and AI responses in real-time.

Jailbreak Detection NIM identifies attempts to bypass AI system restrictions through prompt manipulation.

Each NIM requires a dedicated GPU with 24GB+ memory. Minimum processing latency per NIM ranges from 30ms (jailbreak detection) to 200ms (content safety). When running all three sequentially, total minimum latency is 180-350ms. With parallel execution, total latency equals only the slowest NIM.

This is where GPU-accelerated semantic intelligence earns its latency cost. NVIDIA NIMs catch threats that pattern matching and NLP-based detection cannot: nuanced prompt injection attempts that don't use obvious keywords, subtle harmful content that requires contextual understanding, and off-topic drift that a regex rule would need hundreds of patterns to approximate.

Tier 4: LLM Guard with IBM Granite Guardian

IBM's Granite Guardian 3.3 (8B parameter) is an open-source safety model that provides capabilities other guard providers don't yet offer: hallucination detection and RAG quality assessment.

Granite Guardian uses a prompt-template approach. Each safety task (harm, jailbreak, topic control, hallucination) is triggered by a specific system prompt. The model responds with a structured <score> yes </score> or <score> no </score> evaluation, which the LLM Guard middleware evaluates with block conditions.

Here's the hallucination detection configuration, which is response-only and uses request history for context:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: granite-hallucination-detection
  namespace: apps
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      response:
        systemPrompt: "hallucination"
        useRequestHistory: true
        blockConditions:
          - reason: hallucination_detected
            condition: Contains("yes")

And here's harm detection, which scans both requests and responses:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: granite-harm-detection
  namespace: apps
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      request:
        systemPrompt: "harm"
        blockConditions:
          - reason: harmful_content_detected
            condition: Contains("yes")
      response:
        systemPrompt: "harm"
        useRequestHistory: true
        blockConditions:
          - reason: harmful_response
            condition: Contains("yes")

Granite Guardian is deployed via vLLM and requires one GPU with approximately 16GB memory (FP16) or 8GB (8-bit quantization). The temperature: 0 parameter is required for reliable classification.

The unique value of Granite Guardian in the composable pipeline is hallucination detection and RAG quality assessment. NVIDIA NIMs don't yet offer hallucination detection. Presidio and Regex Guard are structurally unable to detect hallucinations (they match patterns, not factual accuracy). If your application uses retrieval-augmented generation and you need to verify that the model's response is grounded in the retrieved context, Granite Guardian is currently the only guard tier in the pipeline that addresses this.

Parallel Guard Execution: Defense-in-Depth Without the Latency Tax

With four tiers available, the question becomes: can you run them all without making every request unacceptably slow?

The answer is parallel guard execution. Instead of running guards in series (where total latency equals the sum), Traefik Hub runs all guards simultaneously. Total enforcement time equals only the slowest guard.

The parallel-llm-guard middleware orchestrates this. It spawns all configured guards as concurrent Go routines, collects results as they complete, and makes a pass/fail decision. Every guard blocks equally; there is no critical/optional distinction. If any guard blocks, the request is rejected with a 403, and all other guards are immediately cancelled. If any guard errors, the request fails with a 500, and all others are cancelled.

Each guard in the guards array wraps one of four guard types:

chat-completion-llm-guard
chat-completion-llm-guard-custom
llm-guard
llm-guard-custom

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: parallel-safety-pipeline
  namespace: ai-services
spec:
  plugin:
    parallel-llm-guard:
      guards:
        - chat-completion-llm-guard:
            endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
            model: ibm-granite/granite-guardian-3.3-8b
            clientConfig:
              timeoutSeconds: 5
            params:
              temperature: 0
              maxTokens: 50
            request:
              systemPrompt: "harm"
              blockConditions:
                - reason: harmful_content
                  condition: Contains("yes")

        - chat-completion-llm-guard:
            endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
            model: ibm-granite/granite-guardian-3.3-8b
            clientConfig:
              timeoutSeconds: 5
            params:
              temperature: 0
              maxTokens: 50
            request:
              systemPrompt: "jailbreak"
              blockConditions:
                - reason: jailbreak_attempt
                  condition: Contains("yes")

        - llm-guard:
            endpoint: http://nvidia-content-safety.apps.svc.cluster.local:8000
            clientConfig:
              timeoutSeconds: 5
            request:
              blockConditions:
                - reason: unsafe_content
                  condition: Contains("UNSAFE")

In this configuration, all guards run concurrently. Total enforcement time equals the slowest guard, not the sum. If one guard completes and blocks the request, all other guards are immediately cancelled, as there is no need to spend tokens (and GPU cycles) on finishing processing guard executions on a request that's already been rejected by a faster check.

Agent-Aware Enforcement: Graceful Error Handling

A composable safety pipeline that catches threats is only half the story. The other half is what happens when a guard blocks a request.

Traditional gateways return an HTTP 403 Forbidden with a plain-text error message. This works for simple request-response APIs. It doesn't work for autonomous agents or agentic workflows.

When an agent executes a multi-step workflow and one step triggers a guardrail, an HTTP 403 breaks the agent's control flow. The agent can't distinguish between "the guardrail blocked this specific request" and "something is fundamentally wrong with the system." Most agent frameworks treat non-2xx responses as errors that require human intervention or task failure. A single guardrail block can crash an entire multi-step workflow.

Traefik Hub v3.20 introduces the onDenyResponse feature. When configured, guardrail blocks return a custom HTTP status code and message instead of the default 403. This is configured per block condition, so different denial reasons can return different responses:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: llm-guard-with-graceful-errors
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      request:
        systemPrompt: "harm"
        blockConditions:
          - reason: harmful_content
            condition: Contains("yes")
            onDenyResponse:
              statusCode: 200
              message: 'This request was blocked by security policies. Please reformulate your request.'

An agent receiving an HTTP 200 with a refusal message can parse it as a valid turn in the conversation, understand that the request was refused (not that the system failed), and decide what to do next: retry with modified input, skip this step, or escalate to a human.

This also keeps middleware chains intact. Without onDenyResponse, a 403 from an upstream guardrail could potentially break downstream custom plugins ( eg., custom response handlers, loggers) that expect structured JSON and were not created with graceful error handling in mind. With it, the entire chain processes normally.

Existing deployments continue returning HTTP 403 by default. Each block condition can independently define its own onDenyResponse with a custom status code and message template.

Operational Controls: Resilience and Cost

A safety pipeline, even one that's composable and parallel, isn't sufficient for production on its own. Organizations also need their AI to stay up when providers go down and stay within budget. Traefik Hub v3.20 adds both.

Failover Router

The Failover Router uses nested TraefikService resources to build a failover chain across LLM providers. When a primary backend responds with a configured error status code (e.g., 429 rate limit, 500-504 server errors), the request is automatically replayed to the fallback service. Each backend gets its own chat-completion middleware for independent model selection, API keys, and metrics.

# Primary backend: OpenAI GPT-4o
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: openai-primary
  namespace: ai-services
spec:
  failover:
    service:
      name: openai-gpt4o
      middlewares:
        - name: openai-chat-completion
    fallback:
      name: anthropic-fallback
    errors:
      status:
        - "429"
        - "500-504"
---
# Secondary backend: Anthropic Claude
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: anthropic-fallback
  namespace: ai-services
spec:
  failover:
    service:
      name: anthropic-claude
      middlewares:
        - name: anthropic-chat-completion
    fallback:
      name: openai-mini-budget
    errors:
      status:
        - "429"
        - "500-504"

Each chat-completion middleware configures the provider-specific details (model name, API key secret, parameters) independently. This means each backend in the failover chain can use a different provider, model, and set of credentials. The failover mechanism is status-code-based: when the primary responds with a status matching errors.status, the request is replayed to the fallback. This enables cost-optimized degradation: when the premium model is rate-limited or down, fall back to a budget model from a different provider.

Because the Failover Router operates within the Triple Gate architecture, all safety pipeline guards and cost controls continue to apply regardless of which provider is serving the request. The governance doesn't degrade when the model does.

Token Rate Limiting and Quota Management

LLM providers charge by token, not by request. A single request with a 100K-token prompt costs orders of magnitude more than one with a 100-token prompt. Request-count-based rate limiting doesn't capture this.

Token governance tracks input, output, and total tokens independently. It operates in two modes:

Rate limiting handles traffic spikes with short windows (hourly, daily). Prevents burst overages without hard-capping long-term consumption.

Quota enforcement sets hard budget caps over longer periods (monthly). When the quota is reached, requests are rejected entirely.

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ai-token-rate-limit
spec:
  plugin:
    ai-rate-limit:  # or ai-quota
      store:
        redis:
          endpoints:
            - redis.default.svc.cluster.local:6379
      inputTokenLimit:
        limit: 5000
        period: 1h
        jsonQuery: ".usage.prompt_tokens"
      outputTokenLimit:
        limit: 10000
        period: 1h
        jsonQuery: ".usage.completion_tokens"
      totalTokenLimit:
        limit: 15000
        period: 1h
        jsonQuery: ".usage.total_tokens"
      sourceCriterion:
        requestHeaderName: "X-User-ID"
      estimateStrategy:
        simple: {}

The sourceCriterion determines how requests are grouped for rate limiting. It supports three options: requestHeaderName (a simple string identifying which header to use, e.g., "X-User-ID"), ipStrategy (with depth, excludedIPs, and ipv6Subnet for IP-based tracking), and requestHost (a boolean for host-based tracking). For JWT-based per-user limiting, the recommended pattern is to deploy a separate JWT middleware that extracts claims into headers via forwardHeaders, then reference that header with requestHeaderName. Composability enables many use cases, such as group/team-based buckets (using a Group header with the group claim extracted) or department/location-based bucketing.

The key differentiator is proactive token estimation. Most solutions enforce limits reactively: they let the request through, wait for the LLM response, read the token count, and then enforce limits on the next request. By that point, the cost has already been incurred.

Traefik Hub's token rate limiting estimates input tokens before the request reaches the LLM. If the estimated usage would exceed remaining capacity, the request is blocked immediately, before consuming any LLM resources or incurring any cost.

Token counts are stored in a distributed Redis backend, ensuring accurate tracking across multiple gateway instances.

The Full AI Workflow: End-to-End

Here's how all of these pieces can work together in a single request:

Request arrives at the API Gateway (Gate 1). JWT is validated, identity is extracted, and rate limit headers are checked.
Token estimation runs. If the estimated input tokens would exceed the user's remaining rate limit or quota, the request is blocked before anything else runs. No guard processing, no LLM invocation, no cost incurred.
Deterministic guards execute (Gate 2, first pass). Regex Guard scans for pattern-based threats (prompt injection, credential leaks) and masks PII with known formats (SSNs, credit cards). Content Guard (Presidio) performs NLP-based entity detection for fuzzy PII that regex patterns might miss. These are fast (sub-millisecond for regex, ~200ms for Presidio) and block early before heavier guards run.
Parallel LLM guard pipeline executes (Gate 2, second pass). NVIDIA NIMs and IBM Granite Guardian run simultaneously for semantic threat detection. If any guard blocks, all others are cancelled, and a structured refusal is returned (if onDenyResponse is configured) or an HTTP 403 (if not).
Request reaches the LLM via the Failover Router. If the primary provider responds with a configured error status (429, 500-504), the request is replayed to the fallback service. All safety policies continue to apply.
Response passes back through the guards. Response-side rules in Regex Guard check for internal IP addresses and infrastructure identifiers. Granite Guardian checks for hallucinations using the request history as context.
Actual token counts are captured from the LLM response and stored in Redis, updating the user's rate limit and quota tracking.
If the LLM triggers an agent action, the request hits the MCP Gateway (Gate 3). Tool-Based Access Control (TBAC) enforces what the agent is allowed to do: which tasks, which tools, which parameters. The same JWT identity that governed token budgets in Gate 2 now governs agent permissions in Gate 3.

Three gates, one identity context, and a composable safety pipeline that runs in parallel across the entire AI workflow.

Choosing Your Guard Tiers: A Decision Framework

Not every deployment needs all four tiers. Here's a practical framework for choosing:

Start with Regex Guard if you have well-defined patterns to catch. PII with known formats (SSNs, credit cards, phone numbers), prompt injection signatures, credential patterns, infrastructure identifiers. Zero infrastructure cost, sub-millisecond latency, 100% deterministic. This should be the baseline for any deployment.
Add Content Guard (Presidio) if you need NLP-based entity detection that goes beyond fixed patterns. Presidio catches entity variations that a regex might miss (misspelled names, partial phone numbers, context-dependent PII). Requires deploying a Presidio analyzer instance.
Add NVIDIA NIMs if you need semantic threat detection. Jailbreak attempts, subtle harmful content, topic drift, and prompt injection that doesn't use obvious keywords. Requires GPU infrastructure. This is where you catch threats that pattern matching fundamentally cannot.
Add IBM Granite Guardian if you need hallucination detection or RAG quality assessment. No other tier in the pipeline offers this. It’s also valuable for organizations in the IBM ecosystem. Keep in mind, it requires a GPU.
Enable parallel execution whenever you deploy LLM Guards. There's no reason to stack latency in series when the guards are independent.
Enable graceful error handling if your consumers include autonomous agents or if you use middleware chains that expect structured JSON at every stage.

What's Next

The composable pipeline is designed for extension. Traefik has integrated guardrails from NVIDIA, IBM, and Microsoft, and will continue integrating third-party providers as the ecosystem matures. Organizations can combine these vendor integrations with their own custom Regex Guard rules, building a safety pipeline tailored to their specific threat model, compliance requirements, and infrastructure constraints.

Traefik Hub v3.20 is now available as an Early Access release, with general availability planned for late April 2026. To try the composable safety pipeline and the other features covered in this post, sign up for Early Access.

]]>

How to Choose the Right Load Balancing Strategy for Your Use Case

Simon Delicata — Thu, 05 Feb 2026 17:11:06 GMT

In a previous article on advanced load balancing, we explored features like sticky sessions, health checks, and traffic mirroring. Those are the tools. This article covers the strategy.

Your application runs smoothly with round-robin load balancing. Traffic distributes evenly, everything works. Then you notice some requests take 50ms, others 500ms. Monitoring shows all servers are healthy, but user experience is inconsistent. The problem isn't if you're load balancing—it's how.

Load balancing strategies determine which server handles which request, and this decision has a direct impact on performance, cost, and user experience. Choosing the wrong strategy is like taking the highway when you need to navigate city streets—you might eventually get there, but you're not adapting to the terrain.

Traefik's load balancing strategies map to specific technical requirements. Weighted Round Robin (WRR) works for uniform backends. Power of Two Choices (P2C) handles variable connection lifetimes. Highest Random Weight (HRW) optimizes caching. Least-Time adapts to heterogeneous performance.

Understanding the Landscape: Features vs. Strategies

First, an important distinction:

Load balancing features (sticky sessions, health checks, mirroring) define capabilities—what your load balancer can do.

Load balancing strategies define decision-making—which server gets the next request.

In other words, features are your car's options: heated seats, navigation, backup camera. Strategies are your driving style: highway cruising, city maneuvering, off-road driving.

Both matter, but serve different purposes. This article focuses on strategies—the algorithms that route your traffic.

The Four Core Strategies: Overview

Traefik supports four primary load balancing strategies at the server level:

Strategy	How It Works	Why Choose It
WRR (Weighted Round Robin)	Fair rotation with optional weights	General-purpose, predictable workloads
P2C (Power of Two Choices)	Pick 2 random servers, choose the one with fewer connections	Dynamic scaling, variable connection lifetimes
HRW (Highest Random Weight)	Consistent hashing per client	Cache optimization, client affinity
Least-Time	Lowest response time + active connections	Heterogeneous backends, performance optimization

Each strategy maps to specific business scenarios.

Matching Strategy to Your Use Case

These scenarios are presented in order of increasing complexity, but choose based on your technical requirements, not your company size or stage. A caching-heavy product should use HRW from day one, while a large-scale service with uniform workloads might use WRR forever.

Scenario 1: Simple and Predictable—WRR

Great for: Internal tools, stable production services

Use When

Homogeneous backends (all servers are similar)
Predictable traffic patterns
Simplicity and reliability are priorities
You don't need advanced optimization

The Problem You're Solving
You've deployed three identical application servers and need to distribute traffic. Your focus is on uptime and simplicity—not premature optimization.

The Solution: Weighted Round Robin (WRR)
WRR is Traefik's default strategy. It distributes requests in a predictable, fair rotation:

http:
 services:
   my-api:
     loadBalancer:
       servers:
         - url: "http://server-1:8080"
         - url: "http://server-2:8080"
         - url: "http://server-3:8080"

With WRR, if you send 300 requests, each server handles approximately 100 over time. Simple, predictable, reliable.

When to Add Weights
When you have servers with different capabilities, WRR's weight system lets you send proportionally more traffic to more powerful machines. The higher the weight, the more traffic the server gets.

http:
 services:
   my-api:
     loadBalancer:
       servers:
         - url: "http://small-server:8080"
           weight: 1
         - url: "http://medium-server:8080"
           weight: 2
         - url: "http://large-server:8080"
           weight: 4  # Gets 4x traffic over time of small server

Benefits and Trade-Offs
WRR delivers predictable costs and straightforward capacity planning. Troubleshooting is simple since you always know which server handled a request.

The trade-off: WRR doesn't adapt to real-time conditions. All requests are treated equally, regardless of complexity or current server load.

Consider Other Strategies When
Your infrastructure becomes heterogeneous, connection lifetimes vary significantly, or you need caching/performance optimization.

Scenario 2: Variable Connection Lifetimes—P2C

Great for: Real-time applications, auto-scaling environments

Use When

Variable connection lifetimes (WebSockets, long-polling, streaming mixed with short requests)
Auto-scaling infrastructure (servers frequently added/removed)
Connection count imbalance is causing performance issues
You need automatic load distribution without manual tuning

The Problem You're Solving
You've implemented auto-scaling, but you notice uneven load distribution. Some servers are overloaded while others are idle. Your monitoring shows:

Server A: 150 active connections, CPU at 80%
Server B: 30 active connections, CPU at 20%
Server C: 200 active connections, CPU at 95%

Your Weighted Round Robin (WRR) strategy keeps sending traffic to Server C even though it's struggling.

The Solution: Power of Two Choices (P2C)
P2C uses a simple algorithm (hence the name): randomly pick two servers, then choose the one with fewer active connections.

http:
 services:
   my-api:
     loadBalancer:
       strategy: "p2c"
       servers:
         - url: "http://server-1:8080"
         - url: "http://server-2:8080"
         - url: "http://server-3:8080"

How It Works

Request arrives
Traefik randomly selects two servers (say, A and C)
Compares their active connection counts
Routes to the less loaded server (A has 150, C has 200 → choose A)

This simple algorithm balances connections effectively with minimal overhead.

Real-World Scenario: Real-Time Collaboration Platform
You run a SaaS collaboration platform (think Figma, or Google Docs) that combines real-time editing with standard API operations.

Your infrastructure handles:

Quick API calls: Loading documents, saving comments (50-200ms, connection closes immediately)
Live editing sessions: WebSocket connections for real-time collaboration (users stay connected for 30-60 minutes while editing)
File operations: Document exports, image uploads (30-90 second connections)

With WRR, all servers initially get equal requests, but connection lifetimes vary dramatically. Server A has 8 active editing sessions and 8 WebSockets open for the next hour. Server B has 150 quick API calls that all completed, and the server is now idle. Server C has 5 file uploads and the connections are held for 60 seconds.

WRR keeps distributing requests evenly, regardless of active connections. Once a new editing session starts, it's sent to the already-loaded Server A.

Now Server A is overloaded, Server B is underutilized, and users wonder why their editing sessions is so laggy.

With P2C, however, Traefik randomly picks two servers, compares active connection counts. It recognizes that Server A has 8 connections and Server B has 0, so Traefik routes to Server B.

Long-lived editing sessions naturally spread across available capacity, while quick API calls fill gaps on less-loaded servers. The result is smooth real-time collaboration for all users with optimal resource usage.

Benefits and Trade-Offs
P2C automatically distributes load without manual tuning, handling mixed workloads gracefully. It reduces hot spots and improves resource utilization, which translates to lower cloud costs.

The trade-off: P2C only considers connection count, not processing time. A server handling 10 lightweight requests looks the same as one handling 10 heavy requests. The random selection also makes debugging slightly less predictable than WRR.

Consider Other Strategies When
You need client affinity for caching (HRW), or backend performance varies significantly (Least-Time).

Scenario 3: Cache Optimization—Client Affinity with HRW

Great for: Any stage with caching requirements (CDNs, personalization platforms, session-heavy apps)

Use When

Stateful backends or caching layers
Users benefit from hitting the same server repeatedly
Cache hit rate is more important than perfect load distribution
You have session data, user-specific caches, or personalized content

The Problem You're Solving
You have multiple backend servers, each building up its own cache (Redis, in-memory, CDN edge). With P2C or WRR, a single user's requests bounce between different servers, resulting in:

Cache misses (user's data isn't on the selected server)
Repeated cache warming for the same user across servers
Wasted memory and processing power

The Solution: Highest Random Weight (HRW)
HRW (also called Rendezvous hashing) uses consistent hashing to map clients to servers deterministically.

http:
 services:
   cached-api:
     loadBalancer:
       strategy: "hrw"
       servers:
         - url: "http://cache-server-1:8080"
         - url: "http://cache-server-2:8080"
         - url: "http://cache-server-3:8080"

How it works:

Traefik hashes the client's IP address
Calculates a score for each server using the hash
Consistently routes that client to the highest-scoring server
Same client routes to same server unless servers are added/removed
When servers change, only ~1/N clients get remapped (vs. 100% with simple hash % server_count)

Real-World Scenario: User-Specific Content Caching
You run a personalized recommendation platform. Each server builds in-memory caches of user preferences, browsing history, and ML model outputs.

With WRR/P2C, a user requests a recommendation, which is routed to Server A→ cache miss → compute (100ms). The same user makes a second request, which is then routed to Server B → cache miss → compute (100ms). The result is frequent cache misses (users randomly distributed across servers) and high CPU usage across all servers.

With HRW, a user requests a recommendation, which is routed to Server A (based on client hash) → cache miss → compute and cache (100ms). The user's second request, however, is also routed to Server A (same hash) → cache hit (5ms). With HRW, cache hit rates climb since users consistently hit the same server, which significantly reduces CPU usage from cache efficiency.

Benefits and Trade-offs
HRW maximizes cache efficiency by ensuring the same client consistently hits the same server. This reduces origin load and makes debugging straightforward since routing is deterministic.

The trade-off is that HRW prioritizes consistency over load distribution. If one client generates disproportionate traffic, its assigned server becomes a hotspot. Adding or removing servers also requires careful planning since it triggers client remapping.

Consider Other Strategies When
Load imbalance becomes problematic, or backend performance varies significantly and you need adaptive routing (Least-Time).

Scenario 4: Performance Optimization—Adaptive Routing with Least-Time

Great for: High-traffic applications, performance-critical services, and heterogeneous infrastructure

Use When

Backend performance varies (mixed instance types, different hardware)
Every millisecond of latency matters
You want automatic adaptation to performance changes
Backends have different processing speeds

The Problem You're Solving
Your infrastructure uses mixed instance types:

Server A: High-performance dedicated server (5ms average response)
Server B: Standard cloud instance (15ms average response)
Server C: Cheaper instance with variable performance (10-50ms)

With previous strategies, you face impossible trade-offs:

WRR: Treats all servers equally, users randomly get slow responses
P2C: Balances connections but ignores that Server A is 3x faster
HRW: Sticks users to potentially slow servers

The Solution: Least-Time Strategy
Least-Time combines response time measurement with active connection tracking to route intelligently

http:
 services:
   backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://fast-server:8080"
         - url: "http://standard-server:8080"
         - url: "http://variable-server:8080"

How It Works

For each server, Traefik calculates a score based on recent response times (Time To First Byte) and active connections. Requests are routed to the server with the lowest score. Fast servers with few active connections automatically get more traffic, while degrading servers are deprioritized.

Real-World Scenario: Mixed Instance Types with Weighted Least-Time
Your application runs on heterogeneous infrastructure:

Server A: High-performance dedicated instance (5ms average API response)
Server B: Standard cloud instance (15ms average API response)
Server C: Burstable instance with variable performance (10-50ms depending on CPU credits)

With WRR, all servers receive equal traffic. Users randomly experience 5ms, 15ms, or 50ms responses, which creates an inconsistent user experience. Fast Server A is underutilized while slow Server C gets equal load.

With Least-Time, however, Traefik measures each backend's actual response time (TTFB). Server A (fastest) automatically receives more traffic, while Server C receives less traffic when its CPU credits are depleted. Then, when Server C performance improves, traffic naturally increases. The user experience is consistent, and resource utilization is optimal.

Using weights for capacity-aware routing:

http:
 services:
   backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://high-perf-server:8080"
           weight: 3  # Premium instance, can handle more
         - url: "http://standard-server-1:8080"
           weight: 1
         - url: "http://burstable-server:8080"
           weight: 1

Weights indicate capacity. A server with weight=3 can handle 3x the traffic before its score becomes unfavorable, allowing you to maximize value from premium infrastructure.

Benefits and Trade-Offs
Least-Time delivers near-optimal performance by routing to the fastest available backend. It adapts automatically when servers degrade—no manual intervention required. This leads to better resource utilization and graceful degradation without hard failovers. It's particularly effective for mixed infrastructure where backend performance varies.

The trade-off is: accurate routing depends on stable network conditions for reliable measurements. There's also slight computational overhead from tracking response times, though this is negligible in practice.

Beyond Server Strategies: Service-Level Load Balancing

So far, we've discussed server-level strategies—how to distribute traffic among backend instances. Traefik also offers service-level strategies for advanced patterns:

Weighted Round Robin (Service Level)

Distribute traffic between different services (not just servers). This is perfect for:

Canary deployments: 95% to stable version, 5% to new version
Blue-green deployments: Gradual traffic shifting
A/B testing: Route percentage of users to experiment variants

http:
 services:
   app:
     weighted:
       services:
         - name: stable-v1
           weight: 95
         - name: canary-v2
           weight: 5

Mirroring

Duplicate traffic to multiple services, which is essential for:

Testing new versions with real traffic (without risking production)
Performance comparisons between implementations
Data pipeline validation

http:
 services:
   api-with-mirror:
     mirroring:
       service: production-api
       mirrors:
         - name: new-api-version
           percent: 10  # Mirror 10% of traffic

Failover

Automatic fallback when primary service fails:

http:
 services:
   resilient-api:
     failover:
       service: primary-cluster
       fallback: backup-cluster

Combining Server and Service-Level Strategies

You can combine strategies at multiple levels. For example, you can use Least-Time at the server level for each service, and weighted service-level balancing for canary deployments:

http:
 services:
   # Service-level: Weighted between versions
   app:
     weighted:
       services:
         - name: v1-backend
           weight: 90
         - name: v2-backend
           weight: 10

   # Server-level: Least-Time within each version
   v1-backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://v1-server-1:8080"
         - url: "http://v1-server-2:8080"

   v2-backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://v2-server-1:8080"
         - url: "http://v2-server-2:8080"

Conclusion: Choosing Your Strategy

Use this decision tree to select the best strategy for your use case:

Quick Reference Table

Your Scenario	Recommended Strategy	Why It's Best
Identical servers, uniform requests	WRR	Simple, predictable, no overhead
Auto-scaling, variable connection lifetimes	P2C	Balances connection count dynamically
Client affinity for caching, stateful backends	HRW	Consistent client→server mapping
Different server types/performance	WRR with weights	Proportional to capacity
Heterogeneous backends, latency-sensitive	Least-Time	Optimizes for actual performance
Canary/blue-green deployments	Weighted (service-level)	Control traffic percentage
Testing with production traffic	Mirroring	Safe real-world validation

Load balancing isn't one-size-fits-all—but it doesn't have to be complicated either. Start with WRR, measure your system's behavior, and evolve deliberately.

Traefik lets you change strategies without downtime, so you can adapt as your requirements change.

The Infrastructure Reality Behind AI Hype: What the 2026 CNCF Survey Reveals (And What It Doesn't)

Sudeep Goswami — Mon, 02 Feb 2026 23:41:47 GMT

The January 2026 CNCF Annual Cloud Native Survey landed this month. Buried beneath the expected headlines about Kubernetes dominance is something far more interesting: a comprehensive map of where enterprise infrastructure investment is actually going, and where it's not.

The survey, based on 628 respondents across multiple industries and geographies, confirms what many of us building infrastructure have observed. The AI conversation has fundamentally shifted from "who can train the best model" to "who can operationalize inference at scale." This shift has profound implications for where value will be created and captured over the next several years.

The Inference Economy Is Here

Let's start with the most striking finding: 52% of organizations don't build or train their own AI models. They're consumers. The remaining organizations that do "train" are predominantly fine-tuning existing models rather than building from scratch.

This shouldn't surprise anyone who's been paying attention, but it should reframe how we think about AI infrastructure investment. The report's authors put it well: "If we cut through the hype of chatbots and agents, we can clearly see that we will need to greatly decrease the difficulty of serving AI workloads while massively increasing the amount of inference capacity available across the industry."

Take a moment and let that sink in. The infrastructure challenge isn't about building bigger GPU clusters for training. It's about the unglamorous work of routing, caching, rate limiting, and managing inference traffic at production scale.

Consider the deployment frequency data: 47% of organizations deploy AI models only occasionally (a few times per year), and just 7% manage daily deployments. The gap between "we have a model" and "we have a production AI system" remains enormous. That gap is an infrastructure problem, not an algorithm problem.

There's also a question the survey doesn't ask. 37% of organizations using managed generative AI APIs are sending inference traffic somewhere. Where? Under what data residency constraints? With what model governance guarantees? As the EU AI Act takes effect and regulated industries face increasing scrutiny over AI decision-making, the sovereignty dimension of inference infrastructure will become harder to ignore.

Kubernetes Won. Now What?

The survey confirms what we've observed in our customer base: 82% of container users now run Kubernetes in production, up from 66% in 2023. The report aptly describes Kubernetes as "boring," and correctly notes that boring is the highest compliment in infrastructure.

But here's the more interesting number: 66% of organizations are using Kubernetes specifically to host their generative AI workloads. Kubernetes has evolved from "container orchestrator" to "AI infrastructure platform." The report notes that projects like Kubeflow provide end-to-end ML workflows while KServe handles model serving at scale.

This convergence creates a clear market reality: whatever you build for AI workloads needs to be Kubernetes-native. Not Kubernetes-compatible. Not Kubernetes-adjacent. Native. The organizations that have achieved "true MLOps maturity" (the 23% running all inference workloads on Kubernetes) have done so by integrating AI into their existing CI/CD pipelines, GitOps workflows, and observability stacks.

Security Hasn't Gone Away. Complexity Has Caught Up.

The survey asked respondents about their top challenges in deploying containers. "Cultural changes with the development team" came in at #1 with 47%, followed by "lack of training" (36%) and "security" (36%).

That security figure deserves attention. In 2023, security was the top challenge. It hasn't become less important. If anything, it's become more critical as AI workloads introduce new attack surfaces. What's changed is that platform engineering complexity has risen to match it as an operational bottleneck.

The report's authors frame the cultural challenge as evidence for Platform Engineering: building standardized platforms with "paved roads, sensible defaults, and clear guardrails." That framing is correct, but incomplete. The cultural resistance the survey identifies isn't irrational. Developers express skepticism that containers add unnecessary complexity for simple applications. Operations teams worry about troubleshooting containerized systems. Management questions whether investments are distracting from feature delivery.

Now, layer AI agents on top of this. The MCP revolution is enabling AI systems to call external tools, access databases, and take autonomous actions. Each of those capabilities is a new permission surface. Each is a new governance challenge. Organizations struggling with basic container deployment complexity aren't ready for agent-to-agent communication patterns where security and access control become exponentially harder to reason about.

The security challenge hasn't diminished. It's compounding. And it's compounding on top of platform engineering foundations that many organizations haven't solidified.

GitOps as the Maturity Marker

The survey segments organizations into four maturity profiles: explorers (8%), adopters (32%), practitioners (34%), and innovators (25%). The progression between these stages correlates strongly with specific technology and practice adoption.

The most telling correlation is GitOps adoption: 0% of explorers have implemented GitOps, while 58% of innovators run GitOps-compliant deployments. CI/CD shows a similar pattern. 42% of explorers use it versus 91% of innovators.

Figure 17, 2026 CNCF Annual Cloud Native Survey

Figure 18, 2026 CNCF Annual Cloud Native Survey

This isn't just correlation. GitOps requires, and forces, a level of operational discipline that defines mature cloud native organizations: declarative configurations, version control for infrastructure, automated synchronization, and clear audit trails. That last point matters enormously for security. You can't secure what you can't audit. You can't audit what isn't versioned. Organizations that can't achieve this discipline will struggle with AI workloads, which require even more rigorous deployment practices and access controls.

The survey also reveals that innovators operate at a fundamentally different velocity. 74% check in code multiple times daily (versus 35% of explorers). 41% run daily releases (versus 12% of explorers). And 59% automate more than 60% of their deployments (versus 6% of explorers).

If your infrastructure layer requires ClickOps, you're building for yesterday's market. And if your security model depends on manual reviews that can't keep pace with automated deployments, you're building a compliance gap that will only widen.

The Sustainability Warning

Buried in the report is a warning that deserves more attention. A September 2025 open letter from open source infrastructure stewards warned that critical systems operate under "a dangerously fragile premise," relying on goodwill rather than sustainable funding models.

The letter specifically calls out AI/ML workloads as drivers of "machine-driven, often wasteful automated usage" that strains open source infrastructure. The report notes that "commercial-scale workloads often run without caching, throttling, or even awareness of the strain they impose."

This should concern anyone building AI infrastructure. The naive approach (more API calls, more requests, more brute-force retry logic) is technically simple and operationally catastrophic. Organizations that build efficiency into their AI infrastructure from day one will have both cost and sustainability advantages.

What This Means for Infrastructure Strategy

The CNCF survey paints a clear picture of where infrastructure investment should flow:

Inference over training. The majority of organizations are consumers of models, not producers. Infrastructure that optimizes inference through intelligent caching, efficient routing, and scale-to-zero capabilities will capture more value than training-focused solutions.
Kubernetes-native is table stakes. With 66% of organizations running AI workloads on Kubernetes, solutions that require separate orchestration or bolt-on integrations will face adoption friction.
Security and simplicity aren't trade-offs. When "cultural changes" and "security" are both top-three challenges, infrastructure that reduces cognitive complexity while strengthening access controls has a real moat. This becomes even more critical as MCP-enabled agents expand the permission surface.
GitOps compatibility is non-negotiable. The correlation between GitOps adoption and organizational maturity is too strong to ignore. Any infrastructure that requires manual configuration or UI-driven workflows is building for a shrinking market segment. More importantly, GitOps provides the audit trail that security and compliance teams increasingly demand.
Efficiency matters more than we admit. The sustainability concerns raised in the report aren't just ethical considerations. They're operational realities. Inefficient AI infrastructure is expensive AI infrastructure.

Questions for the Road Ahead

The survey opens as many questions as it answers.

As organizations move from occasional AI deployments to production-scale inference, how are they governing agentic workflows? The MCP ecosystem is evolving rapidly, but we have no baseline data on adoption patterns or access control approaches.
Similarly, the Kubernetes-centric view is appropriate for CNCF, but enterprises increasingly manage traffic across VMs, containers, and serverless simultaneously. How are organizations handling that complexity?
And at the most fundamental level: how is traffic actually flowing through these systems? The survey covers orchestration and observability well, but the routing and governance layer between services remains unexplored.

These aren't criticisms. They're signs of a maturing ecosystem where the interesting questions keep evolving.

The Bottom Line

The CNCF survey confirms that we're past the "should we do AI" phase and deep into the "how do we operationalize AI" phase. The winners won't be determined by who has the best model. Foundation model providers have commoditized that race. The winners will be determined by who can move inference workloads from demo to production at scale, with the governance, security, and reliability that enterprises require.

That's an infrastructure problem. And infrastructure problems are what we solve.

]]>

From Zero to Production-Grade WAF (Without the Appliance Pain)

Carlos Villanúa Fernández, Immánuel Fodor — Thu, 29 Jan 2026 20:55:46 GMT

Traefik Hub + Coraza + OWASP CRS give developers a practical, high-powered WAF that fits cloud-native workflows.

Let us start with the uncomfortable truth: your app is getting hammered long before it hits prod.

As of January 2026, Imperva reports that almost 51% of internet traffic is non-human, and bad bots alone make up nearly one-third of all traffic. Data Breach Investigations Report 2025 adds a second punch: a 34% increase in attackers exploiting vulnerabilities to gain initial access. These are the turbulent waters your APIs are shipping into.

And it gets worse. SQL injection alone accounts for 65% of all web application attacks. A 20+ year-old vulnerability class that still dominates. When breaches happen, IBM's 2025 report shows U.S. costs average $10.22 million, while global costs average at $4.44 million.

This is why Web Application Firewalls (WAFs) still matter. Not because they replace secure coding, but because they buy you time and block obvious exploitation while you patch, tune, and ship.

The other truth: WAFs used to mean expensive appliances and slow integrations. That is no longer the case. Traefik Hub API Gateway integrates Coraza WAF as a native feature, providing high-performance protection with an implementation flow that fits developer workflows. Coraza and OWASP CRS are both well-maintained, standalone open-source projects, and now Traefik Hub packages them into a production-ready gateway.

Before we dive into how to deploy and test Traefik Hub's native WAF integration, let's quickly explore why they matter and the technologies behind them.

Why WAFs Matter (Right Now)

A WAF is a shield in front of your app. It inspects HTTP traffic and blocks known malicious patterns before requests hit your code. You still need secure coding, auth, and rate limiting. But WAFs are your safety net when:

You cannot patch instantly
Attackers are automated and relentless
A single vulnerable endpoint could open the door to [insert your most valuable information sources here]

The dataset behind the OWASP Top 10:2025 document includes over 2.8 million applications. In that data, common risk categories remain pervasive: Broken Access Control, Security Misconfiguration, and Cryptographic Failures show up in production apps at scale. Those are not edge cases. They are everyday issues.

Here's how WAF coverage maps to the OWASP Top 10:

Rank	Category	WAF Coverage
A01	Broken Access Control	Custom rules
A03	Software Supply Chain Failures	Partial
A04	Cryptographic Failures	Limited
A05	Injection (SQLi, XSS)	Full CRS coverage
A06	Insecure Design	Limited
A07	Authentication Failures	Custom rules
A10	Mishandling of Exceptions	Partial

Notice that injection attacks (the category that includes SQL injection and XSS) get full CRS coverage out of the box. Given that SQLi represents 65% of web app attacks, that is significant protection for minimal effort.

Your Defense Stack... In Plain English

Here is what each piece does:

ModSecurity: the original open-source WAF engine. The project calls it the "Swiss Army Knife" of WAFs. It defined the standard for rule-based web application firewalls.
SecLang: the ModSecurity rules language. Rules look like SecRule VARIABLES "@OPERATOR ..." "ACTIONS".
Coraza: a modern, high-performance WAF engine that speaks the same language as ModSecurity (SecLang). Written in Go, memory-safe, actively maintained. Think of it as the next-generation engine that runs the same rules.
OWASP CRS: the rule set. A community-maintained collection of generic attack detection rules designed to cover the OWASP Top Ten with minimal false alerts. Over 500 rule bypasses closed through their bug bounty program.
Traefik Hub API Gateway: a commercial gateway that integrates Coraza WAF and makes deployment and tuning practical for teams.

Coraza gives you the WAF engine, CRS gives you the rules, and Traefik Hub turns it into a clean, deployable, version-controlled middleware.

The WAF Landscape: Why Cloud-Native Wins

The Problem with Legacy Enterprise WAFs

Traditional enterprise WAFs come with enterprise pricing: $50,000 to $100,000+ annually. They require dedicated security teams, complex deployment processes, and were not designed for Kubernetes or microservices architectures. Hidden costs abound: throughput limits, overage charges, and mandatory professional services.

Cloud Provider WAFs: Death by a Thousand Cuts

AWS WAF looks affordable at first: $5/month plus $1/rule plus $0.60/million requests. But add Shield Advanced for DDoS protection, and you are looking at ~$3,000/month. Costs scale unpredictably with traffic, and you are locked into a single cloud provider.

The real problem is not just cost. It is opacity. Cloud provider WAFs are black boxes. You cannot see exactly what rules are doing, you cannot customize the detection logic, and you cannot audit the security decisions being made. When a false positive blocks legitimate traffic, debugging is guesswork.

The Traefik Hub Gateway-Backed Approach

Traefik Hub takes a different path:

Affordable pricing: Enterprise-grade WAF without the enterprise “big-vendor” pricing
Cloud-native architecture: Built for Kubernetes and microservices from day one
Native Coraza integration: No sidecars, no WASM overhead, no complex setup
Open-source foundations: Battle-tested Coraza engine & OWASP CRS rules
Transparent rules: You can see exactly what is being blocked, no black box

You get the best of both worlds: an open-source WAF engine with community-hardened rules, commercial support, and native integration through Traefik Hub.

Architecture at a Glance

WAF protection is applied as middleware. That design matters because you can:

Enable it per route or per service
Start in detection-only mode
Layer it with OIDC, JWT, or rate limiting
Tune rules without touching application code

This is the developer workflow we want: security becomes a YAML configuration change, reviewed in Git, shipped with the same pipeline as your app.

Hands-On: From Zero to Protected

Here is a minimal middleware that enables CRS and turns the engine on:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: waf
  namespace: apps
spec:
  plugin:
    coraza:
      crsEnabled: true
      directives:
        - Include @coraza.conf-recommended
        - Include @crs-setup.conf.example
        - Include @owasp_crs/*.conf
        - SecRuleEngine On
        - SecRequestBodyAccess On

Attach that middleware to an IngressRoute, and your service is protected.

Quick Start: A Local Demo You Can Actually Run

This kit uses k3d + Traefik Hub + Coraza + CRS, then runs Bruno (or Postman) tests to validate 200 (OK) vs 403 (Forbidden) behavior.

# 0) Unzip the the Workshop Repository
cd traefik-waf-test-kit

# 1) Create a local cluster
k3d cluster create traefik-waf \
  --port 80:80@loadbalancer \
  --port 443:443@loadbalancer \
  --port 8000:8000@loadbalancer \
  --k3s-arg "--disable=traefik@server:0"

# 2) Add Traefik Helm Repository
helm repo add --force-update traefik https://traefik.github.io/charts

# 3) Create Traefik Namespace and License
kubectl create namespace traefik

# Note: Generate your Traefik Hub token from https://hub.traefik.io  
# Replace <TRAEFIK_HUB_TOKEN> with the token from hub.traefik.io
kubectl create secret generic traefik-hub-license --namespace traefik --from-literal=token=<TRAEFIK_HUB_TOKEN>

# 4) Install Traefik Hub
helm upgrade --install --namespace traefik traefik traefik/traefik \
  --set hub.token=traefik-hub-license \
  --set ingressRoute.dashboard.enabled=true \
  --set ingressRoute.dashboard.matchRule='Host(`dashboard.docker.localhost`)' \
  --set ingressRoute.dashboard.entryPoints={web} \
  --set image.registry=ghcr.io --set image.repository=traefik/traefik-hub --set image.tag=v3.19.0

# 5) Create Application Namespace
kubectl create namespace apps

# 6) Deploy the httpbin app + WAF middlewares
kubectl apply -f httpbin-waf/

Now validate with a normal request and a SQL injection attempt:

# normal request -> 200 OK
curl -i "http://httpbin.traefik.localhost/waf/anything?test=hello"

# SQLi attempt -> 403 Forbidden
curl -i "http://httpbin.traefik.localhost/waf/anything?user=admin&password=test%27%20OR%20%271%27%3D%271"

You can also run the Bruno collection in bruno-waf-tests/ for full coverage across SQLi, XSS, bots, traversal, command injection, and sensitive files examples.

Want to see the full test suite in action? Try Traefik Hub with a complete WAF test kit, which includes 19 pre-built attack scenarios across 8 categories, plus the full middleware config with 50+ custom rules. Request your trial here.

How the Rules Work (So You Can Tune Them)

CRS is not just regex. It is a structured ruleset that targets specific attack classes and uses anomaly scoring to decide when to block. That is why tuning is possible without turning protection into guesswork.

SecLang rules are readable and direct. The syntax is:

SecRule VARIABLES "@OPERATOR OPERATOR_ARGUMENTS" "ACTIONS"

For example:

SecRule ARGS "@detectSQLi" "id:302,phase:2,deny,msg:'SQL Injection detected'"

This rule checks all request arguments (ARGS) for SQL injection patterns (@detectSQLi), runs during phase 2 of request processing, denies matching requests, and logs a message. The syntax is powerful enough to express complex conditions, yet readable enough for security teams to audit.

You can use CRS for baseline protection and then add tight custom rules for your app's specific needs:

# Admin path protection
- SecRule REQUEST_URI "@rx /admin" "id:102,phase:1,log,deny,status:403,msg:'Admin path blocked'"

# Bot and scanner detection
- SecRule REQUEST_HEADERS:User-Agent "@pm sqlmap nmap nikto" "id:501,phase:1,deny,msg:'Scanner detected'"

# Sensitive file protection
- SecRule REQUEST_URI "@pm .git .env .conf" "id:505,phase:1,deny,msg:'Sensitive file access'"

Rollout Strategy That Won't Break Prod

A safe rollout looks like this:

1. Detection-only: Deploy with SecRuleEngine DetectionOnly to see what would be blocked without impacting production. Let it run for at least a week to capture normal traffic patterns.

- SecRuleEngine DetectionOnly

2. Review: Analyze the logs. Group blocked requests by rule ID. Look for false positives (legitimate requests that triggered rules). Common culprits include rich text editors, JSON payloads with code snippets, and query strings that resemble attacks.

3. Tune: Create exclusion rules for legitimate patterns. Be surgical. Do not disable entire rule categories when a narrow exclusion will suffice. CRS uses anomaly scoring, so you can also tune the tx.inbound_anomaly_score_threshold variable.

4. Enforce: Switch to blocking mode once you are confident.

- SecRuleEngine On

5. Expand progressively: Start with your lowest-risk endpoints, verify everything works, then expand coverage. This reduces the blast radius if you missed something during testing.

This is the difference between "WAF as a checkbox" and "WAF that actually helps." It should block real attacks, not your legitimate users.

Production Considerations

Defense in Depth

WAF is one layer among many you can add. Combine it with:

Rate limiting: Prevent brute force and volumetric attacks
Authentication: OIDC, JWT, or OAuth2 for identity verification
Security headers: CSP, HSTS, X-Frame-Options for browser-side protection
mTLS: Certificate-based service-to-service authentication

Traefik Hub supports all of these as composable middleware. Your security configuration lives in Kubernetes manifests: version-controlled, reviewable in pull requests, and deployed alongside your applications through the same GitOps pipelines.

Monitoring and Observability

A WAF that silently blocks attacks is not giving you the security posture improvement you need. Enable structured JSON audit logging and forward to your centralized logging platform (ELK, Loki, Splunk). Set up alerts for:

Blocked request spikes: Often indicates an active attack. Correlate with source IPs.
New attack patterns: Watch for rule IDs you have not seen before.
False positive patterns: Track when legitimate users report issues.
Anomaly score distributions: Monitor how close legitimate traffic gets to your blocking threshold.

Treat WAF logs as a security data source, not just application logs. They tell you what attackers are trying to do, which can inform broader security improvements.

Enterprise Protection with Developer Workflows

The payoff is not just security. It is developer time and operational sanity.

No heavyweight appliance or weeks-long integration
Proven rules mapped to the OWASP Top 10 risks
Local demo and validation in minutes
Configuration lives in Git and deploys with your app
Transparent, auditable rules you can read and customize
Crowdsourced intelligence with constant ruleset updates

WAFs no longer require massive security budgets. With Traefik Hub API Gateway integrating Coraza & OWASP CRS, you deploy a modern WAF in a way that fits developer workflows while still delivering production-grade performance.

Ready to experience cloud-native WAF protection? Click below to request your free trial of Traefik Hub and get our complete WAF Test Kit. It includes the full middleware config with 50+ security rules, Bruno test suite with 19 attack scenarios across 8 categories, Kubernetes deployment manifests, and a step-by-step deployment guide.

Sources

Imperva 2025 Bad Bot Report - 51% non-human traffic
Data Breach Investigation Report 2025 - 34% increase in vulnerability exploitation
SQL Injection = 65% of Web App Attacks - Dark Reading
IBM Cost of Data Breach Report 2025 - $4.44M global avg, $10.22M U.S.
OWASP Top 10:2025
OWASP Core Rule Set
Coraza WAF
Coraza SecLang Syntax
Traefik WAF Solution

]]>

The Illusion of Safety: Why the Ingress NGINX Fork is Not a Security Strategy

Emile Vauge — Tue, 27 Jan 2026 19:57:22 GMT

When a building’s foundation cracks, you don’t repaint the walls. You move.

On November 11, 2025, the Kubernetes SIG Network and Security Response Committee announced the upcoming retirement of the Ingress NGINX Controller. The project is now in a "best-effort" maintenance phase until March 2026. After this date, there will be no further releases, bug fixes, or security updates.

On December 22, 2025, Chainguard announced a maintenance fork of ingress-nginx via its "EmeritOSS" program.

For platform engineers and CTOs, this creates a tempting narrative: "We don't have to migrate. We can just switch the image and stay safe."

Here is the problem: that conclusion confuses Extended Support with Artificial Life Support.

In software, extended support is standard practice for stable, mature platforms (like Ubuntu ESM or Traefik’s own LTS versions). But ingress-nginx isn’t retiring because it is old. It is retiring because its creators deemed its architecture fundamentally unsafe to maintain.

Supporting a solid architecture is stability. Supporting a broken architecture is a liability.

Even Chainguard is explicit about the intent. They are not continuing development. They are maintaining a working version to buy users time to move to a different solution, with CVEs addressed on a "best-effort" basis.

The Hard Truth: Why the Experts Walked Away

To understand why the fork is a strategic error, we have to look at why the project ended. The Kubernetes SIG Network and Security Response Committee didn't retire ingress-nginx solely due to a lack of resources. They retired it because the project suffered from what they officially termed "insurmountable technical debt."

According to the official retirement announcement, the project’s flexibility—once its greatest asset—had become its greatest vulnerability. The maintainers were explicit:

"What were once considered helpful options have sometimes come to be considered serious security flaws, such as the ability to add arbitrary NGINX configuration directives via the 'snippets' annotations."

If the original maintainers—the world's foremost experts on this codebase—could not secure this architecture without a total rewrite, it is improbable that a third-party vendor can do so with a "maintenance-only" approach.

The Fork Fallacy: You Might Patch CVEs, But You Can’t Patch a Fundamentally Broken Architecture

Chainguard is a leader in supply chain security, and their ability to package software is undisputed. However, the corruption in ingress-nginx isn't in the packaging; it's in the fundamental logic.

The fork promises to "keep the lights on" by updating dependencies and patching known CVEs. But this ignores the structural reality that makes the project insecure:

The Templating Trap (The Root Cause)

The core flaw of ingress-nginx is its reliance on text-based templating. It functions by reading Kubernetes resources and mashing them into a single, massive nginx.conf file using templates.

The Risk: Features like "configuration snippets" allow users to inject raw NGINX directives and arbitrary commands directly into this template. This architecture effectively means the controller is continuously compiling untrusted user input into its root configuration.
The Reality: Securing this model against injection attacks without removing the flexibility users rely on is nearly impossible. A maintenance fork cannot fix this without rewriting the controller entirely.

The Memory Safety Gap

ingress-nginx is built on NGINX, which is written in C. It is inherently vulnerable to memory-safety issues like buffer overflows. Modern infrastructure demands memory-safe languages. Traefik is built in Go, which eliminates entire classes of memory vulnerabilities by default. A fork cannot "patch" C into Go.

The “Cold Restart” Model

Because of the templating model, many configuration changes require regenerating the file and reloading the worker processes. At scale, this can cause latency spikes and dropped connections. Traefik was born in the cloud-native era; it is purely dynamic, updating routing tables instantly via the Kubernetes API without ever restarting.

The "Many Eyes" Myth

Open-source security relies on community scrutiny—the concept that "given enough eyeballs, all bugs are shallow." Since the fork, the project has seen minimal traction. Crucially, issues and PRs are turned off in the repository. This is not a thriving community revival; it is a quiet, vendor-specific repository. By switching to the fork, you trade a community-supported standard for a niche, proprietary patch set with significantly fewer eyes on the code.

The Strategic Pivot: Evolution, Not Stagnation

Sticking with ingress-nginx is a decision to invest in the past. The industry is moving toward the Gateway API, a standard that solves many of the original Ingress limitations.

Your strategy for 2026 should focus on adopting these modern standards, not propping up a 2015 architecture. However, we understand that "rewriting everything" is a non-starter for most teams.

The Solution: An On-Ramp, Not a Cliff

The reality is that the new Gateway API standard, though excellent, doesn't yet offer perfect feature parity with the complex annotations many users rely on in Ingress NGINX. Furthermore, moving dozens or even hundreds of existing ingress resources to a completely new configuration format presents a significant and complex re-platforming challenge for any team.

That is why we built the Ultimate Ingress NGINX Migration Kit. It is a comprehensive tool designed to assist you in planning a progressive migration strategy, bridging the gap between where you are (Legacy Ingress) and where you need to be (Traefik & Gateway API).

Our approach includes three primary steps:

1. The Automated Analysis (Audit Your Risk)

Don't guess how complex your migration will be; know for sure. Use our open-source Ingress NGINX Migration Tool.

What it does: Connects to your cluster, analyzes every Ingress resource, and generates a detailed report.
The value: It tells you exactly which annotations you are using and maps them to Traefik features, flagging any edge cases before you change a single line of code.

2. The Drop-In Replacement (The "Buy Time" Feature)

The biggest blocker to migration is rewriting thousands of lines of YAML. We solved this with the Traefik Ingress NGINX Provider.

What it does: Traefik natively reads, understands, and translates your existing ingress-nginx annotations into Traefik configuration.
The value: You can swap the controller binary today without rewriting your application manifests. This eliminates the "Big Bang" migration risk, lands you on a well-maintained controller, and allows you to modernize your config at your own pace.

3. The Bridge to Gateway API (The “Future-Proofed” Path)

Traefik offers full, production-ready support for the Gateway API today. By moving to Traefik via the Migration Kit, you gain the power of coexistence.

What it does: You can run your legacy Ingress resources (using NGINX annotations) side-by-side with modern Gateway API routes on the same controller.
The value: This allows for a truly gradual migration. You don't need to modernize everything at once—you can reconfigure your services to use the new standard piece-by-piece, while Traefik handles traffic for both simultaneously.

Don't Wait for the Next "IngressNightmare"

The retirement of ingress-nginx marks the end of an era. The current Chainguard fork, or any future one, may keep the lights on for a few more months, but it cannot fix the crumbling foundation that led the Kubernetes team to walk away.

Your Action Plan

1. Assess your exposure: Visit ingressnginxmigration.org to understand the timeline and risks.

2. Audit now and help shape the future: Run the migration tool to get your custom report:

$ curl -sSL https://raw.githubusercontent.com/traefik/ingress-nginx-migration/main/scripts/install.sh | bash
$ ingress-nginx-migration

💡 Pro Tip: Share your migration report with us (anonymized by design). We are actively adding support for more annotations based on community feedback. Your report helps us prioritize the features that matter most to your setup.

3. Decommission NGINX Securely: Use the Traefik Ingress NGINX Provider to switch controllers and decommission Ingress NGINX without the headaches.

4. Migrate to Gateway API Progressively: Take control of your Gateway API adoption by planning it on your own schedule. Use regular sprint cycles to assess readiness, and deploy only when you are ready.

Don't let your infrastructure rely on life support. Move to a platform that is active, modern, and secure by design.

]]>

Ingress NGINX Retirement: The Ultimate Migration Kit

Emile Vauge — Tue, 09 Dec 2025 01:54:39 GMT

The ticking clock for Kubernetes networking has a date: March 2026.

For years, Ingress NGINX has been the default choice for routing traffic in Kubernetes clusters. It has served the community well, but recent announcements from the Kubernetes project confirm what we anticipated: Ingress NGINX is retiring, in less than 120 days.

We previously covered the urgency of this move in our articles on migrating from Ingress NGINX to Traefik now and strategies for the transition. Today, we are taking the next step to support the community with dedicated tools and resources.

The Hard Deadline

The official deprecation timeline is set. By March 2026, the project will cease all support. No patches. No updates. No security fixes.

If you are running Kubernetes in production, this is a critical risk. Running an unmaintained ingress controller exposes your infrastructure to future vulnerabilities that will simply go unpatched.

You Need a Plan

Panic is not a strategy, but preparation is. The move to the new Gateway API standard is the ultimate destination for the ecosystem, but rewriting every ingress route and reconfiguration every load balancer overnight is rarely feasible.

You need a strategy that covers both immediate security and future modernization:

Short Term (The Safety Net): Traefik with its NGINX provider is currently the only available solution on the market to provide native support for Ingress NGINX custom annotations. This makes Traefik the only true drop-in replacement, allowing you to migrate from Ingress NGINX with no changes required to your existing Ingress resources. You stay secure and supported without the need for a massive rewrite.
Long Term (The Modernization): Once you are on a stable, supported platform, you can gradually migrate to the Gateway API at your own pace, future-proofing your stack.

We Built The Ultimate Migration Kit

To help the community navigate this transition, we have consolidated all relevant information into a comprehensive project. We launched a dedicated website to guide you through every step of the process.

👉 The Migration Guide: ingressnginxmigration.org This website is your central resource. It breaks down the timeline, the risks, and provides a clear decision tree for your migration strategy.

Screenshot of ingressnginxmigration.org

👉 The Migration Tool: traefik/ingress-nginx-migration We didn't just write a guide; we built a powerful CLI tool to automate the pain away.

Screenshot of Migration Tool

Right now, the tool connects to your cluster, analyzes all your existing Ingress resources, and generates a comprehensive migration report. It identifies exactly which annotations you are using and maps out your path to Traefik’s NGINX-compatible provider.

We are actively developing this tool, with upcoming updates focused on Gateway API. Soon, it will not only help you migrate to a secure Ingress controller but also assist in migrating to the new Gateway API standard.

Making Migration a Non-Event

At Traefik Labs, we are dedicated to helping the community navigate this shift smoothly. Our goal is to turn this deprecation into a non-event for your infrastructure.

If you want to dive deeper into the migration process, join us at the webinar 100 Days to Migrate: Why Traefik is the Only Drop-in Replacement for Ingress NGINX.

Both the migration tool and the Traefik NGINX provider are fully Open Source. We built them for the community, and we believe in the power of collaboration. We invite you to try them, examine the code, and contribute. Whether it’s adding support for a specific annotation or improving the analysis logic, your contributions help make the path easier for everyone.

Don't wait for the deadline. Start planning your migration today.

Essential Resources

Check out the Guide
Ingress NGINX Migration Tool
Traefik NGINX Provider Documentation
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The Traefik Labs Manifesto: The Case for a Unified Runtime Layer

Sudeep Goswami — Thu, 04 Dec 2025 22:47:19 GMT

Modern infrastructure has built itself into a corner.

Over two decades of rational technology choices have produced an irrational result: a fractured architecture that slows development, complicates security, and prevents organizations from operating at the speed their business demands. This is not a problem we can optimize our way out of. It requires a fundamental shift in how we think about application infrastructure.

We believe the enterprise needs a unified runtime layer, not as a future luxury, but as an immediate necessity.

This is Traefik Labs' position. We have built our entire platform around this belief, and the industry is starting to recognize why it matters.

The Problem: Fragmentation Has Become the Default

Modern organizations run on a mosaic of execution environments: virtual machines, Kubernetes clusters, serverless platforms, public clouds, private clouds, sovereign regions, and edge locations. Each environment comes with its own governance model, networking assumptions, security policies, and operational culture. Each choice made sense in isolation. Collectively, they created chaos.

Every environment has its own ingress pattern, every system its own access rules, every team its own API conventions. Every workflow must navigate inconsistent connectivity, and every security model must be re-implemented. We have normalized dysfunction.

Developers spend more time wiring systems together than building features. Security teams struggle to enforce policies across incompatible platforms, while operations teams manage overlapping tools that don't talk to each other. We tell ourselves this is the cost of innovation, but in reality, it is the cost of fragmentation. It is holding the enterprise back.

The Tipping Point: AI Cannot Tolerate What Humans Have Learned to Ignore

Humans have learned to socially engineer solutions to these roadblocks; we open tickets, manually bridge gaps, and route around broken systems. AI cannot do this.

While an LLM can instantly adapt to a changing API schema or logic flow, it cannot bypass the hard physics of the network. When an AI agent hits a firewall, a missing route, or an ambiguous identity policy, it cannot "negotiate" access. It simply fails.

Agentic systems require stable APIs, trustworthy identity, predictable governance, end-to-end visibility, context-aware access, and consistency across environments. They need what the enterprise has historically struggled to provide: a coherent runtime.

AI is no longer confined to specialized projects; inference is becoming ubiquitous. Every application will soon make calls to language models, vision models, embedding models, and specialized AI services. These inference workloads will sit alongside traditional API traffic, demanding the same reliability, security, and observability as mission-critical services require today.

Agentic infrastructure is not creating a new problem. It is exposing the problem we have been tolerating for years, serving as the stress test that proves the current model is broken and forcing the conversation we should have had long ago.

The infrastructure that served the last era cannot serve the next one.

The Solution: A Unified Runtime Layer for All Workloads

A unified runtime layer sits above the execution environments but below the application logic. It standardizes how traffic flows, how identity is enforced, how APIs are exposed, how workloads are protected, and how intelligence is applied. Rather than replacing infrastructure, this approach brings coherence to it.

Unified Ingress Diagram

Our Principles

Consistent governance everywhere. Policies must be defined once and applied universally across Kubernetes, VMs, and edge nodes. By enforcing standards like OAS (OpenAPI Specification) at the definition level, we transform loose integrations into strict, governable contracts.
Universal connectivity. VMs, containers, functions, and legacy systems must behave like a single addressable estate. Connectivity is not negotiable. It is the baseline requirement for everything that follows.
Intelligent routing by default. Modern workloads need more than Layer 7 routing. They need context-aware decisions, semantic caching, and policy enforcement that understands intent rather than just packets.
Security as a standard, not an add-on. Security policies are agnostic to the underlying infrastructure. No exceptions, no workarounds, no environment-specific compromises.
Complete observability across all traffic. Every request, response, and failure must be traceable. Without observability, there is no understanding. And without understanding, there is no control.
Sovereignty and edge readiness. The runtime must function wherever the enterprise operates: global cloud regions, regulated environments, private clusters, and air-gapped infrastructures. Geography cannot break the model.
Developer experience is not cosmetic. If it is hard to use, it will not be used. Simplicity is a requirement for adoption, never a compromise on capability.

These principles define the application fabric enterprises need. They make applications more reliable, security more robust, and create the foundation AI requires to function.

The Traefik Vision: Application Intelligence as the Connective Layer

At Traefik Labs, we’ve spent years building the Application Intelligence Platform: a unified runtime solution that acts as the connective, governed, and intelligent layer for all application and AI traffic across an enterprise.

Our product suite embodies this vision:

Traefik Proxy unifies ingress, routing, and traffic management across containers (Kubernetes, ECS, Swarm, Nomad, etc.), VMs, and hybrid architectures.
Traefik API Gateway brings consistent identity, policy enforcement, and visibility to every environment.
Traefik AI Gateway manages AI inference traffic as a first-class workload, handling guardrails, cost controls, and semantic caching.
Traefik MCP Gateway ensures agentic AI systems interact with enterprise infrastructure safely and predictably.
Traefik API Management provides comprehensive governance, from developer portals to GitOps-driven operations at scale.

Together, they form a coherent architecture. Organizations can start with any Traefik Labs product and add capabilities seamlessly, in seconds, as requirements change. This makes our entire suite less a collection of tools and more a platform with a clear point of view.

Why This Matters

The organization that unifies its runtime layer will move faster, operate more securely, and adopt AI more confidently than those still managing fragmented infrastructure. This is a present requirement, not a distant future state.

Organizations are realizing that the runtime is the control plane for modern applications. Governance is the foundation, not a feature. Connectivity must be consistent everywhere, and APIs have become the enterprise's most critical interface. AI inference workloads will flow through the same infrastructure as every other service, requiring unified observability.

The question is no longer whether to unify, but when.

Our Commitment

Traefik Labs has always believed that connectivity should never be the limiting factor for innovation. As the enterprise enters the age of AI, this belief matters more than ever.

We have built the unified runtime layer that addresses the fragmentation enterprises face today while enabling the intelligent workloads they are deploying now. This architecture spans clouds, data centers, VMs, and edge, with a foundation in open standards and enterprise-grade reliability.

This is the foundation the industry needs. This is the architecture we have built. This is the future we are committed to advancing.

The unified runtime layer is no longer optional; it is the foundation that makes everything else possible.

]]>

AI Sovereignty: Why Control Is Your Ultimate Operating Leverage

Sudeep Goswami — Tue, 02 Dec 2025 23:43:28 GMT

Six months ago, the conversations happening in boardrooms across regulated industries were remarkably consistent. CIOs and CTOs were all fielding the same urgent question: "How do we adopt AI?"

Today, the conversation has flipped. The pressure to adopt hasn't gone away, but a new, more critical question has taken over: "How do we control AI?".

We are living through one of the most significant inflection points in enterprise technology. Artificial intelligence has moved from "science project" status to full-scale production at a rate faster than any technology in history. But this speed has come at a cost. It has created a crisis of AI Sprawl.

Developers started with personal ChatGPT accounts, then moved to corporate accounts, and finally to embedded APIs. Now, API keys are scattered across repositories, "shadow agents" are running in production, and billing statements are arriving with numbers that no one can fully explain.

For most companies, this is a governance headache. For regulated industries—Defense, Healthcare, Financial Services, and the Public Sector—it is an existential risk. The answer to this crisis lies in a concept that is frequently discussed but rarely defined correctly: AI Sovereignty.

The Illusion of Control: What Sovereignty Isn't

The term "sovereignty" is often thrown around loosely in technology discussions. Before we can understand what it requires, we need to be precise about what it is not. There are three common myths that lull organizations into a false sense of security.

1. The Data Residency Myth

Many organizations believe that if they store their data in a specific geography—say, a data center in Frankfurt or a specific US region—they are sovereign. But data residency is not sovereignty. If your data sits in Germany but the control plane managing it lives in Virginia, or if the AI model processes that data via an API call to a US-based hyperscaler, you do not have control. If the "brain" of the operation is elsewhere, the "body" is not sovereign.

2. The Hybrid Cloud Trap

"Hybrid cloud" is often sold as the solution to sovereignty, but it frequently masks deep dependencies. If your on-premise AI workload relies on public cloud services for authentication, billing, or management, you have created a fragility point. If your internet connection goes down, or if the cloud provider suffers an outage, your "sovereign" AI stops working. That isn't sovereignty; it is dependency.

3. Vendor-Managed Sovereignty

Perhaps the most dangerous myth is that you can buy sovereignty as a managed service. If you are operating on someone else’s infrastructure, under their terms, with their ability to change the rules, pricing, or availability at any moment, you are not in control. You are simply a tenant. True sovereignty means you are not operating on someone else's terms.

Defining True Sovereignty

If those are the illusions, what is the reality? In our new eBook, The Sovereign AI Infrastructure Imperative, we define true sovereignty through three non-negotiable capabilities.

First, You Need Architectural Control.

This means you can run your entire AI stack—gateway, models, safety systems, and governance—in your own environment. There are no required connections to external services. There is no "phone home" telemetry. If you pull the network cable, the system keeps working. For defense and intelligence sectors, this isn't a "nice to have"—it is an operational requirement.

Second, You Need Portable Governance.

Your policies, security controls, and audit trails must follow your workload. The same governance rules should apply whether you are running in the public cloud, on-premises, or in an air-gapped bunker. Your governance should be defined as code, not by clicking buttons in a proprietary cloud console that you can't export.

Third, You Need Escape Velocity.

This is the litmus test for sovereignty: Can you leave? True sovereignty means you are not locked into proprietary APIs or opaque platforms. You own the architecture. If a vendor changes their terms or a geopolitical event makes your current setup risky, you can migrate your entire stack without rewriting it.

Simply put: You own it, you control it, and you can move it.

The New Threat: The Agent Governance Crisis

Why is this definition becoming so critical right now? Because the nature of AI is changing. We are moving from "Chat" to "Action."

Six months ago, the risk was mostly about a user pasting sensitive data into a chatbot. Today, we are seeing the rise of Agentic AI—systems that use tools like the Model Context Protocol (MCP) to take actions. These agents read databases, call internal APIs, modify records, send emails, and execute code.

This breaks the traditional security model. In the past, you gave a human employee, let's call her Sarah, access to the database. If Sarah did something wrong, she was accountable. But agents are different. You cannot simply "authenticate once and trust forever." An agent might be doing exactly what it was prompted to do, but that prompt might be adversarial. Or the agent might be hallucinating. Or it might be exploring a solution path you didn't anticipate.

If you rely on a cloud provider for your governance, you are asking an external entity to police your internal agents. That introduces latency, metadata leakage, and security risks that regulated industries cannot accept. You need a security model that assumes agents are unprivileged actors requiring continuous authorization—and that authorization must happen on your infrastructure.

The Architecture of Control

To solve this, organizations need to move beyond simple "perimeter security." You cannot just put a firewall around your AI and hope for the best. You need defense in depth.

We have previously detailed the technical implementation of this approach, which we call the Triple Gate Pattern. It involves three distinct layers of defense:

The AI Gateway to secure the conversation (handling topic control and jailbreak detection).
The MCP Gateway to govern the tools (controlling what agents can actually do).
The API Gateway to protect the backend systems.

For a technical deep dive on how to implement these defensive layers, read our post: The Triple AI Security Gap.

The connection to sovereignty here is vital: You cannot effectively implement the Triple Gate Pattern if you are dependent on the cloud.

Traditional AI safety architectures rely on calling an external API to check if a prompt is safe. This creates three problems. First, Network Dependency: if you are in a defense installation without internet, you can't make that call. Second, Fragility: if the safety API goes down, your operations halt. Third, Metadata Leakage: even if you aren't sending the full data, you are revealing operational patterns to a third party.

True sovereignty requires that these safety checks—the NVIDIA NIMs, the policy enforcement, the agent governance—run locally on your hardware. The architecture must be offline-capable by default.

Sovereignty as Operating Leverage

We often hear sovereignty described as an insurance policy—something you buy to avoid fines from the EU AI Act or to comply with HIPAA. While that is true, it misses the bigger picture.

True sovereignty is offensive operating leverage.

When you build on truly portable, self-hosted infrastructure from day one, you gain strategic advantages that cloud-dependent competitors lack.

1. Negotiating Power

When you have "escape velocity"—the proven ability to move your stack without rewriting it—your relationship with vendors changes. You are no longer a captive customer who has to accept every price hike or term change. You are a partner with options. That optionality has real monetary value.

2. Deployment Agility

This is particularly valuable for Financial Services. These organizations often want to develop in the cloud to move fast but must deploy on-premises for compliance. If your governance is tied to a specific cloud, you have to rewrite everything when you move to production. With sovereign infrastructure, you develop once and deploy anywhere. Your governance code looks the same in AWS as it does in your private data center.

3. Uncompromised Trust

In industries like healthcare and defense, trust is the product. Patients and citizens are asking, "Where does the AI run? Who can see my data?" If your answer is, "It's in the cloud and we trust the vendor," that is no longer acceptable. Being able to prove—architecturally—that no telemetry leaves your environment is a massive competitive differentiator.

The Strategic Choice

We are early in this market. Most enterprises are still in the "Cloud-First" mindset, prioritizing convenience over control. But the organizations that are thinking ahead are asking a different question: "Are we building on infrastructure that gives us options, or are we making decisions now that eliminate sovereignty as a possibility later?".

The narrative has long been that you have to choose. You can have cutting-edge AI in the cloud, or you can have control on-premises. That is a false choice. With the right infrastructure—portable, declarative, and self-hosted—you can have both.

You can innovate fast and maintain control. But you have to architect for it now.

Go Deeper

How do you build an AI stack that creates this kind of leverage? We explore the architectural principles, the agent governance crisis, and the future of regulated AI in our latest guide.

]]>

Cilium + Traefik: The Superhighway and Traffic Cop Pattern for Multi-Cloud Networking

Immánuel Fodor — Tue, 02 Dec 2025 22:57:48 GMT

It's 3 AM. You're the on-call engineer, and your globally distributed payment API is timing out on AWS, but running fine on Azure. The API is spread across multiple clouds for high availability and low latency. You pull up three different dashboards—AWS CloudWatch, Azure Monitor, and your homegrown Grafana—trying to correlate logs across environments.

After an hour of intense archaeology, bug hunting, documentation review, and support forum reading, you discover the culprit: AWS ALB's default timeout of 60 seconds versus Azure App Gateway's 90-second default is causing long-running transactions to time out. It's the same application code, different cloud plumbing, and production is still down.

Different provider settings are the hidden tax of multi-cloud architecture.

We tell ourselves, "Kubernetes abstracts the infrastructure." That's true for compute. But for networking? Kubernetes gives you portability at Layer 7 while leaving you stranded in the quicksand of Layers 3 and 4. The networking rules in EKS differ from those in AKS. Load balancers speak different dialects. At scale, your networking configuration compounds with every new service until troubleshooting feels like defusing a bomb in the dark.

It's like driving across Europe in the 1970s. Every border crossing means new road signs, new traffic laws, and a new currency to fumble with.

But there's a better way. A new architectural pattern is emerging that actually delivers on Kubernetes' portability promise. It combines the kernel-level performance of Cilium (the Superhighway) with the application-layer intelligence of Traefik (the Traffic Cop).

For platform engineers tired of maintaining three different networking stacks, here's why this combo is becoming the new standard.

The Legacy Bottleneck: When Every Packet Hits a Red Light

Let's be honest about what we're replacing.

Traditional Kubernetes networking routes traffic through Linux firewall tools like iptables. This technology, designed in the 1990s, was built for firewalls, not for orchestrating thousands of microservices. Every packet that enters a node triggers a linear scan through potentially thousands of rules. The more services and rules you add, the slower every packet gets processed, and the more difficult it becomes to debug.

Picture a city where every intersection has a stop sign and a traffic cop manually checking every car's paperwork. That's iptables. It works fine for 10 cars (services). It becomes complex at 100 rules, and a maintenance headache at 1,000+.

The symptoms are familiar:

CPU spikes during deployment rollouts as firewall chains rebuild due to the rapid creation and destruction of containers.
Unpredictable latency as rule chains grow longer.
Debugging that requires deep Linux kernel knowledge.
Connection tracking tables that overflow under load.

The Superhighway: How Cilium Rewrites the Rules

The breakthrough came from eBPF (Extended Berkeley Packet Filter). Cilium is its best implementation for Kubernetes networking.

Instead of sequential rule checking, Cilium uses hash maps for instant lookups. Crucially, instead of context-switching between user space and kernel space, eBPF programs run directly in the kernel. It's the difference between stopping at every intersection versus taking a grade-separated freeway at constant speed.

The Real Shift: Cloud Providers Are Standardizing on Cilium

This is what changes the game for multi-cloud architecture: for the first time, the networking data plane is becoming identical across clouds.

Azure: "Azure CNI Powered by Cilium" is now the recommended option for AKS
Google Cloud: "GKE Dataplane V2" is Cilium under the hood
AWS: Cilium is the de facto choice for performance-conscious EKS deployments

The road surface is the same whether you're running in Virginia, Dublin, or Singapore. This means:

Predictable latency: Sub-millisecond service mesh overhead regardless of cloud
Consistent observability: Hubble flow logs look the same everywhere
Portable NetworkPolicies: Your security rules work identically across environments
No more cloud-specific tuning: The kernel-level plumbing just works

The Traffic Cop: Why Speed Alone Isn't Enough

A frictionless highway is powerful, but speed without direction is just a fast way to crash. You need intelligent control at the application layer. You need someone at the exit ramp deciding which requests go where, enforcing speed limits, and blocking bad actors.

You need Traefik.

The Multi-Cloud Load Balancer Problem

Most teams hit multi-cloud and immediately face this choice:

Use each cloud's native load balancer (AWS ALB, Azure App Gateway, GCP Load Balancer, etc.)
Deploy and maintain your own Ingress Controller

Option 1 seems easier until you realize you're hiring a different traffic cop for every city ... and they don't speak the same language or follow the same rules:

AWS ALB:

annotations:
  alb.ingress.kubernetes.io/scheme: internet-facing
  alb.ingress.kubernetes.io/target-type: ip
  alb.ingress.kubernetes.io/healthcheck-path: /health

Azure App Gateway:

annotations:
  appgw.ingress.kubernetes.io/health-probe-path: /health
  appgw.ingress.kubernetes.io/request-timeout: "90"

GCP Load Balancer:

annotations:
  cloud.google.com/neg: '{"ingress": true}'
  cloud.google.com/backend-config: '{"default": "backend-config"}'

Same intention. Three different languages. Three different configuration systems. Three different failure modes to debug at 3 AM.

Traefik: Bring Your Own Traffic Cop

When you deploy Traefik as your Ingress Controller across all clouds, you establish universal laws:

One Configuration Language

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: payment-api
spec:
  routes:
  - match: Host(`api.example.com`) && PathPrefix(`/payments`)
    kind: Rule
    services:
    - name: payment-svc
      port: 8080
    middlewares:
    - name: rate-limit
    - name: auth-oidc

This YAML works identically on AWS, Azure, and GCP. Change it once, deploy everywhere.

Portable Intelligence

Rate-limiting rules that follow your app across clouds
OIDC/OAuth flows that don't require cloud-specific IAM translation when using a central Identity Provider (IdP)
Circuit breakers that understand your application's failure modes
A/B testing and canary deployments with consistent behavior

Developer Velocity
Your developers define routing via standard Kubernetes resources. They don't need to know whether they're deploying to EKS or AKS. The platform team maintains a single Traefik configuration, rather than three cloud-specific load balancer setups.

Why Not Cilium Ingress?

Cilium does offer a basic Ingress controller built on the eBPF data plane. However, ingress is not its primary focus, and it typically offers the bare minimum necessary for Layer 7 entry.

For complex, multi-cloud environments, Traefik is better suited as the Traffic Cop because it provides an application-centric control plane. Traefik delivers advanced Layer 7 features—like portable middleware, integrated OIDC/OAuth, A/B testing, and comprehensive circuit breakers—that are central to managing a complex application portfolio at scale.

The "Better Together" Synergy

The architecture gets elegant when you look at how these two layers interact.

Traefik operates at Layer 7 (HTTP/gRPC), making intelligent routing decisions based on headers, paths, and application-level logic. Cilium operates at Layer 3 and 4, handling the actual packet forwarding with kernel-level efficiency.

The handoff is seamless:

External request hits your LoadBalancer Service (backed by a simple cloud LB)
Traefik receives the request, processes middleware, terminates TLS, and selects a backend
Traefik hands off to a Kubernetes Service (just an IP:port)
Cilium bypasses iptables entirely, using eBPF to route directly to the pod
Response follows the same optimized path back

The result: You get Layer 7 intelligence without Layer 3/4 bottlenecks. No tradeoffs.

Full-Stack Observability: The "X-Ray" Effect

This multi-layer observability is the key to platform team sanity.

If you've ever debugged a 502 Bad Gateway error, you know the frustration:

Traefik logs: "Request sent to backend, connection refused".
Application logs: "I never received a request".
Network monitoring: "I see packets moving, but no idea what they contain".

Three different monitoring systems, three different stories, zero root cause. This is the observability air gap that kills productivity.

The breakthrough: Distributed tracing stitched across network layers. Traefik and Cilium solve multi-layer observability together through a deceptively simple mechanism:

Traefik mints (or reuses) the trace ID: When a request hits your edge, Traefik generates a unique TraceParent header (OpenTelemetry W3C standard), or reuses one if it already exists. Think of this as stamping a license plate on the car before it enters the highway.
Cilium carries the context: As the request flows through the network, Cilium's Hubble observability engine reads HTTP headers at the kernel level, without slowing the packet down. It sees the same trace ID and logs the network path, latency, and any TCP-level failures.
One timeline, complete picture: In your observability backend (Grafana Tempo, Jaeger, Datadog), you search for that trace ID and get:

Edge ingress at Traefik (Layer 7)
Network path through Cilium with exact latency and packet drops (Layer 3/4)
Application processing time (Layer 7)

Real-World Example
A developer reports "users in Tokyo are seeing 2-second response times". You pull up the trace ID:

Traefik reports: Request took 2100ms total, 2000ms waiting for backend.
Hubble reports: TCP connection from Tokyo pod exists to both Tokyo database AND us-east database.
Application reports: Query took 50ms locally, 1950ms on cross-region query.
Root cause: Misconfigured service selector routing 10% of Tokyo traffic across continents.

Same debugging workflow whether you're troubleshooting EKS, AKS, or GKE.

The Technical Detail (Read This Before Deploying)
By default, Cilium prioritizes maximum speed, so it acts like a courier who reads the address on the box (IP/Port) without opening it. This makes it incredibly fast, but it means it can't see the specific "Trace ID" stamped inside the package headers.

To connect the dots between Traefik and the network, you must explicitly enable "Deep Packet Inspection" for your specific services. This is done using a CiliumNetworkPolicy.

Think of this as flipping a switch that says, "For this specific application, it's okay to peek inside the envelope to read the tracking number." It is a simple, one-time configuration per namespace, but without it, your observability layers won't communicate.

The Honest Complexity Tradeoff

Let's be clear: This isn't the right stack for everyone.

You probably don't need this if:

You're running on a single cloud with no plans to expand.
You have 10 microservices and 3 engineers.
Your cloud provider's defaults work fine for your scale.

This stack pays off when:

You're running production workloads across 2+ clouds.
You're managing 50+ microservices with complex routing needs.
You need consistent security policies and observability across environments.
Your platform team is tired of maintaining cloud-specific configurations.
Developer velocity matters more than "using cloud-native defaults".

The upfront complexity is real. You're deploying and managing Cilium and Traefik instead of clicking "enable" in a cloud console. But the payoff is a platform that your team masters once and runs everywhere.

The Bottom Line: Platform Team Sanity as a Service

Multi-cloud isn't going away. Whether you're pursuing vendor negotiation leverage, data sovereignty requirements, or genuinely distributed architecture, you need networking that works the same everywhere.

The old approach (maintaining separate networking configurations per cloud) doesn't scale. Your platform team becomes translators, converting concepts between AWS-speak, Azure-speak, and GCP-speak. Your developers slow down, waiting for platform teams to implement the same feature three different ways.

The Cilium + Traefik stack inverts this:

Cilium: Universal, high-performance data plane (the cloud providers are standardizing on it anyway)
Traefik: Portable, developer-friendly control plane (you own it, not the cloud vendor)

You get predictable performance, consistent debugging, and configurations that travel with your applications. Your platform team maintains one networking stack, not three. Your developers deploy with confidence, knowing the rules are the same everywhere.

The roads are open. The traffic is moving. And for the first time, the highway and the traffic cop speak the same language. No matter which cloud you're driving through.

Ready to build a truly portable Kubernetes platform?
Start with Cilium for your CNI and Traefik for your Ingress Controller. You'll thank yourself the next time production spikes.

]]>

The Hub-and-Spoke Ingress Pattern: Unifying EKS, ECS, and EC2 at Scale

Zaid Albirawi — Sun, 30 Nov 2025 18:27:52 GMT

Large-scale AWS architectures rarely maintain homogeneity over time. They evolve through eras.

A service might begin life on Amazon EC2 because the team needed full control over the OS kernel. Years later, a new initiative embraces Amazon EKS for microservices. Meanwhile, other teams deploy onto Amazon ECS or AWS Fargate because the bursty, event-driven nature of their workloads demands it.

Individually, these are sound technical decisions. EC2 offers stability; EKS offers orchestration; ECS offers simplicity. The problem does not lie with the platforms themselves, but with how they are exposed to the outside world.

As these compute "islands" grow, they create a fractured edge. EKS uses Ingress controllers; EC2 relies on ALBs/NLBs with custom proxy logic; ECS uses ALBs mapped 1-to-1 with services. The result is operational incoherence: inconsistent authentication, fragmented routing rules, and a nightmare for anyone trying to debug a request that spans multiple environments.

In our latest technical AWS builders-style ebook, Unifying Ingress Across Distributed AWS Compute Environments, we detail an architectural pattern that solves this: the Hub-and-Spoke Ingress Model.

Here is a deep dive into how it works and how to implement it using the Traefik AWS Elastic Provider.

The Anatomy of Fragmentation

Organizations do not set out to build fragmented systems; fragmentation is a side effect of growth.

When you view these platforms from the inside, they make sense. But from the application edge—where users and clients interact with your system—the cracks begin to show.

Identity Fragmentation: EKS might validate JWTs via middleware, while EC2 relies on a totally different auth proxy.
Routing Complexity: Migrating a service from EC2 to ECS often requires DNS cutovers because there is no single routing plane that spans both.
Security Risk: To let a central ingress talk to backend services across VPCs, teams often widen Security Group rules or create permissive IAM roles, expanding the blast radius.

To solve this, we need an architecture that can see the entire system from the outside while interacting with each environment from the inside—without breaking isolation boundaries.

The Hub-and-Spoke Architecture

The solution is to decouple the global entry point from the local service discovery. We call this the Hub-and-Spoke model.

The Unified Ingress as the Hub

For the hub, you need a "Unified Ingress," a Traefik instance running on your primary compute platform (often EKS). It is responsible for high-level concerns: terminating TLS, validating identity (OIDC/JWT), applying global rate limits, and handling the routing logic.

The Compute Spokes

The spokes in our Hub-and-Spoke Model are lightweight Traefik instances deployed in remote environments—on EC2 instances, in ECS clusters, or in other EKS clusters.

Crucially, the Unified Ingress does not query the AWS API to find services. Instead, it polls the spokes. Each spoke inspects its local environment (via a Docker socket, the ECS API, or local tags) and exposes a sanitized list of available services via an internal API endpoint.

This separation of concerns yields a modular ingress fabric:

The spokes are authoritative for their own environments.
The Unified Ingress hub is authoritative for routing.
Neither requires deep knowledge of the other's internal workings.

Why Polling? (The Security Argument)

In modern cloud-native development, "polling" is often considered a dirty word compared to event-driven architectures. However, in a multi-account, multi-VPC AWS environment, polling is actually the superior architectural choice for three reasons.

1. Eliminating Cross-Account IAM Complexity

If you use a centralized ingress controller that discovers services by querying the AWS API directly, that controller needs "God Mode" permissions. It needs to AssumeRole into every other AWS account to list instances and tasks. This creates a massive security risk—if the Ingress is compromised, the attacker can map your entire cloud estate.

With the Hub-and-Spoke Model, the Unified Ingress never touches AWS APIs outside its own account. It simply sends an HTTP request to the Spoke. The Spoke needs local permissions, but the Unified Ingress needs none. This aligns strictly with least-privilege principles.

2. Predictable Failure Modes

In distributed systems, simple failure modes are valuable. If a spoke goes down, the Unified Ingress stops polling it, times out, and removes those routes. When the spoke recovers, the routes return on the next interval. There is no "stale state" hidden in an event queue or a stream processor that stopped silently.

3. Tight Network Boundaries

Event-driven push models often require bidirectional connectivity. Polling is unidirectional. The Unified Ingress calls the spoke. The spoke requires no inbound rules beyond a single HTTPS listener on a specific port, accessible only from the Unified Ingress’s Security Group.

Solving the Identity Crisis

One of the most painful aspects of heterogeneous infrastructure is unifying authentication.

In a fragmented system, EKS might handle auth via an Ingress Controller, while legacy EC2 apps handle it in code. This makes it nearly impossible to enforce policies like "Only users with the admin claim in their JWT can access the /admin path on any service."

Because all external traffic flows through the Ingress, authentication becomes centralized by design.

The Unified Ingress receives the request.
The Identity Middleware validates the JWT against your OIDC provider (Cognito, Okta, etc.).
The Unified Ingress extracts claims (Group, Tenant, Role).
The Unified Ingress uses those claims to make routing decisions before forwarding the packet to the EKS, EC2, or ECS instances.

This transforms identity from a platform-specific headache into a consistent network primitive.

Cross-VPC Isolation

Many AWS environments segment workloads across multiple VPCs for compliance or to reduce the blast radius. Without a unified ingress, teams often resort to complex VPC Peering meshes or Transit Gateways with overly permissive security groups to allow east-west traffic.

The Hub-and-Spoke Model respects AWS’ "segmentation first" philosophy.

The Unified Ingress acts as the only cross-VPC touchpoint. Workloads in VPC A (ECS) never talk directly to Workloads in VPC B (EC2). They don't even need to know that the others exist. Security groups open only a narrow set of ports from the Unified Ingress’ CIDR range to the spoke’s Traefik instance.

This allows you to maintain strict network isolation while presenting a unified API surface to the internet.

The Traefik AWS Elastic Provider

While the Hub-and-Spoke is an architectural pattern, implementing it from scratch requires significant engineering. You need to build the spokes, standardize the metadata format, and build the reconciliation logic in the Unified Ingress.

This is why we built the Traefik AWS Elastic Provider.

It acts as the concrete realization of this architecture. It operationalizes the model by providing:

Unified Discovery Fabric: Spokes automatically enumerate local services (EC2 tags, ECS tasks, Kubernetes services) and normalize the data.
Resilience Logic: The provider handles the polling intervals, timeouts, and route eviction logic automatically.
Incremental Adoption: You can add a spoke to one ECS cluster today without touching your EKS or EC2 environments. The architecture evolves as you do.

When to Use This Pattern

This approach is not for everyone. If you are running a single EKS cluster in a single VPC, the AWS Load Balancer Controller is likely sufficient.

However, the Hub-and-Spoke Model offers substantial operational leverage for environments that exhibit:

Multiple Compute Platforms: A mix of EC2, ECS, and EKS.
Multi-VPC/Multi-Account: Strict segmentation requirements.
Migration Projects: Moving monoliths to microservices where traffic must be shifted gradually between platforms based on identity or weights.
High Governance: Environments requiring centralized audit trails for ingress traffic.

Conclusion: Consistency at the Edge

AWS will continue to evolve. New compute services will launch; existing ones will change. If your ingress strategy is tightly coupled to the underlying compute platform (e.g., "We use ALBs because we use ECS"), you will always be playing catch-up.

The Hub-and-Spoke architecture offers a principled path forward: a consistent edge atop an inherently diverse landscape. It allows you to choose the right compute tool for the job—EC2 for stateful legacy apps, ECS for batch jobs, EKS for microservices—without fragmenting the user experience.

It enables organizations to retain the advantages of heterogeneity while presenting a unified, reliable, and secure face to the world.

Read the Builders Guide

We have compiled a detailed AWS builders-style architectural guide including sequence diagrams for authentication flows, cross-VPC security group configurations, and failure recovery scenarios. Click the promo below to get your free copy.

]]>

120 Days Until Ingress NGINX Dies: Traefik is the Only True Drop-in Replacement

Emile Vauge — Wed, 19 Nov 2025 00:35:24 GMT

The Kubernetes community was shaken on November 12, 2025, when maintainers of the Ingress NGINX project announced: Ingress NGINX Controller will be retired in March 2026. After March 2026, there will be no releases, no bug fixes, and critically, no security updates.

Organizations now have just 4 months to migrate away from one of the most widely deployed ingress controllers in the ecosystem. This isn't theoretical—if you're running production workloads on Ingress NGINX after March 2026, you're running unmaintained software with potential security vulnerabilities. Historically, there have been critical vulnerabilities discovered in this project, including the #IngressNightmare vulnerabilities that demonstrated how attackers could compromise entire clusters through malicious configuration injection.

The timeline is brutal and many organizations are already behind schedule before they've started.

Why Traefik is Your Only Realistic Option

Earlier this year, we published a deep-dive into the #IngressNightmare situation and how to transition from Ingress NGINX to Traefik. In short, we added an Ingress NGINX compatibility layer to Traefik, introducing native support for many custom annotations and creating the only drop-in replacement for Ingress NGINX in the industry.

Here's the reality: every other ingress controller requires you to rewrite your configurations. F5 NGINX Ingress, HAProxy Ingress, cloud provider solutions—e.g., AKS, AWS, etc.—they all provide migration guides, annotation mappings, or promote a Gateway API transition, not migration compatibility. You'll need to manually convert every custom annotation, test every configuration, and essentially rebuild your ingress layer from scratch.

Traefik's NGINX Provider works differently. Most used nginx.ingress.kubernetes.io annotations are handled natively by Traefik. Your Ingress objects work unchanged. This is a huge differentiator compared to other solutions.

But compatibility alone isn't enough in this crisis. Traefik offers three critical advantages that make it the only realistic choice:

Security by Design: The #IngressNightmare vulnerabilities revealed fundamental architectural flaws in NGINX-based solutions. Traefik's architectural choices—Go over C/C++, static linking, structured parsing over templating—prevent the memory safety and configuration injection attacks that plague NGINX controllers.

Broad Ecosystem Adoption: With 3.4+ billion downloads, 58,000+ GitHub stars, and 900+ contributors, Traefik isn't experimental—it's battle-tested across startups and enterprises. Unlike solutions adapted for Kubernetes, Traefik was designed specifically for dynamic, cloud-native environments.

Gateway API Leadership: As a key contributor to Gateway API specification with full v1.4 support in Traefik v3.6, you're positioning for the future of Kubernetes networking, not just solving today's crisis. While others treat Gateway API as experimental, Traefik has made it core.

When you're facing a 120-day deadline with no margin for error, Traefik emerges as the obvious choice—seamlessly compatible with your existing deployments while positioning you for the future of cloud-native networking.

Stop the Bleeding First, Then Modernize

The retirement of Ingress NGINX is an urgent, time-sensitive problem demanding a structured migration strategy. It can be tempting to directly think of Gateway API as the obvious solution. Indeed, this new Kubernetes specification handles much more than the Ingress specification, and we are not only a big fan at Traefik Labs, but we are leading this standard as the future of Kubernetes networking.

However, Gateway API requires a complete migration of all your ingress resources to a new set of resources: Gateway, GatewayClass, HTTPRoute, TLSRoute, etc. And this is clearly not something you can (or should) do in a short period of time.

Trying to simultaneously solve the immediate crisis of a looming deprecation deadline and the long-term goal of ingress modernization is a mistake. The prudent, risk-averse approach separates these concerns into distinct, manageable phases.

Phase 1: Ingress NGINX Decommissioning. Get off Ingress NGINX before security updates stop. Migrate to Traefik with the NGINX Provider for drop-in compatibility. Maintain existing workflows and ensure production stability through the deadline.

Phase 2: Gateway API Modernization. Plan Gateway API adoption on your timeline, not under deadline pressure. Assess readiness during normal sprint cycles, test thoroughly, implement when business conditions are optimal.

Your immediate problem is Ingress NGINX retirement, not Ingress API modernization. Solve one crisis at a time.

How to Decommission Ingress NGINX, in a Nutshell

First, let’s dig deeper into the basics with a nominal use case. Let’s say:

You operate a Kubernetes cluster
There is a running ingress-nginx controller instance
A Kubernetes IngressClass named nginx with the controller value set to k8s.io/ingress-nginx and Kubernetes Ingresses have an ingressClassName set to nginx
And a Kubernetes service of type LoadBalancer exists exposing the ingress-nginx controller instance on port 80 and 443

You have this ingress with TLS and basic authentication deployed:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: ingress-with-basicauth
  namespace: default
  annotations:
    # Configure basic authentication for the Ingress
    nginx.ingress.kubernetes.io/auth-type: "basic"
    nginx.ingress.kubernetes.io/auth-secret-type: "auth-file"
    nginx.ingress.kubernetes.io/auth-secret: "default/basic-auth"
    nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"
    # SSL Redirect
    nginx.ingress.kubernetes.io/ssl-redirect: "true"

spec:
  ingressClassName: nginx
  rules:
    - host: myappdomain
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
  tls:
    - hosts:
        - myappdomain
      secretName: external-certs

---
kind: Secret
apiVersion: v1
metadata:
  name: basic-auth
  namespace: default
type: Opaque
data:
  # user:password
  auth: dXNlcjp7U0hBfVc2cGg1TW01UHo4R2dpVUxiUGd6RzM3bWo5Zz0=

Deploy Traefik alongside your existing Ingress NGINX setup. Traefik is installed in `ClusterIP` mode to be able to test the routing.

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ helm upgrade --install traefik traefik/traefik --namespace traefik \
  --create-namespace \
  --set="image.tag=v3.6.2" \
  --set="logs.general.level=DEBUG" \
  --set="service.type=ClusterIP" \
  --set="additionalArguments={--providers.kubernetesIngressNGINX}"

Check that the routing is working with a port-forward

$ kubectl port-forward -n traefik services/traefik 9000:80 9443:443
Forwarding from 127.0.0.1:9000 -> 8000
Forwarding from [::1]:9000 -> 8000
Forwarding from 127.0.0.1:9443 -> 8443
Forwarding from [::1]:9443 -> 8443

$ curl https://myappdomain:9443
401 Unauthorized

$ curl -u user:password https://myappdomain:9443
Hostname: whoami-b85fc56b4-5pvcv
...

Depending on how your Ingress NGINX controller has been deployed, you might also need to recreate the nginx IngressClass, if it’s removed when Nginx is uninstalled.

Then, you can remove the Ingress NGINX deployment, update the Traefik deployment to change the Service type to LoadBalancer, update the DNS record, and check that Traefik serves your untouched ingress resources 🙂

$ helm upgrade --install traefik traefik/traefik --namespace traefik \
  --create-namespace \
  --set="image.tag=v3.6.2" \
  --set="logs.general.level=DEBUG" \
  --set="service.type=LoadBalancer" \
  --set="additionalArguments={--providers.kubernetesIngressNGINX}"

$ curl -u user:password https://myappdomain              
Hostname: whoami-b85fc56b4-q6427
...

It’s as simple as that. nginx.ingress.kubernetes.io annotations are handled natively by Traefik and your ingresses stay unchanged.

For large Kubernetes deployments with numerous workloads and Ingress resources, we recommend a progressive migration. Deploy new clusters with Traefik, and then gradually transition your existing workloads and associated Ingress configurations to these new clusters. This approach allows for thorough validation, provides an easy rollback mechanism if necessary, and ensures the entire migration process remains manageable.

The Traefik Labs team possesses extensive experience with various Kubernetes deployments and is well-equipped to provide commercial assistance, helping you plan and execute your migration efficiently and seamlessly. Feel free to reach out to us if you need help.

Time to Execute

The retirement of Ingress NGINX is serious, but it's not the Kubernetes apocalypse. With 120 days remaining and Traefik's proven compatibility layer, you have enough time to migrate safely if you act now.

Don't let deadline pressure force you into premature Gateway API adoption or manual configuration rewrites. Traefik's NGINX Provider offers the most realistic path forward with immediate compatibility, proven track record, and clear upgrade path to Gateway API when you're ready.

Our current implementation focuses on supporting the most commonly used Ingress NGINX custom annotations, covering 80% of real-world usage patterns. If you need additional annotation support, we can implement it. Feel free to open GitHub issues in Traefik's repository to tell us your needs. Better yet, contribute directly—adding new annotation support is straightforward and helps the entire community.

Our goal is simple: make this crisis a non-event. Every organization should be able to migrate off Ingress NGINX safely and quickly, without disrupting their operations or forcing architectural decisions under pressure.

Essential Resources

Traefik NGINX Provider Documentation
Download Traefik 3.6: Docker Hub
Documentation: Complete Documentation
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The Feature You Didn't Know You Needed: Multi-Layer Routing in Traefik

Immánuel Fodor — Mon, 10 Nov 2025 22:15:41 GMT

Here's a question: how do you route traffic when the routing decision depends on information that doesn't exist in the original request?

Maybe you need to authenticate first, then route based on the user's role. Or check a feature flag service to decide whether someone gets the new microservices or the old monolith. Or look up a customer's subscription tier to route enterprise users to stable infrastructure while free users are directed to your canary.

The problem is that routing decisions are typically based on the information present in the request when it arrives—the path, the headers, the host, or other similar information. Even with Traefik’s flexible routing expression system, if the information you need for routing resides elsewhere (an auth service, a feature flag system, a database), you're stuck with workarounds.

Most teams end up doing one of these things:

Duplicating logic—Every microservice calls the auth service, parses JWTs, or checks feature flags independently
Building a routing proxy—A separate service sits in front of your main routing logic, makes the decision, adds routing headers (and adds latency)
Complex configuration—Giant routing files that try to handle every edge case and break when requirements change
Giving up—Route everyone to the same backend and handle the logic in application code

None of these are great. They're slow, brittle, or just push the problem somewhere else.

Traefik has just released multi-layer routing, and it solves this entire class of problems. Let me show you how.

The Problem: Risk Management Meets Routing Logic

Let's focus on one scenario that nearly every engineering team faces: safely deploying a new backend version.

You've built a new version of your API service. It's faster, cleaner, better—but it's also risky. Your enterprise customers have stricter SLAs. Breaking their production workflows would be catastrophic. But your free-tier users? They're well-suited for a canary-based deployment workflow, as free tiers typically have lesser SLA commitments and a lower risk of immediate business impact.

You need to route requests to https://api.example.com/users to different backends based on customer tier. Enterprise goes to stable, free tier goes to canary.

Your current options are all flawed.

The "naive" approach: Parse subscription level in every microservice. Now every service needs to be aware of customer tiers, call your billing system, handle caching, and deal with failures. You've just coupled your entire architecture to your billing logic.

The "proxy" approach: Build an external "customer routing service" that sits in front of your main routing logic. It checks the customer tier and adds custom headers to give hints to routing. Congratulations, you've added 30-500ms latency to every request and created a critical single point of failure. The added latency may even vary depending on your custom logic, database lookup speeds, distributed infrastructure, and many other factors.

The "manual config" approach: Update your routing configuration every time a customer upgrades or downgrades. This doesn't scale, it's error-prone, and you'll eventually route an enterprise customer to your canary at 2 AM by mistake.

The fundamental problem: Routing decisions are based on the original request. If your auth service knows the customer tier, even Traefik’s intelligent routing system can't use that information for routing—it's trapped in your auth system's response.

How Multi-Layer Routing Solves the Problem

Traefik's new multi-layer routing introduces hierarchical relationships between routers. Parent routers process requests with attached middleware, enriching them with additional context. Child routers then make routing decisions based on that enriched request.

Here’s the request flow:

Diagram of Multi-Layer Routing in Traefik

The router hierarchy has three types:

Root routers sit at the top with no parents, attached to entryPoints. They define tls and observability configuration. They can either have children (making them parent routers) or directly route to a service (standalone routers). Middleware is a key piece in request enrichment—root routers typically apply shared middleware that modifies requests for downstream routing decisions.

Intermediate routers reference their parents via parentRefs and can have their own children. They also cannot define a service and inherit entryPoints, tls, and observability from their root. Both intermediate and leaf routers can define additional middleware beyond what the parent applies. Importantly, they can't be called directly—requests must flow through their parent router, ensuring you can't circumvent authentication or other parent-level logic.

Leaf routers reference their parents via parentRefs and must define a service. They also inherit configuration from their root router and can apply their own middleware.

The magic is progressive request enrichment: each layer adds context that subsequent layers use for increasingly specific routing decisions. A key benefit is that you don't have to repeat the previous layers' matchers—they all add up as you traverse the routing tree. Authentication happens once at the parent, and routing happens at the child based on the enriched result.

Show, Don't Tell: Progressive Rollout Configuration

Let's solve that customer tier routing problem. Here's the complete configuration:

## Routing configuration
http:
  routers:
    # Root router - matches all API requests and applies auth
    api-parent:
      rule: "Host(`api.example.com`) && PathPrefix(`/`)"
      middlewares:
        - auth-with-tier
      entryPoints:
        - websecure
      tls: {}
      # No service defined - this is a parent router

    # Leaf router - routes enterprise customers to stable
    api-enterprise:
      rule: "HeaderRegexp(`X-Customer-Tier`, `(enterprise\|business)`)"
      service: stable-backend
      parentRefs:
        - api-parent

    # Leaf router - routes free tier to canary
    api-free:
      rule: "Header(`X-Customer-Tier`, `free`)"
      service: new-version
      parentRefs:
        - api-parent

  middlewares:
    # ForwardAuth validates request and enriches it with customer tier
    auth-with-tier:
      forwardAuth:
        address: "http://auth-service:8080/validate"
        authResponseHeaders:
          - X-Customer-Tier
          - X-Customer-Id
          - X-User-Email

  services:
    stable-backend:
      loadBalancer:
        servers:
          - url: "http://api-v1-stable:8080"

    new-version:
      loadBalancer:
        servers:
          - url: "http://api-v2-canary:8080"

What happens when a request arrives:

Request to https://api.example.com/users hits the websecure entrypoint
api-parent router matches based on Host and PathPrefix
auth-with-tier middleware executes, forwarding the request to http://auth-service:8080/validate
Auth service validates the JWT, looks up the customer’s subscription, and returns headers:
- X-Customer-Tier: enterprise
- X-Customer-Id: cust_abc123
- X-User-Email: admin@bigcorp.com
Traefik adds these headers to the request and evaluates child routers
api-enterprise router matches because X-Customer-Tier matches enterprise
Request is forwarded to stable-backend at http://api-v1-stable:8080
When a free-tier user makes the same request:
Steps 1-4 are identical, but the auth service returns X-Customer-Tier: free
api-free router matches instead
Request is forwarded to the new-version service at http://api-v2-canary:8080

The contrast: Without multi-layer routing, you'd need to either check customer tier in every microservice (slow, duplicated), build an external routing proxy (latency, complexity), or somehow encode customer tier in URLs or domains (terrible UX). Now it's declarative: authenticate once, enrich the request, and route based on the enriched context.

What Else Is Possible

Multi-layer routing unlocks patterns that were previously painful or impossible to implement cleanly. We show two additional examples below.

Authentication-Based Routing

The problem: You have admin endpoints and user endpoints, but you don't want separate URL paths. Admins calling /api/reports should see all reports. Regular users calling the same endpoint should only see their own reports. The user's role only exists after authentication.

The solution: The parent router authenticates all /api requests using the ForwardAuth middleware. The auth service validates credentials and returns X-User-Role header (either admin or user). Child routers match on that header: one routes admin requests to the admin-backend with full data access, another routes user requests to the user-backend with restricted access.

Key config pattern:

# Parent applies auth, no service
api-parent:
  rule: "PathPrefix(`/api`)"
  middlewares: [auth-middleware]

# Children route based on enriched headers
api-admin:
  rule: "Header(`X-User-Role`, `admin`)"
  service: admin-service
  parentRefs: [api-parent]

api-user:
  rule: "Header(`X-User-Role`, `user`)"  
  service: user-service
  parentRefs: [api-parent]

Use case: Multi-tenant SaaS platforms, admin panels with role-based access, internal tools with permission-based routing.

Feature Flag-Based Migration

The problem: You're migrating from a monolith to microservices. Some customers are on the new architecture, others on the old. Feature flag decisions reside in an external service (LaunchDarkly, Unleash, custom system). You need Traefik to route based on those flags without calling the feature flag service from every microservice.

The solution: Parent router calls your feature flag service via ForwardAuth or a custom plugin. The service evaluates flags for the user and returns X-Feature-Microservices: enabled or X-Feature-Microservices: disabled. Child routers match on that header and route to either the new microservices or the legacy monolith. One feature flag check, unlimited routing decisions downstream.

Key config pattern:

# Parent checks feature flags
migration-parent:
  rule: "Host(`app.example.com`)"
  middlewares: [feature-flag-checker]

# Route to microservices if enabled
microservices-route:
  rule: "Header(`X-Feature-Microservices`, `enabled`)"
  service: new-microservices
  parentRefs: [migration-parent]

# Route to monolith otherwise
monolith-route:
  rule: "Header(`X-Feature-Microservices`, `disabled`)"
  service: legacy-monolith
  parentRefs: [migration-parent]

Use case: Progressive monolith decomposition, gradual infrastructure migrations, per-customer feature rollouts, A/B testing at the gateway level without application changes.

When to Use This

Use multi-layer routing when:

✅ You're enriching requests (authentication, feature flags, customer context, or you name it) and need to route based on that enriched data

✅ You have shared middleware that applies broadly, but different routing logic after that middleware executes

✅ Your routing rules are becoming unmaintainable—complex combinations of conditions that would be clearer as hierarchical layers

✅ You're making the same external call (auth service, feature flags) across multiple routes and want to do it once

Skip multi-layer routing when:

⏩ You have simple routing needs—basic path or host matching doesn't need this complexity

⏩ You need Docker labels or standard Kubernetes Ingress—multi-layer routing at the time of writing only works with Kubernetes IngressRoute CRD, File providers (YAML/TOML), and KV stores (Consul, etcd, Redis, ZooKeeper).

Important consideration: Multi-layer routing changes your debugging model. You're now tracing requests through a hierarchy, rather than a single router. Enable Traefik's OpenTelemetry-based observability and distributed tracing—you need visibility into which routers match at each layer and why. Think of it like debugging microservices instead of a monolith: a little more moving parts, but clearer separation of concerns.

Performance note: Each layer adds minimal overhead (microseconds for rule evaluation). The middleware execution (ForwardAuth calls, etc.) is where latency happens, but you'd be doing those calls anyway—multi-layer routing just lets you do them once instead of repeatedly, and without any hacks that could bite back in the long run.

Get Started

The fastest way to understand multi-layer routing is to try it. The "aha moment" occurs when you realize how naturally complex routing logic can be decomposed into simple layers. Authentication, then routing. Context extraction, then decisions. One job per layer, rather than cramming everything into brittle, single-line rules or scattering it across multiple services.

Take inspiration from the examples above, make adjustments to match your environment, and deploy it to a test infrastructure. Make requests with different authentication credentials and observe Traefik routing them to different backends based on enriched request properties.

For complete documentation, additional examples, and provider-specific configuration formats, see the official Traefik multi-layer routing reference.

If you're still duplicating auth middleware across routes, building external routing services just to make decisions, parsing customer data in every microservice, or maintaining complex single-line routing rules with multiple conditions, there's a better way now.

The feature has just launched. The patterns are still emerging. What will you build with hierarchical routing layers?

Do you have a multi-layer use case in mind that you think is even more interesting than the ones shared above? I’d love to hear it at product@traefik.io!

]]>

Scale Serverless Workloads with Traefik & Knative

Shedrack Akintayo — Mon, 10 Nov 2025 22:02:14 GMT

Modern infrastructure spans diverse workload types—from traditional VMs to cloud-native containers and serverless functions. Managing networking across these environments has traditionally required multiple specialized proxies, increasing complexity and operational overhead.

With Traefik 3.6, we're changing that. The introduction of Knative provider support marks a key milestone: Traefik is now the only proxy that can seamlessly operate at the edge of Cloud Native workloads (containers), traditional workloads (Virtual Machines), and serverless workloads. This unprecedented versatility makes Traefik a truly unique platform capable of handling any kind of workload your infrastructure demands.

Knative, the leading platform for serverless workloads on Kubernetes, provides scale-to-zero capabilities, automatic scaling based on traffic, and sophisticated revision management. The Knative provider in Traefik automatically handles service discovery, scaling events, and traffic routing for these serverless workloads, providing a unified experience across your entire application landscape.

Teams can now use a single proxy solution for their complete infrastructure, dramatically simplifying operations while maintaining the specialized optimizations each workload type requires.

Prerequisites

Before we begin, make sure you have:

Helm 3.x installed
k3d installed for local testing
kubectl configured to access your cluster
Basic familiarity with Kubernetes concepts

Create a k3d Cluster

For this guide, we'll use k3d to create a local Kubernetes cluster with port mappings that allow direct access to services without port-forwarding:

k3d cluster create traefik \
  --port 80:80@loadbalancer \
  --port 443:443@loadbalancer \
  --k3s-arg "--disable=traefik@server:0"

This command creates a cluster with:

Port 80 and 443 mapped to your localhost
The default k3s Traefik installation disabled (we'll install our own)

If you're using a cloud provider instead, you can skip this step and use your existing cluster.

Install Knative Serving

Let's start by installing Knative Serving. We'll be using version 1.19.0, which has been tested with Traefik's Knative provider.

Install the Knative Custom Resource Definitions (CRDs):

kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.0/serving-crds.yaml

Then install the Knative Serving core components:

kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.0/serving-core.yaml

Wait for all Knative pods to be ready:

kubectl get pods -n knative-serving

You should see the activator, autoscaler, controller, and webhook pods all in a 'Running' state.

Now configure Knative to use Traefik as its networking layer:

kubectl patch configmap/config-network \
  -n knative-serving \
  --type merge \
  -p '{"data":{"ingress.class":"traefik.ingress.networking.knative.dev"}}'

Install Traefik as Knative Networking Controller

Now let's deploy Traefik with Knative support enabled using Helm.

Create a values.yaml file that enables the Knative provider:

experimental:
  knative: true

providers:
  knative:
    enabled: true

The configuration enables both the experimental Knative feature flag and activates the Knative provider.

Next, execute the following commands to deploy Traefik in the traefik namespace using the previously described configuration:

helm repo add traefik https://traefik.github.io/charts
helm repo update
kubectl create namespace traefik
helm upgrade --install --version 33.2.0 --namespace traefik \
  traefik traefik/traefik -f values.yaml

Verify that Traefik is running:

kubectl get pods -n traefik

You should see the Traefik pod in a 'Running' state. And voilà! Your Traefik instance is now ready to handle Knative workloads.

Deploy your first Knative Service

Now that our infrastructure is ready, let's deploy a serverless application. We'll use Traefik's whoami service, which provides useful debugging information about the request.

Create a file called whoami.yml:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          env:
            - name: WHOAMI_NAME
              value: "Serverless with Traefik"

Deploy the service:

kubectl apply -f whoami.yml

Watch as Knative creates the necessary resources:

kubectl get ksvc whoami

After a moment, you should see output indicating the service is ready:

Test Your Deployment

Test your application by sending a request with the appropriate Host header:

curl -H "Host: whoami.default.example.com" http://localhost

You should see output showing request details, including the service name and headers that Knative automatically adds.

Name: Serverless with Traefik
Hostname: whoami-00001-deployment-86654cdc46-xvkcc
IP: 127.0.0.1
RemoteAddr: 127.0.0.1:36324
GET / HTTP/1.1
Host: whoami.default.example.com
User-Agent: curl/8.7.1
Accept: */*
Accept-Encoding: gzip
Forwarded: for=10.42.0.1;host=whoami.default.example.com;proto=http, for=10.42.0.6
K-Proxy-Request: activator
X-Forwarded-Host: whoami.default.example.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-784b46c9ff-l8ckj

Congratulations! You've just deployed your first serverless application with Knative and Traefik.

Observe Scale-to-Zero

One of Knative's standout features is scale-to-zero. Watch this behavior in real-time by running the following command to watch the Knative pods scale up and down:

kubectl get pods -w

After about 30 seconds of no traffic, you'll see the application pod terminate. When you make another request, Knative will automatically spin up a new pod.

Go Beyond with Traffic Splitting

One of Knative's most powerful features is traffic splitting, which allows you to route a percentage of traffic to different revisions. This is perfect for canary deployments, A/B testing, or gradual rollouts.

Modify whoami.yml to include a traffic splitting configuration:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          ports:
            - containerPort: 80
          env:
            - name: WHOAMI_NAME
              value: "Version 2"
  traffic:
    - tag: v1
      revisionName: whoami-00001
      percent: 75
    - tag: v2
      revisionName: whoami-00002
      percent: 25

Apply the updated configuration:

kubectl apply -f whoami.yml

This configuration implements two routing strategies:

Percentage-based routing: 75% of traffic goes to revision 1, while 25% goes to revision 2
Tag-based routing: Each revision gets its own dedicated URL using tags

Test the percentage-based routing by making 10 requests and observe the distribution

for i in {1..10}; do
  curl -H "Host: whoami.default.example.com" http://localhost
done

You'll notice that approximately 75% of responses show "Serverless with Traefik" (v1) as the service name and 25% show "Version 2" (v2).

You can also access specific revisions directly:

# Access version 1 directly
curl -H "Host: v1-whoami.default.example.com" http://localhost

# Access version 2 directly
curl -H "Host: whoami.default.example.com" http://localhost

More Advanced Features

One of the most powerful aspects of Knative is its automatic revision management. Every time you update your service, Knative creates a new revision, and Traefik seamlessly updates its routing configuration.

Check your revisions:

kubectl get revisions

You'll see the following output:

Each revision represents a snapshot of your service at a particular point in time. Traefik automatically routes traffic according to the traffic splitting rules you define, handling all the complexity of load balancing and service discovery.

You can also fine-tune Traefik's Knative provider with additional options:

experimental:
  knative: true

providers:
  knative:
    enabled: true
    # Watch specific namespaces only
    namespaces:
      - default
      - production
    # Filter by labels
    labelselector: "environment=production"

See the Traefik Knative provider documentation for all available configuration options.

Conclusion

With Traefik 3.6's Knative provider support, we've achieved a significant milestone in proxy technology: a single solution capable of managing traditional workloads, cloud-native containers, and serverless functions. This unprecedented versatility eliminates the need for multiple specialized proxies, dramatically simplifying your infrastructure while maintaining the performance and features each workload type demands.

You now have a unified proxy solution that automatically handles service discovery, scaling events, and traffic routing across your entire application landscape. Whether you're running VMs, Kubernetes deployments, or Knative serverless workloads, Traefik provides consistent, enterprise-grade networking with advanced routing, middleware, and observability features.

Traefik is committed to the Knative ecosystem, and we're actively working on improving the integration based on community feedback. The Knative provider is currently experimental in Traefik 3.6, with plans to enhance features like additional middleware support and deeper integration with Knative's advanced routing capabilities.

Useful Links

]]>

Traefik Proxy 3.6 "Ramequin": Where Every Layer Counts

Emile Vauge — Mon, 10 Nov 2025 22:01:39 GMT

Traefik Proxy 3.6 has arrived, and like the exquisite Ramequin cheese 🧀—a rare specialty from France's Bugey region—this release achieves something unique. With native Knative integration completing the trilogy, Traefik becomes the only proxy capable of seamlessly handling Cloud Native workloads (containers), Traditional workloads (VMs), and Serverless workloads in one unified platform. Revolutionary multi-layer routing solves complex traffic decisions by enriching requests through hierarchical layers, eliminating the painful workarounds teams have endured for years. Combined with the fresh new Gateway API 1.4 support, this makes v3.6 a significant milestone in Traefik’s journey. Let’s taste it!

Unique Features That Set New Standards

Multi-Layer Routing: The End of Workarounds

The most anticipated feature of this release solves a fundamental challenge in modern cloud-native routing: making routing decisions based on information that doesn't exist in the original request. Whether you need to authenticate first, then route based on user roles, check feature flags to decide between microservices and monoliths, or route enterprise customers to stable infrastructure while directing free users to staging infrastructure, multi-layer routing transforms complex workarounds into elegant, declarative configuration.

Traditional routing approaches force teams into painful compromises: duplicating authentication logic across every microservice, building separate routing proxies that add latency and complexity, or maintaining giant configuration files that break when requirements change.

Multi-layer routing introduces hierarchical relationships between routers, where parent routers enrich requests with additional context, and child routers make routing decisions based on that enriched information.

The magic lies in progressive request enrichment. Each layer adds context that subsequent layers use for increasingly specific routing decisions. Authentication happens once at the parent level, while routing decisions are made at the child level based on the enriched result—no duplication, no external proxies, no compromises.

Here's a real-world example: routing API requests to different backends based on customer subscription tier:

http:
  routers:
    # Root router - matches all API requests and applies auth
    api-parent:
      rule: "Host(`api.example.com`) && PathPrefix(`/`)"
      middlewares:
        - auth-with-tier
      entryPoints:
        - websecure
      tls: {}
      # No service defined - this is a parent router
    
    # Leaf router - routes enterprise customers to stable
    api-enterprise:
      rule: "HeaderRegexp(`X-Customer-Tier`, `(enterprise|business)`)"
      service: stable-backend
      parentRefs:
        - api-parent
    
    # Leaf router - routes free tier to canary
    api-free:
      rule: "Header(`X-Customer-Tier`, `free`)"
      service: new-version
      parentRefs:
        - api-parent

  middlewares:
    auth-with-tier:
      forwardAuth:
        address: "http://auth-service:8080/validate"
        authResponseHeaders:
          - X-Customer-Tier
          - X-Customer-Id
          - X-User-Email

  services:
    stable-backend:
      loadBalancer:
        servers:
          - url: "http://api-v1-stable:8080"
    new-version:
      loadBalancer:
        servers:
          - url: "http://api-v2-canary:8080"

The hierarchy supports three router types: Root Routers attached to entryPoints that apply shared middleware and define TLS configuration, Intermediate Routers that can have children while inheriting configuration from their root, and Leaf Routers that must define a service and handle the final routing decision. Child routers cannot be called directly—requests must flow through their parent, ensuring you can't circumvent authentication or other parent-level logic.

This unlocks previously complex patterns like authentication-based routing (where admin and user requests to the same endpoint route to different backends based on role), feature flag-based migrations (routing users to microservices or monoliths based on feature flag evaluation), and sophisticated progressive rollouts where routing decisions depend on customer context, subscription tiers, or experimental configurations. Learn more in the documentation and in the dedicated blog post.

Knative Integration: The Universal Proxy for Any Workload

Traefik v3.6 introduces Knative provider support, achieving a key milestone: Traefik is now the only proxy that can seamlessly operate across Cloud Native workloads (containers), traditional workloads (Virtual Machines), and serverless workloads. This unprecedented versatility makes Traefik a truly unique platform capable of handling any kind of workload your infrastructure demands.

The Knative provider automatically handles service discovery, scaling events, and traffic routing for Knative workloads, providing a unified experience across your entire application landscape. Teams can now use a single proxy solution for their complete infrastructure, dramatically simplifying operations while maintaining the specialized optimizations each workload type requires.

Let’s enable the KNative provider in Traefik first:

experimental:
  knative: true
providers:
  knative:
    namespaces:
      - "serverless-apps"
      - "production"

The following Service manifest configures the running Traefik controller to handle the incoming traffic.

---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          ports:
            - containerPort: 80
          env:
            - name: WHOAMI_NAME
              value: "Knative Test Service"

Once everything is deployed, sending a GET request to the HTTP endpoint should return the following response:

$ curl http://whoami.default.example.com

Name: Knative Test Service
...

Special recognition goes to idurgakalyan for spearheading this innovative integration that completes Traefik's transformation into the universal proxy for modern infrastructure (#11448). Learn more in the documentation and in the dedicated blog post.

Gateway API 1.4 Support: Leading the Standards Evolution

Traefik v3.6 embraces the future with full Gateway API v1.4 support, incorporating significant advancements that solidify Kubernetes networking standards. This release brings two major features from the Experimental to the Standard channel, making them production-ready for enterprise deployments.

BackendTLSPolicy graduates to Standard, enabling secure TLS configuration from Gateway to backend services with production-grade reliability. This critical security feature allows you to enforce end-to-end encryption, ensuring that traffic remains protected even between internal services.

SupportedFeatures status reporting also reaches Standard, providing clear visibility into which Gateway API features your implementation supports. This eliminates guesswork and enables better operational planning by exposing feature compatibility directly through Kubernetes status fields.

Additional Enhancements That Make a Difference

Health Checking Evolution

TCP Health Checks (#11238 by ddtmachado)—Native TCP health checking capabilities for non-HTTP services. Learn more.
Passive Health Checks (#11351 by Nelwhix)—Intelligent health monitoring that observes actual traffic patterns to determine service health. Here’s the documentation.

New Load Balancing Strategies

Least Time Strategy (#12167)—Route requests to servers with the lowest response times for optimal performance. Learn more in our configuration guide.
Highest Random Weight Algorithm (#9946 by mathieuHa)—Advanced probabilistic load balancing for enhanced traffic distribution. Check out Traefik MCP Gateway, which already leverages it to handle agent requests more efficiently.

Provider Enhancements

AWS ECS IPv6 Support (#12179 by wizbit)—Full IPv6 compatibility for Amazon ECS deployments. ECS provider docs.
Docker Non-Running Container Discovery (#10645 by acouvreur)—Enhanced container discovery capabilities for development workflows.

Kubernetes Integration

ExternalName Service Support (#12065 by james-callahan)—Publish Kubernetes ExternalName services through Traefik for external service integration.
Highest Random Weight in CRD (#12061)—Advanced load balancing options in Kubernetes Custom Resources.

Security and Performance

ACME Certificate Resolver Options (#11977 by ldez)—Enhanced certificate management capabilities with new resolver configurations. See our ACME documentation.
HTTP/2 HPACK Table Size Configuration (#12050 by GCHQDeveloper548)—Fine-tune HTTP/2 performance with configurable HPACK table sizes.

Developer and User Experience

Plugin Syscall Support (#11939 by david-garcia-garcia)—Extended plugin capabilities with syscall support for advanced integrations.
Dashboard Improvements (#12145 by leccelecce)—More compact table layouts for improved dashboard usability.

Observability

Provider Namespace Logging (#12002 by shreealt)—Enhanced visibility during startup for Consul, Consul Catalog, and Nomad providers.
Authentication Middleware Warnings (#12085 by kianelbo)—Improved security awareness with maxBodySize configuration warnings.

The Artisans Behind Every Layer

Like the careful aging process that creates Ramequin's distinctive character, Traefik v3.6 is the result of months of dedication from our incredible community. To every developer who opened an issue, every contributor who submitted a pull request, every user who provided feedback—you are the artisans behind Traefik's continued excellence. Your passion and expertise continue to drive innovation in cloud-native networking and contribute to routing requests to millions of applications and APIs worldwide.

Ready to taste the fresh new Traefik 3.6 "Ramequin"?

Essential Resources and Next Steps

Download Traefik 3.6: GitHub Releases | Docker Hub
Documentation: Complete Documentation | Migration Guide
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The AI Triple Security Gap: Why Your Gateway Strategy is Already Obsolete

Immánuel Fodor — Tue, 14 Oct 2025 15:02:02 GMT

MCP Changes Everything, and Most Security Teams Don't Know It Yet

The enterprise AI landscape just experienced a seismic shift. Anthropic's Model Context Protocol (MCP) has rapidly become the de facto standard for how AI agents interact with enterprise tools, data sources, and APIs. Within months of its release, MCP achieved what typically takes years: widespread adoption across major agent frameworks and AI development platforms, with thousands of MCP servers now deployed across the ecosystem.

But here's the problem: MCP introduces attack surfaces that traditional API security was never designed to handle.

While security teams focused on securing their APIs, a new threat vector emerged. Malicious instructions hide in prompts, legitimate tools transform into data exfiltration mechanisms overnight, and AI agents can be manipulated into bypassing every security control you've carefully implemented.

The uncomfortable truth: If your security strategy relies solely on API gateways, you're protecting the wrong layer.

Your Critical Data is Now Agent-Accessible

The MCP ecosystem is expanding at an unprecedented pace across enterprise infrastructure. Within months of release, hundreds of MCP servers have been deployed, with new implementations appearing weekly. Here's what's already agent-accessible through verified MCP server implementations:

Databases & Data Stores: Oracle DB23ai (official SQLcl MCP server), PostgreSQL, MySQL, MongoDB, SQLite, Redis, InfluxDB, CockroachDB, and other popular database systems

Data Warehouses & Analytics Platforms: Snowflake (official Cortex AI integration from Snowflake-Labs), BigQuery, Databricks, Apache Druid, and modern data platforms

Cloud & Developer Platforms: Salesforce (official DX MCP Server and community implementations), AWS services (CloudTrail, Bedrock, CDK), Supabase, Heroku, Firebase, and Cloudflare Workers

Collaboration & Communication: Google Drive, Slack, GitHub, Linear, Notion, Intercom, and productivity platforms

File Systems & Storage: Local filesystems with configurable access controls, cloud storage services, and document repositories

Development & DevOps Tools: Git repositories, CI/CD pipelines, container platforms, observability systems, and infrastructure-as-code tools

This isn't some distant future. It's happening right now. The MCP ecosystem is growing exponentially, with community and official implementations appearing across every category of enterprise software. Your perimeter security remains essential, but it's no longer sufficient once autonomous agents can chain multiple tools together across different systems before any single control point can see the complete attack.

The Critical Gap

Take database MCP servers as an example: they typically run with their own static database connection credentials, exposing tools that let AI assistants execute queries. The pattern is consistent across implementations. Authorization operates at the database connection level, and whatever the MCP server's authenticated user can do determines what the agent can do.

That's true, and it perfectly crystallizes the gap. Once connected, authorization collapses to the authenticated principal. There's no agent-aware control over specific tools, methods, or parameters before execution, nor any governance over the non-database tools the agent might chain together in the same session.

This pattern repeats across every MCP server type. Database MCP servers inherit database user permissions. Salesforce MCP servers inherit the user's CRM role and permissions. Cloud platform MCP servers inherit the authenticated user's access scope. File system MCP servers inherit filesystem permissions. GitHub MCP servers use the personal access token's full permissions. Slack MCP servers can access any channel the token allows.

The Attack Scenario: Cross-System Data Exfiltration

Consider this realistic attack: "Summarize sensitive customer data and share it externally"

Without proper controls, the attack unfolds across multiple systems. First, the Slack MCP Server reads confidential customer discussions from private channels. Next, the Salesforce MCP Server queries opportunity data with revenue projections. Then the Google Drive MCP Server accesses strategic planning documents. Finally, an HTTP tool posts the consolidated summary to an external webhook. Your competitive intelligence is gone.

Why traditional security fails here: The attack chain spans multiple systems (Slack, Salesforce, Google Drive, and external HTTP) that your individual system security controls can't govern holistically. Each individual access looks legitimate. The combination is malicious.

The Triple Gate Pattern: Defense in Depth for AI

The cloud-native community learned this lesson with containers: layered defense beats perimeter security. We didn't secure Kubernetes with a single firewall. We built defense in depth with network policies, pod security standards, RBAC, and admission controllers.

AI agent security demands the same architectural rigor.

Figure: Traefik's Unified Triple Gate Pattern

Each gate enforces policies appropriate to its layer. Unlike single-gateway approaches, attacks must defeat all three gates simultaneously to succeed, dramatically reducing your attack surface.

Gate 1: AI Gateway (Securing the Conversation)

The problem starts at the conversation layer. Attackers don't need to break your encryption or steal credentials anymore. They just need to convince your AI agent to do their bidding. Traditional security tools can't help here because they were built for APIs, not conversations with language models.

This is where Traefik fundamentally changes the game. Instead of relying on a single AI security model or vendor, Traefik orchestrates multiple specialized AI models into a unified policy enforcement pipeline. Think of it as having multiple security analysts, each trained to spot different attack patterns, all working in concert.

Topic control models detect when someone's trying to extract data or integrate with external systems. Content safety models identify privacy violations across more than 22 categories of sensitive information. And jailbreak detection models catch the sophisticated attempts to bypass your AI's restrictions, the digital equivalent of social engineering.

The key advantage? Unlike point solutions or cloud-based AI services that force you to send your data elsewhere, Traefik's entire security pipeline runs in your infrastructure. Deploy GPU-accelerated models like NVIDIA NIMs for maximum performance, integrate your custom classifiers trained on your specific threats, or combine multiple vendors' models. All without changing your architecture. No external API calls putting your data at risk. No vendor lock-in constraining your choices. Just one unified policy layer that adapts to your needs.

This gate blocks prompt injection attacks, jailbreak attempts, PII extraction requests, off-policy conversations, and inappropriate interactions before they ever reach your AI systems.

Gate 2: MCP Gateway (Governing Tool Access)

The MCP protocol presents a unique challenge. By design, it delegates all access control to implementers, creating what amounts to a protocol-level security vacuum. The specification is refreshingly honest about this, explicitly stating that "the core protocol lacks standardized permission or sandbox mechanisms."

This deliberate gap demands a new approach to authorization. Traditional access control models fail spectacularly when applied to AI agents.

Why Traditional Access Control Fails for AI Agents

Consider RBAC: agents don't have job titles or departments. When an agent inherits a "sales manager" role in Salesforce, it suddenly has access to all opportunities, all forecasts, and all customer data across the entire organization. That's absurd when all it needs to do is send follow-up emails to customers who haven't responded in 30 days.

ABAC seems better at first, since you can add context like time and location. But as you add more attributes, the complexity explodes exponentially. Try managing attribute combinations for hundreds of autonomous agents accessing dozens of different systems, each with their own attribute schemas. It quickly becomes an opaque mess that no one can audit or understand.

Delegation and impersonation make things even worse. When an agent impersonates a user across Salesforce, Google Drive, and Slack simultaneously, tracking who's responsible for what becomes nearly impossible. The audit trail

turns into fiction, and agents routinely gain access far beyond what any single task requires.

The fundamental issue is that these models were designed for human users with relatively stable, role-based permissions. They assume someone clocks in at 9 AM with a fixed set of responsibilities. AI agents need different access across different systems for different workflows, often changing by the minute.

How TBAC Works: Three Dimensions of Control

This is why we developed Task-Based Access Control (TBAC), a new authorization paradigm that focuses on the work being done rather than the identity doing it. TBAC creates a progressive permission funnel through three dimensions:

📋 Tasks (High-level business objectives): Tasks represent business-level goals and intent like customer_followup, generate_sales_report, or resolve_support_ticket. This is the coarsest filter and your first gate. Before considering which systems the agent can access, you validate that the agent is authorized to perform this type of business work at all. An agent without the financial_reporting task can't even begin to access financial systems, regardless of what specific tools it might need.
🔧 Tools (System and resource access): Tools control which specific MCP servers and their exposed resources the agent can access. Even if an agent is authorized for the customer_followup task, it doesn't get blanket access to all systems. Instead, it receives precisely scoped access to specific MCP servers (salesforce_crm, email_service) and their individual tools/methods (query_opportunities, send_email). Your customer follow-up agent might need Salesforce's opportunity queries and email capabilities, but has no business accessing the export_all_data tool or your google_drive server with financial documents.
💰 Transactions (Granular runtime controls): This finest-grained layer validates the specific parameters and context of each operation. It's not just "can this agent query Salesforce opportunities" but "can it query opportunities with these specific filters at this time from this location?" Transactions act as runtime guardrails, preventing dangerous operations even when the agent has legitimate access to the tool. They block queries without filters, updates to protected fields, or operations that could exfiltrate large datasets.

# An example JWT schema
{
  "sub": "<target agent>",
  "tasks": ["<list of tasks>"],
  "tools": {
    "<mcp server>": {
      "actions": ["<action names>"],
      "filters": { <additional transaction filters> }
    }
  }
}

These three dimensions work as a progressive permission funnel. First, validate the business objective (task). If authorized, validate which systems and specific tools can be accessed (tools). Finally, validate the specific operation parameters and context (transactions). Each layer further constrains what the agent can do, ensuring minimum necessary permissions across all enterprise systems.

The Innovation: Dynamic Variable Substitution

Here's where traditional policy systems break down completely. Want to give agent-123 access to Salesforce? You write a policy. Does it also need Google Drive? Another policy. Add agent-456 with different permissions? More policies. Scale this to hundreds of agents accessing dozens of systems, and you're drowning in thousands of policy rules that become impossible to maintain or audit.

Traefik flips this entire model through JWT claim and MCP parameter substitution: ${jwt.claim_name} and ${mcp.parameter_name}.

Now one policy works for every agent:

policies:
  # Task-level authorization
  - match: "Contains(`jwt.tasks`, `${mcp.params.task}`)"
    action: allow
  
  # Tool-level authorization
  - match: "Contains(`jwt.allowed_tools`, `${mcp.params.name}`)"
    action: allow
  
  # Transaction-level constraints
  - match: |
      Equals(`mcp.params.name`, `query_opportunities`) &&
      Contains(`jwt.tools.salesforce_crm.allowed_fields`,         `${mcp.params.field}`)
    action: allow

The same policy line works for every agent. Their specific permissions come from their JWT token issued by your existing identity provider or any OAuth compliant provider like Okta, Azure AD, Auth0, Keycloak, etc. No more policy explosion. No more maintenance nightmare.

Real-World Example: Customer Follow-Up Campaign

Let's see how this works in practice. A sales agent needs to query recent opportunities in Salesforce, draft follow-up emails based on templates in Google Drive, and log activities back to Salesforce. Nothing else.

The JWT token structure defines exactly what this agent can do:

{
  "sub": "agent:sales-followup-bot",
  "tasks": ["customer_followup"],
  "tools": {
    "salesforce_crm": {
      "actions": ["query_opportunities", "create_activity"],
      "filters": {"status": "Open", "days_since_contact": ">30"}
    },
    "google_drive": {
      "actions": ["read_file"],
      "allowed_folders": ["email_templates"]
    },
    "email_service": {
      "actions": ["send"],
      "allowed_domains": ["customer.com", "prospect.com"]
    }
  }
}

The JWT token acts like a security badge that precisely defines what the sales agent can do. The tasks field authorizes only customer follow-up work (not financial reporting or other business functions). The tools section grants access to specific systems and their methods: Salesforce access is limited to querying open opportunities inactive for 30+ days and logging activities. Google Drive access is restricted to reading only the email templates folder, not financial documents. Email capabilities are constrained to sending messages only to customer domains, not internal addresses. Each nested level further restricts permissions, creating an airtight scope for this specific workflow.

The result? The agent completes its workflow with precisely scoped permissions across all three systems. If compromised, it cannot access closed deals or sensitive forecast data in Salesforce. It cannot read financial planning documents in Google Drive. It cannot send emails to internal company addresses. And it cannot access Slack, databases, or any other MCP servers.

GitOps Governance Layer

Beyond runtime enforcement, every MCP server undergoes human validation before deployment. Tool definitions are stored in Git, requiring approval before reaching production. Change detection alerts trigger when tool behavior deviates from the approved baseline. And a complete audit trail tracks who approved what, when.

This blocks tool poisoning attacks (malicious instructions in descriptions), rug pull attacks (tools that transform after approval), tool shadowing (malicious tools with identical names), privilege escalation, cross-system data aggregation attacks, and configuration drift.

Gate 3: API Gateway (Protecting the Backend)

Even with Gates 1 and 2 in place, MCP servers eventually call your backend APIs, databases, cloud services, and enterprise systems. These underlying services need their own layer of protection.

Traefik's battle-tested API management capabilities provide this final defensive layer. Centralized credentials management automatically injects service credentials, rotates tokens, and manages authentication to backend APIs without exposing secrets to MCP servers. Fine-grained API authorization ensures agents can only invoke authorized endpoints. Distributed rate limiting protects backend services from overwhelming request volumes with intelligent throttling across your infrastructure.

API plans and subscriptions define consumption quotas and access tiers for different agent types, preventing resource exhaustion. Operation filtering exposes only specific API operations to agents, hiding sensitive or administrative endpoints. TLS/mTLS prevents man-in-the-middle attacks with encrypted transport and mutual authentication. Input validation sanitizes requests before they reach vulnerable servers. And DLP-style inspection detects sensitive data patterns in responses before they leave your infrastructure.

This comprehensive protection blocks network-level attacks, API abuse and resource exhaustion, unauthorized endpoint access, command injection attempts, SSRF attacks, credential theft and token leakage, and traditional API vulnerabilities.

How the Triple Gate Stops the Attack

Let's return to our cross-system exfiltration scenario and trace exactly how each gate responds to the threat: "Summarize sensitive customer data and share it externally."

The first line of defense activates at the AI Gateway. The topic control model immediately recognizes the dangerous pattern: data extraction combined with external sharing targeting sensitive information. Before the request even reaches your AI model, it's blocked with a 403 Forbidden response.

But sophisticated attackers don't give up that easily. They'll try to obfuscate their intent, perhaps breaking the request into seemingly innocent pieces. That's when the content safety model steps in, trained to identify privacy violations even when they're cleverly disguised. And if that fails? The jailbreak detection model catches the manipulation attempt, recognizing the linguistic patterns that indicate someone's trying to bypass restrictions.

Even if an attacker somehow slips past all three AI-level defenses (perhaps through a zero-day prompt injection technique), they hit the MCP Gateway's TBAC enforcement. Now they face a completely different challenge. The agent's token only authorizes customer_followup tasks, not data aggregation. Its tool allowlist explicitly excludes access to Google Drive's strategic documents. The Salesforce query filters prevent any access to sensitive forecast data. HTTP tool calls to external domains are blocked entirely. And cross-system data correlation? Impossible with the scoped permissions.

Should the impossible happen and the attack penetrates both previous layers, the API Gateway provides the final line of defense. Rate limiting algorithms detect the unusual data access patterns. Why is this agent suddenly reading hundreds of customer records? DLP inspection identifies sensitive data patterns in the aggregated responses, blocking them before they can leave your infrastructure. And egress controls ensure that even if everything else fails, that external webhook destination remains unreachable.

Five independent security layers, each operating on different principles, each capable of stopping the attack on its own. That's defense in depth.

Threat Coverage at a Glance

Attack Vector	Without Triple Gate	With Triple Gate
Prompt Injection	❌ Undetected	✅ Blocked (AI Gateway)
Tool Poisoning	❌ Executes	✅ Blocked (AI Gateway + GitOps)
Cross-System Data Exfiltration	❌ Succeeds	✅ Blocked (All Three Gates)
Rug Pull Attacks	❌ Transforms unnoticed	✅ Detected (Change Alerts)
Tool Shadowing	❌ Hijacks requests	✅ Prevented (Namespacing)
Privilege Escalation	❌ Full access	✅ Blocked (TBAC)
External Data Sharing	❌ Posts externally	✅ Blocked (MCP Gateway)
API Abuse & Resource Exhaustion	❌ Overwhelms backends	✅ Blocked (Rate Limiting + Quotas)
Unauthorized Endpoint Access	❌ Accesses admin APIs	✅ Blocked (Operation Filtering)
Credential Theft	❌ Exposes secrets	✅ Prevented (Upstream Auth)
MITM Attacks	❌ Intercepts traffic	✅ Blocked (TLS/mTLS)

Why One Unified Platform Matters

The Problem with Fragmented Tools

Deploying standalone AI security, MCP governance, and API gateway tools creates operational nightmares that compound over time.

Attack chains span multiple systems. When an attack moves from Salesforce to Google Drive to Slack to an external webhook, you're correlating logs from three different security products across different formats and timelines. By the time you piece together what happened, the damage is done.

Policy conflicts become inevitable. The AI gateway allows a request, the MCP gateway blocks it, but the API gateway allows it again. Who wins? How do you debug this? Which team owns the resolution?

Configuration drift accelerates with every product you add. Three products mean three configuration languages, three deployment pipelines, and three teams managing different policies. What starts as minor inconsistencies becomes major security gaps.

The blind spots between products become unmonitored attack surfaces. Each vendor assumes the others are handling certain threats. Attackers exploit these assumptions.

And the cost explosion is real: three vendors, three contracts, three support organizations, three renewal cycles, three sets of professional services. Your security budget triples while your actual security posture deteriorates.

The Traefik Advantage: One Platform. One Binary. Three Gates.

Traefik delivers protocol-level MCP protection that addresses the security gap inherent in MCP's design with native TBAC enforcement across all MCP server types. True AI sovereignty means NVIDIA NIMs run entirely in your infrastructure. No external API calls. No vendor lock-in.

The platform works with your existing infrastructure, integrating seamlessly with your OAuth-compliant IdP (Okta, Azure AD, Auth0, Keycloak, etc), deployment patterns (Kubernetes, Docker, VMs), and monitoring tools. Production performance is guaranteed through sub-millisecond authorization using stateless JWT validation. No database queries, no external calls.

Unified observability lets you correlate attack chains across databases, file systems, CRMs, and communication platforms in a single platform. And this isn't theoretical. Traefik is battle-tested at scale, trusted by millions of deployments worldwide, proven in production environments.

Of course, no security solution is perfect. Traefik cannot fix vulnerable server code or prevent command injection vulnerabilities in poorly written MCP server implementations, though it provides comprehensive logging and behavioral monitoring to mitigate risks. It cannot secure vulnerable clients or fix security flaws in client-side implementations, but enforces policies regardless of client behavior. It cannot prevent sandbox escapes or stop runtime containment breakouts in underlying infrastructure, though TBAC limits the blast radius. And it cannot replace secure coding practices. Developers must still write secure MCP servers following security best practices.

What this means is that Traefik provides comprehensive protection for protocol-level and infrastructure-level threats across your entire MCP ecosystem. This covers the vast majority of real-world attack vectors. The remaining gaps require secure development practices and infrastructure hardening that no gateway can replace.

The Bottom Line

MCP adoption is accelerating faster than security teams can respond. Developers are deploying AI agents with access to Salesforce, Google Drive, Slack, databases, and dozens of other critical systems, often without security review.

The question isn't whether your organization will adopt MCP-based agents. The question is whether you'll secure them before or after a breach.

The cloud-native community learned security lessons through painful incidents. Apply those lessons to AI agents before your organization becomes the cautionary tale.

Next Steps

Traefik Labs has pioneered the Triple Gate Pattern, the industry's most unified and declarative runtime platform that delivers enterprise-grade AI security without operational fragmentation.

Ready to Secure Your Agentic Enterprise?

Download the Complete Implementation Guide - Detailed technical documentation with code examples, configuration samples, deployment patterns, and testing procedures
Schedule a Technical Briefing - See the Triple Gate Pattern in action with your MCP architecture and threat scenarios
Request a Security Assessment - We'll analyze your MCP exposure and map TBAC policies to your agent workflows

Secure your agentic enterprise with Traefik's Triple Gate Pattern—available today, proven in production, ready to scale.

]]>

Nutanix + Traefik Labs: Modern Application Intelligence for Distributed Hybrid Infrastructure

Sudeep Goswami — Tue, 30 Sep 2025 15:33:29 GMT

Enterprise Infrastructure Has Evolved, Application Delivery Must Follow

Recent analyst reports from Gartner and Forrester reveal a fundamental shift in enterprise infrastructure: container orchestration has matured, multicloud adoption is accelerating, and distributed hybrid infrastructure (DHI) is exploding from 15% to 55% of enterprises by 2028. Nutanix has led this transformation and been named a Leader in Gartner's Magic Quadrant for Distributed Hybrid Infrastructure (DHI), delivering unified infrastructure that works consistently across any environment.

Yet even as Nutanix customers achieve infrastructure excellence, they face a broader industry challenge: fragmented Layer 7 application delivery capabilities. While Nutanix Flow Virtual Networking (FVN) provides robust L2-L4 capabilities and network security, enterprises also require advanced L7 services. These include identity-aware routing and load balancing, API gateways, WAF protection, and AI governance. Traditionally, these capabilities required separate, expensive point solutions from multiple vendors.

The VMware licensing disruption has accelerated migration to Nutanix AHV, and these customers bring established application delivery requirements. Rather than force customers to assemble disparate tools, Nutanix partnered with Traefik Labs to deliver a validated, integrated solution that extends Nutanix's infrastructure excellence into comprehensive application intelligence. This partnership reflects Nutanix's strategic approach: provide best-of-breed capabilities through validated integrations rather than forcing customers into monolithic stacks.

The hybrid reality persists: virtual machines and containers will coexist for years, AI workloads demand enterprise sovereignty and governance, and edge deployments require consistent policies without constant connectivity. Organizations need unified application delivery that spans compute substrates, eliminates tool sprawl, and enables safe modernization. All of this must be delivered through the operational simplicity that Nutanix customers expect.

Modern Application Intelligence Layer: Extending Nutanix Infrastructure Excellence

The Application Intelligence Layer (AIL) transforms fragmented application delivery into a unified, portable control fabric that works consistently across any infrastructure. Unlike point solutions that solve individual problems, AIL provides comprehensive application-aware capabilities that travel with workloads regardless of where they run or how they're packaged.

Through native integration with Nutanix AHV and NKP, Traefik Labs extends infrastructure capabilities into modern intelligent application delivery. The cloud-native platform combines identity-aware routing, comprehensive security, policy-driven governance, end-to-end observability, and AI runtime policy management—while leveraging the power of familiar Nutanix constructs like Prism Central, Flow networking, and category-based policies.

This Nutanix Ready validated integration eliminates the complexity of managing separate tools for different compute substrates while providing the enterprise-grade Layer 7 services that modern applications demand. Organizations can safely modernize legacy VM-based applications alongside new containerized services, leveraging Nutanix's unified management approach with Traefik providing seamless application-layer intelligence on top.

The result is infrastructure that doesn't just run applications. It makes them intelligent, resilient, and governed across every environment, from edge deployments to sovereign clouds, all while maintaining the operational simplicity that defines the Nutanix experience.

Business Value

For organizations migrating from VMware to Nutanix, the Traefik integration completes the infrastructure story. Rather than assembling separate tools for application delivery, customers gain enterprise-grade Layer 7 services as a natural extension of Nutanix's unified platform approach. This eliminates the tool proliferation that VMware customers historically managed through separate products and vendors.

The integration strengthens Nutanix's value proposition by providing:

Complete stack consistency: Customers manage application delivery through familiar Nutanix constructs (Prism Central, Flow networking, categories)
Faster time-to-value: Pre-validated Nutanix Ready integration reduces deployment risk and accelerates production readiness
Unified operations: Same infrastructure-as-code approach (Terraform, Ansible) across compute and application layers
Future-proof architecture: Built-in AI governance capabilities position customers for emerging requirements

Engineering productivity increases dramatically as teams leverage Nutanix's unified management while gaining advanced application services. Rather than managing separate tools from different vendors, organizations extend their existing Nutanix investment to cover comprehensive application delivery requirements. AI sovereignty becomes achievable through complete governance pipelines deployed entirely within Nutanix infrastructure. This includes content safety, agent control, and more, without external dependencies that compromise data security or regulatory compliance.

Key Use Cases

The integration addresses five critical scenarios where Nutanix infrastructure excellence extends naturally into application delivery:

Use Case	Solution	Benefits
VMware Migration	Nutanix Ready validated AHV integration with enterprise Layer 7 services	Complete infrastructure and application delivery story. Validated, supported solution reduces deployment risk. Faster migration with proven integration patterns.
Hybrid Modernization	Unified policies across VM and container workloads on Nutanix	Safe incremental modernization without business disruption. Consistent security across all Nutanix compute substrates. Bridge legacy and modern apps seamlessly.
Tool Consolidation	Single integrated platform for WAF, load balancing, API gateway, AI gateway, observability	Eliminate expensive vendor sprawl and licensing costs. Reduce operational complexity and training overhead. Extend Nutanix simplicity to application layer.
AI Sovereignty	Safe and Responsible AI governance pipelines with NVIDIA NIMs	Deploy AI safely in regulated industries and air-gapped environments. Maintain complete data sovereignty on Nutanix platforms. Enterprise-controlled AI without external cloud dependencies.
Deploy Anywhere	Consistent operation across all Nutanix deployment models	Works across edge, Nutanix Cloud Clusters, private cloud, air-gapped environments. Complete operational independence from external systems. Compliance-ready evidence for regulatory audits.

VMware Migrations

Organizations moving to Nutanix AHV gain not just infrastructure replacement, but infrastructure advancement. The Nutanix Ready validated integration with Traefik delivers enterprise Layer 7 services that integrate seamlessly with Flow Virtual Networking. This provides capabilities that match or exceed what VMware customers assembled from multiple vendors, but with Nutanix operational simplicity.

Nutanix Customer Modernization

Nutanix has always excelled at hybrid infrastructure. This integration extends that excellence into application delivery, providing unified policies across AHV virtual machines and NKP containers. Organizations modernize safely using the same Nutanix management constructs they trust: Prism Central, Flow networking, and category-based policy. They gain sophisticated application-layer intelligence in the process.

Strategic Infrastructure

Nutanix customers who standardized on Nutanix for its operational simplicity and vendor-neutral approach now have a matching application delivery solution. Nutanix infrastructure excellence combines with best-of-breed application intelligence, without vendor lock-in or operational complexity.

Regulated Industries

Nutanix's strength in sovereign cloud deployments extends to application delivery. The Traefik integration operates entirely within customer-controlled Nutanix infrastructure. This supports the air-gapped and compliance-focused deployments where Nutanix excels: defense, healthcare, financial services, and government organizations that cannot accept external dependencies.

Global Operations

Nutanix infrastructure spans edge to cloud. This includes retail locations, data centers, and Nutanix Cloud Clusters on AWS and Azure. The Traefik integration extends consistently across this entire footprint, providing the same application intelligence whether workloads run in the core data center, at remote edge sites, or in public cloud extensions of Nutanix infrastructure.

Technical Solution

Technical Architecture and Implementation

The Traefik-Nutanix integration delivers a comprehensive Application Intelligence Layer across both Nutanix AHV virtual machines and Nutanix Kubernetes Platform (NKP), creating a unified control plane that extends Nutanix's infrastructure management philosophy to the application layer.

High-Level Architecture: Modern Application Intelligence for AHV and NKP

Native Integration with Nutanix Infrastructure

Flow Virtual Networking (FVN) Integration: The platform provides proven integration across Flow Virtual Networking for both Tenant and Transit VPCs. This enables sophisticated multi-tenant architectures that leverage Nutanix's native networking capabilities. Traefik integrates directly with Prism Central using v4.1 APIs for automatic VM discovery and category-based grouping. This enables zero-touch service provisioning with configurable polling intervals as low as 2 seconds. The lightweight integration extends Nutanix's policy-driven management approach to application delivery.

The auto-discovery mechanism leverages Nutanix's native category system. Infrastructure teams define application groupings using familiar Prism constructs. As VMs are created, modified, or destroyed within Nutanix environments, Traefik automatically updates its routing configuration without manual intervention. This maintains the operational simplicity that Nutanix customers expect while adding sophisticated application-layer capabilities.

High Availability Architecture: Enterprise resilience is achieved through Nutanix NLB VIP load balancing across multiple Traefik instances. In validated tenant VPC configurations, two Traefik instances provide redundancy. Transit VPC deployments typically run three instances for higher availability requirements. This distributed architecture leverages Nutanix's proven HA mechanisms to ensure that application delivery continues uninterrupted even during maintenance windows or infrastructure failures.

Unified Application Delivery Capabilities

Identity-Aware Routing and Security: The platform leverages OIDC, mTLS, and JWT processing to enforce authentication and authorization policies that span VM and container boundaries across Nutanix infrastructure. Organizations can define routing rules based on user groups or organizational identity. For example, administrators can be directed to different application endpoints than developers, or region-specific content can be served based on authenticated attributes. All of this leverages Nutanix's security constructs.

Built-in WAF protection blocks common attacks like SQL injection and cross-site scripting using Traefik's high-performance Coraza engine. Policies apply uniformly whether traffic targets a legacy VM-based application on AHV or a modern containerized microservice on NKP.

Intelligent Traffic Management: The integration provides circuit-breaking, retry logic, and canary deployment capabilities for resilient application delivery across Nutanix environments. Distributed rate limiting operates across all Traefik instances, preventing any single instance from becoming a policy enforcement bottleneck. Traffic shaping policies can be defined once and enforced consistently across the entire Nutanix hybrid infrastructure. This is critical for organizations managing thousands of application endpoints across distributed deployments.

Enterprise Networking Patterns on Nutanix

Multi-Tenant VPC Architectures: The validated reference architecture demonstrates a Transit VPC pattern with multiple Tenant VPCs, each containing isolated application workloads, and leveraging Flow Virtual Networking's advanced capabilities. Traefik instances in the Transit VPC provide a centralized entry point for external traffic. They route requests to the appropriate Tenant VPC based on path, host, or identity attributes. This architecture supports:

Path-based routing across VPC boundaries leveraging Flow networking
Unified L7 load balancing across application instances in multiple VPCs
Consistent security policies integrated with Nutanix security constructs
Centralized observability through Prism while maintaining tenant isolation

Infrastructure-as-Code Automation: Full deployment and management automation is achieved through Ansible and Terraform integrations. These are the same tools Nutanix customers use for infrastructure provisioning. Infrastructure teams can also define Traefik configurations, routing policies, and security rules as code. This enables repeatable deployments across development, staging, and production environments. GitOps workflows support version control and rollback capabilities, treating infrastructure configuration with the same rigor as application code. This is consistent with Nutanix's infrastructure-as-code philosophy.

Sovereign Cloud and Edge Deployment

Zero-Dependency Architecture: The platform supports both connected and air-gapped deployments through signed policy bundles that enable sovereign cloud operations. This is critical for the defense, healthcare, financial services, and government organizations where Nutanix excels. Traefik operates entirely within Nutanix infrastructure. All routing logic, security enforcement, and policy decisions execute locally without requiring connectivity to external control planes or policy servers. This maintains the data sovereignty and operational independence that Nutanix customers require.

Comprehensive Observability: OpenTelemetry integration provides distributed tracing, metrics, and audit-grade logging that feeds existing SIEM systems. Unlike proprietary observability solutions that create vendor lock-in, Traefik's standards-based approach aligns with Nutanix's vendor-neutral philosophy. Organizations can use their existing monitoring investments. Traces span VM and container boundaries across Nutanix infrastructure, providing end-to-end visibility into request flows regardless of where application components execute.

AI Workload Governance on Nutanix

NVIDIA Safety NIMs Integration: For AI workloads running on Nutanix infrastructure, the integration brings complete sovereignty through NVIDIA Safety NIMs integration. This enables enterprise-controlled AI deployment without external cloud dependencies. Content safety, agent control, and governance pipelines operate entirely within Nutanix private infrastructure. This makes AI adoption viable for organizations with strict data sovereignty requirements who have chosen Nutanix for exactly this reason.

The Traefik AI Gateway capability supports model routing governance. Organizations can direct inference requests to appropriate models based on request characteristics, user identity, or compliance requirements. This becomes critical as enterprises deploy multiple large language models on Nutanix infrastructure with different cost, performance, and privacy characteristics.

Performance and Scale Characteristics

The lightweight Traefik binary delivers enterprise-grade performance without compromising the efficiency that makes Nutanix infrastructure attractive. Key characteristics include:

Sub-millisecond routing decisions for typical HTTP requests
Horizontal scalability across dozens of instances without coordination overhead
Minimal memory footprint enabling deployment even on resource-constrained edge nodes running Nutanix
Near-zero configuration drift through automated reconciliation loops integrated into CI/CD pipelines

This efficiency profile enables deployment patterns across the full range of Nutanix infrastructure. Application delivery logic can run on edge devices or in highly multi-tenant Nutanix environments, all without sacrificing the operational simplicity that defines the Nutanix experience.

Infrastructure and Application Intelligence, Unified

Nutanix transformed infrastructure by eliminating the complexity of managing separate compute, storage, and virtualization layers. The Traefik integration brings the same philosophy to application delivery: unified management, consistent operations, and freedom from vendor lock-in.

Organizations choosing Nutanix gain:

Infrastructure excellence recognized by Gartner as market-leading in Distributed Hybrid Infrastructure
Application intelligence that extends naturally from that foundation through this validated integration with Traefik
Operational consistency across the entire stack, from infrastructure to application delivery
Strategic flexibility to modernize at their own pace without disruption
Future readiness for AI, edge, and emerging workload requirements

The Traefik-Nutanix integration strengthens what Nutanix customers value most: operational simplicity without sacrificing capability, vendor-neutral architecture without sacrificing integration quality, and the freedom to deploy anywhere without sacrificing functionality. It extends Nutanix's infrastructure leadership into intelligent application delivery, transforming infrastructure from a platform that runs applications into one that makes them intelligent, resilient, and governed across every environment.

Ready to evolve from "infrastructure anywhere" to "intelligent applications everywhere"?

Request your pilot and start building your application intelligence foundation.

Learn more about the Traefik-Nutanix integration at traefik.io

]]>

The Application Intelligence Layer: Beyond Infrastructure to Intelligent Applications Everywhere

Sudeep Goswami — Tue, 30 Sep 2025 14:57:18 GMT

Three Landmark Reports Paint a Clear Picture of Infrastructure's Future

In recent months, three pivotal analyst reports have defined the trajectory of enterprise infrastructure:

Gartner's Magic Quadrant for Container Management (2025) reveals that container platforms are reaching maturity, but application-layer capabilities remain fragmented
Forrester's Wave™: Multicloud Container Platforms, Q3 2025 demonstrates that multicloud complexity demands unified governance and policy management
Gartner's Distributed Hybrid Infrastructure (DHI), 2025 projects that 55% of enterprises will initiate DHI proofs of concept by 2028, driven by VMware disruption and sovereignty requirements

Together, these reports establish an undeniable consensus: the future is hybrid, multicloud, distributed, and demands application intelligence.

The catalysts driving this transformation are seismic. VMware licensing changes have created urgent migration pressures across enterprises. Sovereign cloud mandates are rewriting compliance playbooks. AI inferencing is pushing workloads closer to users and data. And the enduring reality highlighted across all three reports is that VMs and containers will coexist for years to come.

Infrastructure & Operations (I&O) leaders have already proven their capability—mastering Kubernetes adoption, driving modernization initiatives, and maintaining uptime in increasingly complex environments.

But as these reports make clear, the next critical challenge is emerging: How do we ensure that all this distributed infrastructure actually delivers resilient, secure, and intelligent applications everywhere?

What the Analysts Reveal: The Infrastructure Revolution Is Here

Reading these three reports together reveals a fundamental shift happening right now in enterprise infrastructure:

Container Orchestration Has Matured, But Application Delivery Hasn't

Gartner's Container Management Magic Quadrant shows that while Kubernetes and container platforms have achieved enterprise readiness, Layer 7 capabilities remain fragmented across vendors. Organizations are left stitching together ingress controllers, API gateways, and security tools.

Multicloud Success Requires Unified Application Governance

Forrester's multicloud analysis proves that the winners achieve policy consistency without vendor lock-in. But most organizations struggle with different authentication systems, inconsistent security policies, and fragmented observability across AWS, Azure, GCP, and sovereign clouds.

Distributed Hybrid Infrastructure (DHI) Adoption Is Exploding Due to VMware Disruption

Gartner's DHI research shows adoption expected to jump from 15% to 55% of enterprises by 2028—a once-in-a-decade transformation window. Organizations need cloud-native capabilities everywhere, but existing solutions force them to choose between infrastructure flexibility and application intelligence.

Edge and Sovereignty Are Strategic Imperatives

All three reports emphasize that latency-sensitive workloads, disconnected operations, and national sovereignty mandates are pushing infrastructure closer to users and under stricter jurisdictional control. Cloud-native principles must work without constant cloud connectivity and with complete data sovereignty.

The Hybrid Reality Demands Bridge-Building

All three reports acknowledge that virtual machines and containers will coexist for years. Yet most application delivery solutions work for one compute substrate or the other—not both seamlessly.

AI Workloads Need Immediate Governance

While not explicitly covered in these infrastructure reports, the elephant in the room is clear: AI inferencing is scaling faster than governance. Organizations deploying AI models across distributed infrastructure need consistent quotas, cost controls, and safety guardrails—not another complex tool to manage.

This convergence of analysis points to an undeniable conclusion: infrastructure has evolved, but application delivery is the bottleneck

The Solution: Modern Application Intelligence as a Unified Layer

The Application Intelligence Layer (AIL) is the missing infrastructure component that transforms fragmented application delivery into a unified, portable control fabric. Unlike point solutions that solve individual problems, modern AIL provides comprehensive application-aware capabilities that work consistently across any infrastructure.

This isn't just another tool—it's the connective tissue that transforms infrastructure investments into intelligent application delivery, regardless of where workloads run or how they're packaged.

For I&O leaders, AIL represents the natural evolution of infrastructure mastery, extending hard-won orchestration capabilities into the application layer where competitive advantage is ultimately determined.

The Five Pillars of Modern Application Intelligence

5 Key Pillars of Modern Application Intelligence

Traefik Labs has defined the essential capabilities that comprise a complete Application Intelligence Layer:

API-First & AI Readiness: moves beyond basic load balancing to offer comprehensive API management, API Gateway, and AI Gateway capabilities like model routing, semantic caching, and safety guardrails.
Security & Traffic Intelligence: delivers identity-aware routing (OIDC, mTLS), built-in WAF protection, intelligent traffic management, and end-to-end observability through OpenTelemetry.
Hybrid Orchestration: provides unified policy management across both virtual machines and containers, with auto-discovery and support for gradual modernization patterns.
Sovereign Cloud Enablement: ensures complete operational independence through zero external dependencies, offline policy bundles, and comprehensive audit-grade logging.
Edge-to-Cloud Continuity: guarantees consistent behavior across distributed deployments with local gateway enforcement during connectivity failures and central policy management.

Application Intelligence in Action: Real-World Impact

Financial Services: Safe Hybrid Modernization

A global bank routes /v1/payments to VM-based core banking systems while /v2/payments serves containerized microservices—all with unified OIDC authentication and WAF protection. This enables safe, incremental modernization without the downtime risks that could cost millions per hour.

Healthcare: Sovereign PHI Protection

Hospitals manage patient data in air-gapped environments using offline policy bundles and signed attestation. Exportable audit evidence streamlines HIPAA compliance while maintaining complete operational independence from external systems.

Government: Multi-Jurisdiction Services

Agencies deliver citizen services with per-jurisdiction policy packs and complete data sovereignty. Zero external dependencies ensure compliance while tenant isolation maintains security across different government departments.

Manufacturing: Resilient Edge Operations

Factories rely on local Traefik gateways to enforce policies during WAN outages. Intelligent back-pressure prevents equipment failures while maintaining operational continuity during network disruptions.

AI Teams: Governed Model Deployment

Enterprises route inference traffic based on cost, latency, and data sensitivity. Semantic caching reduces redundant API calls by 25%, usage quotas control GPU spending, and safety filters ensure responsible AI deployment across all endpoints.

Each scenario demonstrates the same fundamental truth: application intelligence is what transforms infrastructure investments into competitive advantage.

The Business Case: Measurable ROI Through Risk Reduction

Application Intelligence Layer implementation delivers immediate, measurable returns through risk mitigation and operational efficiency:

Downtime Prevention: Even conservative estimates place enterprise downtime costs at $300K per hour. Preventing just a few hours of outages annually pays for AIL implementation many times over.

Compliance Assurance: Regulatory fines under GDPR, HIPAA, and other frameworks routinely exceed millions of dollars. Audit-grade evidence packs and consistent policy enforcement significantly reduce compliance risk.

AI Cost Control: GPU and LLM costs are escalating rapidly. Semantic caching and intelligent quota management can reduce AI infrastructure spending by 15-25%, often saving six figures annually.

Engineering Productivity: Teams spend 20-30% of their time managing configuration drift, static IP assignments, and firewall exceptions. Unified policy management reclaims this time for innovation.

Vendor Consolidation: By unifying WAF, ingress, API gateway, and observability functions, organizations reduce vendor sprawl and eliminate redundant licensing costs.

For I&O leaders, this isn't speculative ROI—it's embedded in avoided costs and operational efficiencies that every CFO immediately understands.

The Transformation Window Is Closing Fast

The convergence of analyst insights and market forces creates an urgent, limited-time opportunity:

VMware Migration Momentum: Gartner's DHI research directly links VMware disruption to accelerated infrastructure transformation. Organizations are already re-platforming—this strategic migration window won't reopen for years. Companies that integrate application intelligence now will avoid costly retrofitting later.

AI Governance Crisis Looming: While container and DHI platforms focus on infrastructure, AI workloads are scaling into production without proper governance. GPU costs are exploding, compliance risks are mounting, and the organizations that implement AI guardrails now will avoid the financial and regulatory crises facing late adopters.

Multicloud Complexity Compounding: Forrester's analysis shows that multicloud complexity increases exponentially without unified governance. Every month of delay makes consistent policy management exponentially harder to achieve.

The future-ready organizations identified across all three reports share one characteristic: they don't just migrate or orchestrate infrastructure—they transform it with application-aware intelligence from day one.

The Traefik Advantage: Completing the Stack

Traefik Labs uniquely positions enterprises for this transformation:

Proven Integration: Native embedding in Rancher K3s, Nutanix NKP, and Nutanix AHV provides validated deployment paths
Open Source Foundation: Millions of developers trust Traefik's core technology, with clear upgrade paths to enterprise governance features
Platform Agnostic: Traefik enhances rather than replaces DHI and container platforms, extending their orchestration capabilities with application-layer intelligence

Traefik don't compete with your infrastructure investments—it completes them.

Act Now: Your Application Intelligence Foundation

The analyst consensus is clear, the market forces are aligned, and the technology is proven. The question isn't whether to add application intelligence to your infrastructure strategy—it's how quickly you can implement it to transform current investments into lasting competitive advantages.

Start with immediate validation. Choose the approach that fits your most pressing need:

5-Day Infrastructure Bridge Pilot: Deploy unified WAF protection, identity-aware routing, and observability across both VM and container workloads. Prove policy consistency and validate rollback procedures in a controlled environment.

AI Governance Quick Win: Add usage quotas, safety guardrails, and cost tracking to existing AI inference endpoints. Demonstrate immediate ROI while establishing governance patterns for enterprise scale.

Multicloud Policy Unification: Implement consistent authentication and security policies across two cloud environments. Validate the portable governance model that Forrester identifies as critical for multicloud success.

Sovereign Cloud Readiness Test: Deploy signed policy bundles in a disconnected environment and generate audit evidence packs. Prove operational independence capabilities for compliance requirements.

The transformation window identified by Gartner and Forrester is closing. Organizations that act decisively will establish application intelligence capabilities that become increasingly difficult for competitors to replicate.

Ready to evolve from "infrastructure anywhere" to "intelligent applications everywhere"?

Request your pilot and start build your application intelligence foundation now.

]]>

Aged to Perfection: Traefik's 10-Year Journey from Zero to Standard

Emile Vauge — Mon, 22 Sep 2025 13:42:32 GMT

10 years ago on September 22nd, 2015, after months of research and hacking, I posted a link on Hacker News about this small project I'd been working on called Traefik. Honestly, I was just hoping a few fellow developers might find it useful for their own container routing headaches.

Well, things escalated quickly…

Early Days: The Microservices Wild West

Let's be honest—2015 was a bit of a mess for anyone trying to manage microservices. Docker containers were getting some hype, Kubernetes was still this intimidating thing that required a PhD to set up, and everyone was manually editing NGINX configs like it was 2005.

The problem was simple: traditional load balancers didn't understand the dynamic infrastructure of modern platforms. Your services were constantly coming and going, scaling up and down, and your poor reverse proxy was sitting there completely oblivious to what was happening behind it.

My idea was equally simple: what if the reverse proxy could just... figure it out automatically? What if it could watch your containers, understand your services, and configure itself? Or to put it differently, what if we could delegate the routing configuration to the application level. In a world where there are now hundreds or thousands of services, each should come with its own ingress properties attached to it.

The response surprised me, to put it mildly. Within a week of the public announcement, Traefik was in the top trending projects on GitHub, and I found myself frantically responding to issues and pull requests from developers around the world.

Looking back, I have to admit I had no idea this would become what it is today. I set out to solve a real life problem I was experiencing, maybe thinking that a few other microservice nerds might be facing the same issue. The idea that Traefik would eventually become such a critical piece of today’s platforms? That seemed pretty unlikely at the time.

But here we are, 10 years later, with some numbers that still make me do a double-take:

3.4 billion downloads on Docker Hub
56,000+ GitHub stars
5,000+ merged pull requests
6,000+ issues (we've accidentally created the world's most persistent TODO list)
Nearly 900 contributors from around the world
500+ releases (we've been... productive)
26 cheeses for codenames (in fact the most important metric)
16 maintainers
1 company founded

The Maturing Process

Over the past decade, Traefik has evolved through fundamental milestones that each marked a critical innovation in the cloud-native revolution. Every major release wasn't just an upgrade—it was a response to where the industry was heading, often arriving just as developers realized they needed exactly those capabilities.

Traefik v1 started with the fundamentals that felt revolutionary at the time:

Automatic service discovery (no more manual config updates)
Let's Encrypt integration (because HTTPS should be easy)
Support for Docker, Kubernetes, Marathon, and other orchestrators
Live configuration reloading (remember when you had to restart everything?)

Traefik v2 was a major effort at future proofing the architecture:

Complete redesign with routers, middlewares, and services
TCP and UDP support (not just HTTP)
Proper Kubernetes CRDs
Middleware chaining for complex traffic handling

We'll be honest—the v1 to v2 migration wasn't our smoothest moment. We learned a lot about backward compatibility the hard way.

Traefik v3 focused on modern standards and smoother upgrades:

Gateway API support (the future of Kubernetes ingress)
OpenTelemetry integration (the future of observability)
Much gentler migration experience (we learned our lesson)

Looking back, it's clear that the cloud-native industry itself underwent a fundamental shift during these 10 years. What started as a pure innovation and creativity stage—where everyone was experimenting with containers, orchestrators, and new architectural patterns—has matured into a productivity era. The wild experimentation phase is largely over; now it's about making these technologies reliable, secure, and easy to operate at scale.

Traefik's evolution mirrors this industry transformation, moving from solving bleeding-edge problems to providing stable, production-ready infrastructure that teams can actually depend on. Traefik has consistently helped define what cloud-native infrastructure should look like. We didn't just adapt to the productivity era; we helped create the tools and patterns that made it possible.

What's Next

After a decade of helping shape the cloud-native revolution, we're not slowing down. The industry has moved into its productivity era, but that doesn't mean the innovation stops—it just becomes more focused on solving the real operational challenges that platform teams face every day.

The community has been pretty clear about where they want us to go next, and honestly, the next steps feel like natural evolutions of what we've been building all along.

v3.5: The NGINX Compatibility Layer

Let's talk about the elephant in the Kubernetes room: ingress-nginx is in maintenance mode. No new features, only critical fixes, and a growing list of security concerns that keep platform teams up at night.

This isn't just another deprecation announcement—it's a fundamental shift that will impact every team running Kubernetes in production. The controller that powers many clusters is entering its endgame, and platform teams everywhere are asking the same question: "What now?"

We saw this coming and decided to do something practical about it. Traefik 3.5 includes a brand new NGINX provider that understands NGINX Ingress annotations. That means you can migrate from ingress-nginx to Traefik without rewriting a single manifest.

No massive refactoring projects. No weekend migrations that keep everyone on edge.

You keep your existing annotations, your existing workflows, and your existing sleep schedule. You just get a more actively maintained ingress controller that's designed for the cloud-native world we're actually living in. Bonus: you can also plan a later Gateway API migration using the same controller thanks to Traefik’s flexibility.

We'd love contributions from the community to make it even better. If you're migrating from NGINX Ingress and hit edge cases we haven't covered, or if you have ideas for improving the compatibility layer, we're all ears.

The migration path is straightforward, the security posture is solid, and the future roadmap is clear. Sometimes the best innovation is just making the obvious choice easier to make.

v3.6: More Power, More Features

Version 3.6 is shaping up to address some complex routing scenarios that our community has been requesting.

Multi-layer routing is probably the most significant architectural addition. Right now, Traefik follows a simple path: entrypoint → router → service → server. Version 3.6 will let routers forward requests to other routers instead of just services, creating routing trees for complex use cases.

Think about it this way: you could have a first layer of routers dedicated entirely to authentication and authorization, then forward authenticated requests to a second layer that routes based on the user's permissions or role. No more cramming everything into a single routing decision, or weird loopback hacks—you can build sophisticated traffic management that actually matches how your applications work.

KNative integration (thanks to our community contributors) will enable Traefik to be a first-class citizen in serverless Kubernetes environments, making it easier to route traffic to both traditional microservices and serverless functions from a single ingress controller.

v4: Perfectly Aged

We're taking a different approach with Traefik v4. Instead of the traditional "big bang" major release with tons of new features and breaking changes, we're trying something smarter.

All the new features destined for v4 will actually land in v3.x minor releases first. NGINX support, multi-layer routing, KNative, enhanced Gateway API support—you'll get to use them while they're fresh, not after waiting years for a major release.

At the same time, we'll be deprecating legacy features well in advance, giving everyone plenty of time to migrate gradually. By the time v4 arrives, it will essentially be a cleaned-up version of the latest v3.x—all the modern features you're already using, minus the deprecated baggage you've hopefully already migrated away from.

You'll be able to adopt new features as they become available, migrate away from deprecated ones at your own pace, and when v4 finally ships, the upgrade should feel more like a routine update than a major undertaking.

This approach should make the v4 migration much smoother than previous major releases. Sometimes the best way to move fast is to move incrementally.

Open Source at its Best

Here's what really amazes me: more than half of all pull requests merged into Traefik come from contributors who aren't on the core team. The community doesn't just use Traefik—they improve it, extend it, and take it in directions we never imagined.

We've seen contributions from developers in dozens of countries, solving problems we didn't even know existed. Someone in Japan fixes a bug that someone in Brazil discovers. A developer in Germany adds support for a protocol that a team in California needs. A technical writer in France improves documentation that helps a team in Australia get started faster. A security researcher in India reports a vulnerability that protects users worldwide. Even feature requests and bug reports from users who never write a line of code help us understand what's needed. Every type of contribution—code, documentation, testing, reporting issues, suggesting improvements—makes the whole user experience better. It's open source at its best.

The community has written crazy providers (hello KNative 🫡), created awesome plugins for use cases we never considered (👋 Sablier), made the forum such a welcoming space (❤️ bluepuma77), and generally made Traefik far more useful than it would have been with just the core team.

Anniversary Contest

To celebrate this incredible milestone, we're doing something special. The next 50 contributors who get a pull request merged into the Traefik repository will receive a limited-edition 10th Anniversary Traefik t-shirt. These aren't just any t-shirts—we're talking exclusive anniversary design and enough geek cred to make your coworkers jealous.

The design itself is pretty special (see below). Since every Traefik release comes with a cheese codename, we've created a shirt featuring all the cheeses that have graced our releases over the past decade.

From Reblochon to Mimolette, from Roquefort to Saint-Nectaire – it's a delicious timeline of Traefik's journey. We have to give huge thanks to Pierre Keersbulik, who has been creating these awesome designs for us throughout the past 10 years. His creativity has given Traefik its distinctive visual identity, and this anniversary design is no exception.

Whether you're fixing a typo in the documentation, adding a feature to a plugin, or contributing to the core codebase, your contribution counts.

These will probably become collector's items, so don't say we didn't warn you 😉.

A Vintage Worth Celebrating

10 years ago, I had a simple idea about making reverse proxies smarter. Today, Traefik is part of the infrastructure that powers modern applications used by millions of people. Today, an entire team and community of developers continues to improve what has become a major open source project.

The cloud-native ecosystem will keep evolving, the AI revolution is reshaping our infrastructure, new challenges and possibilities will emerge, and developers will keep building amazing things. As long as there's traffic to route and services to connect, we'll be here—probably with some new feature you didn't know you needed.

Thank you for 10 incredible years. Thank you for the contributions, the feedback, the bug reports, and the debugging sessions. Thank you for the thoughtful feature requests, the patient explanations in forum threads, and the conference talks that help spread the word. Thank you to the maintainers who keep the community welcoming, the documentation writers who make complex concepts accessible, and yes, even the trolls who keep us on our toes and remind us that the internet is still the internet. Thank you for making this little project into something we're all proud of.

Here's to the next decade of making infrastructure just a bit more automatic.

]]>

Spring Cloud Gateway vs. Traefik Hub: When to Choose a Purpose-Built Gateway

Immánuel Fodor — Mon, 04 Aug 2025 22:46:22 GMT

After years of working with organizations navigating the microservices evolution, we want to set the record straight. The API gateway you choose today could save your teams' sanity or haunt your architecture for years to come. As teams break apart monoliths into distributed services, the need for reliable API gateway strategies has become critical.

Yet many organizations find themselves in gateway chaos as different teams implement routing, authentication, and rate limiting in entirely different ways. We've seen too many teams make the same mistake with Spring Cloud Gateway. We don’t want you to look back six months later, drowning in custom Java code scattered across a dozen microservices, wondering how you got there.

So let's talk about what Spring Cloud Gateway is, why purpose-built alternatives like Traefik Hub exist, and—most importantly—when each one makes sense for your specific situation. We want to help you understand when embedded gateway logic makes sense, and when it's time to centralize, because if you've been considering Spring Cloud Gateway for your API gateway needs, you might discover that what seemed like the obvious choice could become a significant maintenance burden as your organization scales.

What Spring Cloud Gateway Really Is (Hint: It's Not What You Think)

Let's clear up the biggest misconception first: despite its name, Spring Cloud Gateway is not a cloud-hosted service or managed gateway solution. It's a Java-based framework that you embed directly into your Spring Boot applications.

When you add it to your project via Spring Initializr, you're essentially embedding gateway functionality within your application code. This means your microservice becomes responsible for both its business logic AND gateway concerns like routing, filtering, and request transformation.

Here's what that looks like in practice:

// Code from the Spring Cloud Gateway getting-started guide:
// https://spring.io/guides/gs/gateway

@SpringBootApplication
@EnableConfigurationProperties(UriConfiguration.class)
@RestController
public class Application {

  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }

  // Gateway logic
  @Bean
  public RouteLocator myRoutes(RouteLocatorBuilder builder, UriConfiguration uriConfiguration) {
    String httpUri = uriConfiguration.getHttpbin();
    return builder.routes()
      .route(p -> p
        .path("/get")
        .filters(f -> f.addRequestHeader("Hello", "World"))
        .uri(httpUri)) // Upstream service
      .route(p -> p
        .host("*.circuitbreaker.com")
        .filters(f -> f
          .circuitBreaker(config -> config
            .setName("mycmd")
            .setFallbackUri("forward:/fallback"))) // Local path serving
        .uri(httpUri))
      .build();
  }

  // Business application logic mixed with gateway code
  @RequestMapping("/fallback")
  public Mono<String> fallback() {
    // Write your custom handler logic here
    return Mono.just("fallback");
  }
}

@ConfigurationProperties
class UriConfiguration {
  private String httpbin = "http://httpbin.org:80";
  public String getHttpbin() {
    return httpbin;
  }
  public void setHttpbin(String httpbin) {
    this.httpbin = httpbin;
  }
}

The above simple example shows the gateway code living inside your Spring Boot app, making your service a gateway and application rolled into one. While this technique offers tremendous flexibility, it also means that gateway logic becomes tightly coupled with your business logic. Every time you need to change, for example, a simple routing logic, you’ll be deploying application code. Fun times during incident response, right?

When Spring Cloud Gateway Makes Sense

Don't get us wrong, we're not here to bash Spring Cloud Gateway. Spring Cloud Gateway has legitimate use cases and impressive strengths.

If your organization has a small engineering team (think 5-10 developers) who are already neck-deep in the Spring ecosystem, and your API gateway needs are straightforward, it can be perfect. With a minimal learning curve, you can leverage familiar Spring patterns, integrate seamlessly with other Spring packages, and use existing tooling around Spring Boot. For teams that think in Java, implementing gateway logic as program code can be faster than learning new configuration formats or deployment patterns.

Because you're writing everything in Java, you can implement virtually any custom logic you need. Want to perform complex request transformations based on database lookups? No problem. Need to implement custom authentication flows that integrate with your existing Java libraries? Spring Cloud Gateway gives you complete control.

However, these strengths come with significant caveats that become more apparent as organizations scale.

The Hidden Costs of Embedded Gateway Logic

We met a fintech startup that said the Spring-only approach worked for them beautifully at first. They had three microservices, all Java-based, and needed some custom auth logic that integrated with their existing setup. Spring Cloud Gateway let them move fast without introducing new infrastructure complexity.

But the company reached out to us 18 months later when they had grown to 15 services across four product teams. Different teams were implementing policies differently, making API versioning and security requirements inconsistent. Debugging routing issues meant digging through application logs across multiple services. What started as a simple, pragmatic choice had become a coordination nightmare.

As your microservices architecture grows, embedding gateway logic in your applications creates several challenges, as Spring Cloud Gateway makes it so easy to customize everything:

Governance nightmares: Every team implements gateway logic differently. Your user service might handle rate limiting one way, while your payment service does it completely differently. Auditing security policies across dozens of services becomes nearly impossible.
Deployment complexity: Gateway changes require full application deployments. Need to update a rate-limiting rule? You're deploying an entire application, not just a tiny configuration change.
Knowledge silos: Gateway logic becomes tribal knowledge within each team. Onboarding new developers means they need to understand both business logic and gateway patterns specific to each service.
Performance overhead: The JVM startup time and memory footprint affect every service, even those that might not need complex gateway features.

And we could continue with different error handling patterns, no central visibility into your API surface, multi-service dependencies for routing changes, and so on.

We've seen it first-hand. It's not fun.

Introducing Traefik Hub: The Purpose-Built Alternative

Traefik Hub represents a fundamentally different approach. Built on the high-performance Go language and optimized for cloud-native environments, it operates as a dedicated, standalone gateway service that sits in front of your applications—completely decoupled from your business logic.

Instead of embedding code representing gateway logic in your business applications, you define routing and policies as declarative configurations. For example, in Kubernetes environments, using Kubernetes-native Custom Resource Definitions (CRDs):

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: user-api-route
spec:
  entryPoints:
    - websecure
  tls:
    certResolver: letsencrypt
  routes:
    - match: Host(`api.example.com`) && PathPrefix(`/users`)
      kind: Rule
      services:
        - name: user-service
          port: 8080
      middlewares:
        - name: rate-limit
        - name: oidc-auth-check

This configuration lives outside your application code, making it easier to audit, version control, and modify without touching your business logic.

What's great about this approach is that your gateway policies become infrastructure as code. Want to update rate limiting? You're not deploying Java applications; you're updating simple YAML files that can be reviewed, automatically tested, and rolled back independently.

Technical Head-to-Head Comparison

Let's break down the technical differences that actually matter in production:

Aspect	Spring Cloud Gateway	Traefik Hub
Technology Stack	- Java (JVM) - Spring Framework - Project Reactor - Spring Boot	- Go-based - Built on Traefik Proxy - Kubernetes-native
Environment Support	- Spring Boot microservices - JVM environments	Infrastructure-agnostic: - Cloud - Multi-cloud - Hybrid - On-premises
Configuration	- Imperative - Java program code-driven	- Fully declarative - GitOps-native - YAML/TOML/text labels
Protocols Supported	- HTTP/1 - HTTP/2 (only when using Reactor Netty, Tomcat, Jetty, or Undertow as web servers) - Other protocols are either experimental, or require further dependencies, or are not supported	- HTTP/1/2/3 - TCP - UDP - WebSockets - gRPC -All of them are production-ready and out of the box
Traffic Management	- Predicate/filter-based routing - Path rewriting	- Dynamic expression-based routing - Automatic service discovery through various providers - Load balancing - Traffic mirroring - Blue/green - Canary deployments
Governance	- Decentralized, per-team - Rate limiting - Circuit breakers - Relies on Spring Security extensions	- Centralized policies - Built-in distributed rate limiting - OPA integration - Multi-cluster dashboard - Audit trails
Security	OAuth2, JWT, mTLS, RBAC via Spring Security integration	OIDC, LDAP, JWT, HMAC, OAuth2, API keys, OPA policy enforcement, Coraza WAF
Deployment	Embedded in various sprawling Spring Boot microservices, JVM runtime required	Standalone gateway service, native Kubernetes and other container orchestration integration, and optimized for cloud-native environments
Performance	- Intermediate byte code - JVM startup + runtime overhead	- Native binary - High throughput - Low latency
Observability	Per-service monitoring, usually requiring additional tools such as Micrometer and Spring Boot Actuator	Unified real-time dataflow metrics with first-class OpenTelemetry metrics, distributed tracing, and logs; pre-built Grafana dashboards, traffic debugger
Learning Curve	Familiar with Java teams	New concepts to Java folks, but standardized, shared by 3.4+ billion Traefik downloads worldwide
Extension Model	Spring-based filters/predicates; custom Java plugins and packages	Go and WebAssembly (WASM) plugins; language-independent middleware

With Spring Cloud Gateway, as you can see, most of the time you're either writing custom code or hoping someone else has solved your problem.

Migrating Custom Logic: From Java Filters to Go Plugins

One common concern about moving from Spring Cloud Gateway to Traefik Hub is: "What about our custom logic? We're a Java shop!" The good news is that most custom gateway logic can be translated into Traefik's built-in middleware or plugins.

Here's an example of implementing API Key authentication:

Spring Cloud Gateway (Java):

// Custom-built API Key auth since it's not available out of the box
@Component
public class ApiKeyAuthFilter implements GatewayFilter {

    private static final String API_KEY_HEADER = "X-API-Key";
    private static final String VALID_API_KEY = "your-api-key"; // Ideally fetched from a configuration file with additional file handling logic polluting the gateway

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        List<String> apiKeyHeaders = exchange.getRequest().getHeaders().get(API_KEY_HEADER);
        if (apiKeyHeaders == null || apiKeyHeaders.isEmpty() || !VALID_API_KEY.equals(apiKeyHeaders.get(0))) {
            exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
            return exchange.getResponse().setComplete();
        }
        return chain.filter(exchange);
    }
}

Traefik Plugin (Go):

// You don't need a custom plugin for API Keys, it's already built-in! :)

# Use this YAML config instead:
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: apikey-auth
spec:
  plugin:
    apiKey:
      keySource:
        header: X-API-Key
      secretValues:
        - "urn:k8s:secret:apikey:secret" # your-api-key

And as you can see, most of the time, for the usual, standard auth and API traffic management use cases, it’s not even needed to write custom code, just a couple of configuration lines.

While there's a learning curve for Java teams adopting Go, even writing a custom plugin often results in simpler, more maintainable code that's easier to test and deploy independently. And here's the kicker: you can also write plugins in WebAssembly, which means you could technically write them in Java if you really wanted to (though we wouldn't recommend it).

The Strategic Shift: Centralized Gateway Governance

The real decision isn't just about technology: it's about organizational strategy. Embedding gateway logic in applications might seem a simpler effort initially, but it often creates long-term governance challenges:

With Spring Cloud Gateway, auditing authentication and authorization policies requires examining code across dozens of repositories. With centralized gateways, your security policies are defined in one place, version-controlled, and auditable by anyone with basic YAML knowledge.
When you need to optimize performance, routing, or implement new caching strategies, embedded gateways require coordinated updates across multiple teams. Centralized gateways allow instant, organization-wide improvements.
New regulatory requirements often demand changes to logging, rate limiting, or data handling. Centralized gateways make these compliance updates trivial; embedded gateways turn them into massive coordination efforts.

And the hidden costs nobody talks about: debugging time when issues span multiple services with different gateway implementations. Lost revenue during the extended time to recover from failures. Coordination overhead when you need to make changes across teams. Infrastructural cost overhead from running JVM instances under every service. Knowledge transfer costs when developers move between teams.

With centralized gateways, you're trading some initial investment for long-term operational simplicity. Most teams find the break-even point somewhere around 6-12 months.

Planning for API Growth: The Developer Portal Advantage

One often-overlooked benefit of centralized gateways like Traefik Hub is their integration with API management platforms. As your API ecosystem grows, you'll likely need:

API documentation: Centralized gateways can automatically generate OpenAPI specifications even for legacy services
Developer onboarding: Self-service API access with automated provisioning on a developer portal
Usage analytics: Understanding which APIs are used, by whom, and how
Lifecycle management: Versioning, deprecation, and migration strategies

These capabilities are much harder to implement when gateway logic is scattered across individual services.

A Decision Framework That Actually Works

Stick with Spring Cloud Gateway if:

You're a small, cohesive team (under 10 developers), and don’t plan to grow
You're deeply invested in Spring and don't plan to introduce new frameworks or languages
Your gateway needs are simple and unlikely to evolve significantly
You value development speed over operational consistency

Move to Traefik Hub if:

You're scaling beyond a single team
Governance and standardization are becoming pain points
You want to separate infrastructure concerns from application logic
Performance and resource efficiency matter
You're adopting cloud/Kubernetes/GitOps-native practices and want infrastructure as code
You need unified observability across all your APIs
You find beauty in configuring existing middlewares and plugins without writing too much custom code
You want to future-proof your infrastructure

The Migration? It's Doable!

If you're considering a migration, here's a timeline that's worked for several organizations we've helped.

Months 1-2: Assessment Phase

Start by cataloging what you actually have. Most teams are surprised by how much custom logic they've accumulated. Document the patterns, identify the truly custom stuff versus the boilerplate.

Months 3-4: Proof of Concept

Pick your simplest service and migrate it first. This gives you experience with Traefik Hub without risking your most critical flows. Translate any custom logic to built-in middleware or plugins during this phase.

Months 5-8: Gradual Rollout

New services use the centralized gateway first. Then migrate existing services in complexity order: simple routing rules first, complex custom logic last. Or the most painful first and the easiest ones last.

Months 9-12: Optimization

This is where you get the payoff. Implement organization-wide policies, set up proper monitoring, and start seeing the operational benefits.

The key is not to rush it. We've seen teams try to migrate everything in a month and create more problems than they solved.

Why This Should Matter for You

The technology choices you make today affect your team's growth and flexibility tomorrow. Teams locked into Spring Cloud Gateway are essentially committed to Java-centric environments for the foreseeable future. Teams that understand and embrace modern gateway patterns can work across different technology stacks more easily. From a hiring perspective, it's easier to find developers who can work with YAML than it is to find experienced Spring specialists. The talent pool is just broader.

The question isn't really "Spring Cloud Gateway vs. Traefik Hub." It's "embedded gateway logic vs. centralized gateway architecture." The technology choices flow from that strategic decision. And the best technology decision is the one that lets your team ship features instead of debugging infrastructure. Choose accordingly.

]]>

Beyond the Models: Operationalizing Enterprise AI

Sudeep Goswami — Wed, 30 Jul 2025 14:53:13 GMT

Today’s AI landscape moves at a breakneck speed. Hugging Face alone now hosts nearly two million AI models—hundreds of thousands added each month, exemplifying the astounding pace of innovation. While remarkable, this proliferation creates an often-overlooked challenge: the true difficulty in enterprise AI isn't picking the perfect model; it’s building a flexible, scalable architecture around it.

The Real Challenge: Operationalizing AI

Choosing a model (e.g., GPT-4, Claude, Grok, or Llama) is just the start. The deeper, more pressing issue for enterprises is operationalizing these models effectively. Organizations must navigate complex decisions:

Model Selection: Should your organization use open-source models for flexibility, proprietary models for advanced capabilities, or custom-trained models tailored to specialized applications?
Deployment Environment: Is the public cloud the right choice for its scalability, or does your enterprise require the enhanced security of a private cloud? Or maybe the best option is a carefully managed hybrid solution.
Implementation Approach: Will your teams benefit more from assembling best-of-breed components internally, choosing vendor-provided turnkey solutions for rapid deployment, or adopting a balanced hybrid approach?

Most enterprises land somewhere in between—heterogeneous environments are the norm, not the exception. Successfully managing this complexity becomes the defining factor for scalable AI operations.

No AI Without APIs. And No APIs Without API Management.

AI endpoints and APIs are deeply intertwined. APIs have become the lifeblood of agentic AI systems, acting as secure bridges for agent workflows, enabling dynamic endpoints for tools, data access, and maximizing operational efficiency. This explosion of APIs necessitates robust API management, turning APIs into the vital control plane for your entire AI infrastructure. They manage secure access, enforce governance, and facilitate dynamic behavior, which is crucial for agentic AI workflows.

APIs also enable enterprises to implement consistent policy enforcement, streamlined developer experience, and comprehensive audit capabilities across their entire AI ecosystem. The rise of agentic AI further underscores the importance of APIs as essential bridges connecting diverse models and applications securely and efficiently.

Avoiding the Biggest Anti-Pattern in AI Architecture

One common yet detrimental practice is embedding API runtime capabilities directly within the model runtime environment. This creates substantial problems that limit scalability and agility:

Vendor Lock-in: Ties your organization to specific model providers, reducing flexibility and increasing costs.
Limited Enterprise Capabilities: Embedded APIs frequently lack advanced features needed for large-scale, mission-critical operations.
Non-Standard Compliance: Frequently breaks compatibility with widely accepted standards, such as OpenAI’s API specifications, complicating integration and maintenance.

You might already face these issues if:

Your organization struggles to integrate API token generation seamlessly with existing identity management systems.
You lack standardized, vendor-neutral observability tools for effective monitoring.
Your AI endpoints aren't easily shareable through centralized developer portals.

Recognizing and addressing these indicators early can significantly enhance your ability to scale AI responsibly.

Strategic Decoupling Through AI Gateways

A superior architectural choice involves strategically decoupling your model runtime from your API runtime, connecting them via an AI gateway such as Traefik Hub. Here’s why it matters:

1. Future-Proof Flexibility

Models constantly evolve. Decoupling ensures seamless integration of new models, allowing your enterprise to adopt emerging technologies quickly without disrupting existing operations.

2. Centralized, Scalable Security

Manage authentication, authorization, and access control across diverse models centrally, maintaining consistent, enterprise-grade security and guardrail policies regardless of which model your applications consume.

3. Infrastructure and Policy-as-Code

Use Kubernetes-native, cloud-native, and infrastructure-as-code (IaC) approaches to automate deployments, significantly reducing operational overhead. Implement and enforce policy standards like semantic caching, guardrails, and content filtering in a structured and version-controlled manner.

4. Vendor-Neutral Observability

Leverage OpenTelemetry to achieve consistent, vendor-neutral monitoring of metrics, logs, and traces across your entire infrastructure, ensuring deep visibility and efficient troubleshooting capabilities.

Practical Deployment Scenarios

Let’s look at how strategic decoupling works practically:

Private Cloud with Containerized Models

Deploy GPU-accelerated, containerized models using platforms like NVIDIA NIMs or KServe:

Centralized API gateways handle developer portals, JWT authentication, rate limiting, and PII masking.
Semantic caching via Redis, Milvus, or Weaviate improves efficiency and cost management.
Advanced routing and canary deployments enable the safe and dynamic introduction of new models, reducing risk and accelerating innovation.
Observability metrics allow for custom scaling flows based on AI specific metrics like token count, latency per model, and more.

Public Cloud Managed Services

Using managed services like AWS Bedrock, Azure AI Foundry, Google Vertex AI, or OCI GenAI:

Standardize API interactions via AI gateways integrated with OpenAI-compatible proxies.
Maintain governance through centralized policy controls and cloud identity providers.
Enable model switching and updates without any client-side disruptions.
Provide unified developer experiences across different managed services.

Scaling Responsibly: Guardrails and Cost Efficiency

To scale AI responsibly and economically, you’ll need the following capabilities:

Content Guardrails: Use gateways to enforce responsible AI practices by masking, blocking, or simply flagging sensitive data, inappropriate data, or off-topic content.
Semantic Caching: Implement semantic caching to reduce redundant inferencing, significantly cutting costs and enhancing performance.
Dynamic Routing and Experimentation: Avoid hard-coding models to applications; dynamically route traffic to test and optimize model performance seamlessly, ensuring flexibility and robustness.

Start Small, Build for Scale

Enterprises often face complexity paralysis. The solution is incremental yet strategic:

Begin with an AI Gateway: Establish this crucial decoupling foundation to enable flexibility from the outset.
Incorporate API Management: Layer in robust governance, comprehensive observability, and efficient developer experience.
Progressively Add Policies: Gradually expand with semantic caching, content filters, and detailed usage policies to enhance security and efficiency.
Prepare for Agentic AI: Build readiness for autonomous workflows and multi-model dynamic interactions, positioning your enterprise as a leader in next-generation AI.

The Bottom Line

True scalability in enterprise AI isn’t about betting on today's "best" model—it’s about building an adaptable, resilient architecture. By decoupling runtimes, implementing robust API management, and adhering to open standards, enterprises can confidently face the challenges that lie ahead in AI advancements.

Remember: Models are commodities. Architecture is your strategic advantage. Build wisely, and you’ll not just scale AI, you'll scale your enterprise's agentic future.

]]>

The Great Kubernetes Ingress Transition: From ingress-nginx EoL to Modern Cloud-Native

Emile Vauge — Thu, 24 Jul 2025 15:01:13 GMT

The Uncomfortable Reality of Maintenance Mode

The Kubernetes ingress landscape faces an unprecedented crisis. With ingress-nginx officially projected to enter maintenance mode, thousands of organizations running critical production workloads suddenly find themselves at a crossroads. No new features. No major improvements. Just maintenance patches for a controller that powers much of the internet's Kubernetes traffic.

This isn't just another project lifecycle transition—it's an infrastructure emergency waiting to happen. The #IngressNightmare vulnerabilities revealed fundamental architectural flaws that go beyond patchable bugs. When your ingress controller can be compromised through malicious configuration injection, you're dealing with systemic design problems, not isolated incidents.

The proposed solution? InGate—a complete rewrite that exists today as little more than an ambitious roadmap with a handful of commits. The timeline? Years of development before reaching production readiness. The reality is organizations can't wait years to address critical security vulnerabilities that exist now.

Meanwhile, platform teams face an impossible choice: continue running outdated and vulnerable infrastructure or embark on disruptive migrations to entirely different ingress solutions, losing years of operational knowledge and configuration investments in the process.

Key Requirements for a New Ingress Controller

With ingress-nginx entering maintenance mode and InGate still years from production readiness, a gap has emerged in the ingress controller world. The Kubernetes community faces an immediate and critical challenge: the need for a new ingress controller. The stakes are incredibly high, given that ingress-nginx currently powers millions of these workloads. Therefore, any viable solution must meet four non-negotiable requirements.

1. Security by Design

The controller must be architecturally immune to the template injection vulnerabilities that plague NGINX-based solutions. This means no raw configuration templating, no dynamic library loading, and no string interpolation of user inputs into configuration files.

2. Broad Ecosystem Adoption

Any replacement must have proven production scalability and a thriving community. Organizations can't afford to bet their infrastructure on experimental or niche solutions during a critical transition period.

3. Gateway API Leadership

The future of Kubernetes networking lies in Gateway API, not “legacy” Ingress resources. The ideal controller should be a leader in Gateway API implementation, providing a clear migration path toward modern standards.

4. ingress-nginx Compatibility

Most critically, there must be a practical migration path that respects existing investments. Organizations have spent years building expertise around ingress-nginx annotations and configurations—this knowledge shouldn't become obsolete overnight.

Can Traefik be the One?

Given Traefik's wide adoption across the cloud-native ecosystem, it's worth exploring whether it could satisfy these four critical requirements and serve as a practical bridge during this transition period. Here's how Traefik aligns to the four requirements we just discussed:

Security: A Decade of Secure-by-Design Architecture

Traefik's security advantages aren't accidental—they're the result of fundamental architectural decisions made a decade ago. When I designed Traefik ten years ago, I made some critical architecture decisions:

Go over C/C++: Choosing Go eliminated entire classes of memory safety vulnerabilities and optimized for interoperability with the cloud-native ecosystem
Static linking over dynamic libraries: It is impossible to execute code not already part of the binary and the code base
Structured parsing over templating: Configuration inputs are parsed into strongly typed Go structs, not interpolated into template strings
Minimal attack surface: No external component, like admission controllers, have elevated cluster privileges

These weren't performance optimizations—they were security-first decisions that have proven prescient in light of today's threats.

Adoption: Battle-Tested at Scale

Traefik isn't an experimental solution—it's a proven platform trusted by organizations worldwide:

3.4+ billion downloads across its lifetime
55,000+ GitHub stars and growing showcasing a strong preference
800+ contributors actively developing and maintaining the codebase
Production deployments in a wide range of startups and enterprises
Cloud-native DNA built specifically for dynamic, containerized environments

Unlike solutions built for traditional infrastructure and later adapted for Kubernetes, Traefik was designed from the ground up for cloud-native environments where services appear and disappear dynamically. For a decade, Traefik's modern and flexible architecture has been proven extremely reliable at scale across a wide range of enterprise deployments.

Gateway API: Leading the Standard

Traefik leads Gateway API development and ensures seamless compatibility. As a key contributor to the Gateway API specification, Traefik Labs makes compatibility a fundamental design principle from day one.

With Gateway API now production-ready and positioned to become the new standard for exposing workloads in Kubernetes over the coming years, early adoption becomes a strategic advantage rather than a risky experiment. Organizations investing in Gateway API today are building toward the inevitable future of Kubernetes networking.

With full Gateway API v1.3 support in Traefik v3.5, organizations gain access to the most advanced Kubernetes networking capabilities available.

This represents true leadership in the future of Kubernetes networking. While other solutions treat Gateway API as an experimental add-on, Traefik has made it a core competency, positioning organizations for the networking standards that will define the next decade of cloud-native infrastructure.

Ingress NGINX Compatibility: The Missing Piece

The final requirement—practical migration support—has been the missing piece ... until now.

Making Traefik NGINX Compatible

Rather than building yet another ingress controller from scratch, we focused on three core objectives that would provide immediate value to the community:

Support the most common use cases: We analyzed real-world ingress-nginx deployments and prioritized the annotations that represent 80% of actual usage.
Preserve existing investments: We wanted organizations to use their existing ingress-nginx resources without modification. Teams shouldn't need to rewrite manifests, retrain engineers, or disrupt operational workflows in this transition.
Eliminate injection vulnerabilities: Most critically, we wanted to securely parse NGINX ingress resources and transform them into Traefik's internal object configuration.

This approach offers compelling advantages:

Leverage existing strengths: Security, performance, and Gateway API leadership
Respect user investments: Preserve years of ingress-nginx expertise and configurations
Accelerate migration: Provide immediate relief rather than years-long development cycles
Enable gradual transition: Bridge from Ingress to modern Gateway API resources.

Here's a quick look at how it works:

How to Replace ingress-nginx with Traefik, Step-By-Step

Let’s say you have an ingress resource deployed on your cluster with ingress-nginx as an ingress controller:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-nginx-annotation
  namespace: default
  annotations:
    # BASIC Authentication
    nginx.ingress.kubernetes.io/auth-type: "basic"
    nginx.ingress.kubernetes.io/auth-secret-type: "auth-file"
    nginx.ingress.kubernetes.io/auth-secret: "default/basic-auth"
    nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"
    # SSL Redirect
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: whoami.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
  tls:
    - hosts:
        - whoami.localhost
      secretName: external-certs
---
kind: Secret
apiVersion: v1
metadata:
  name: basic-auth
  namespace: default
type: Opaque
data:
  # user:password
  auth: dXNlcjp7U0hBfVc2cGg1TW01UHo4R2dpVUxiUGd6RzM3bWo5Zz0=

This snippet exposes a web server with basic authentication and http to https redirection, that can be accessed with the following `curl` command:

curl http://whoami.localhost -L -u "user:password" --location-trusted

If you want to expose with Traefik Proxy, you only need to deploy Traefik and enable the NGINX Provider:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik
  namespace: default

spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik

  template:
    metadata:
      labels:
        app: traefik

    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v3.5.0
          imagePullPolicy: IfNotPresent
          args:
            - --log.level=DEBUG
            - --api.insecure
            - --api.debug
            - --entrypoints.web.address=:80
            - --entrypoints.websecure.address=:443
            - --experimental.kubernetesIngressNGINX
            - --providers.kubernetesIngressNGINX
          ports:
            - name: web
              containerPort: 80
            - name: admin
              containerPort: 8080
            - name: websecure
              containerPort: 443

---
apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: default
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: websecure
    - protocol: TCP
      port: 8080
      name: admin

--providers.kubernetesIngressNGINX will enable Traefik to discover and parse ingress-nginx ingress resources.

Et voilà! The web server is exposed by Traefik using the same ingress resource:

Hostname: whoami-6f57d5d6b5-19ncq
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.34
IP: fe80:: c498:5aff: fef7:be99
RemoteAddr: 10.42.0.38:47476
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/8.7.1
Accept: */*
Accept-Encoding: gzip
Authorization: Basic dXNlcjpwYXNzd29yZA==
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-7578bc64c5-pdjqm
X-Real-Ip: 10.42.0.1

As you can see, the migration path from ingress-nginx to Traefik could not be simpler! Bring your existing NGINX ingress resources, Traefik will do the rest.

As a cherry on top, Traefik Hub extends the open-source foundation with advanced API & AI Gateway & Management capabilities. This allows teams to start with secure ingress migration and evolve toward full API management capabilities without architectural disruption.

Traefik API Runtime Platform Core Capabilities

Full Support for High-Usage Annotations

The NGINX Provider doesn't attempt to support every possible ingress-nginx annotation—instead, we applied a data-driven approach to annotation selection based on real-world usage patterns and migration priorities.

The resulting annotation support spans the most critical operational requirements:

Authentication and authorization mechanisms
SSL/TLS configuration and backend communication
Session management and load balancing preferences
CORS policies for modern web applications
Advanced routing capabilities

This curated approach means most organizations can migrate their core ingress configurations immediately while gradually adopting Traefik-native features for advanced use cases.

For the complete list of supported annotations, behavioral differences, and configuration examples, see the comprehensive NGINX Ingress Provider routing documentation.

The Path Forward: Community-Driven Evolution

The NGINX Provider isn't a finished product—it's a foundation designed for community-driven evolution toward becoming a true drop-in replacement for ingress-nginx. As mentioned, our current implementation focuses on 80% of use cases.

The provider's architecture makes community contributions straightforward. Adding support for a new annotation typically involves:

Mapping the annotation to an equivalent Traefik middleware or option
Adding tests and documentation for the new functionality

This isn't just about building software—it's about building a sustainable community effort around practical migration needs. Each contribution makes the provider more useful for everyone facing the ingress-nginx transition.

Our long-term vision is clear: build a true drop-in replacement for ingress-nginx that provides both compatibility and modern capabilities.

Solutions, Not Promises…

The Kubernetes ingress crisis is happening now. ingress-nginx is about to be in maintenance mode. Many critical vulnerabilities have been discovered recently. Organizations need practical solutions today, not promises about tomorrow.

The Traefik NGINX Provider represents a pragmatic approach to an urgent problem. Instead of waiting years for InGate to mature or forcing disruptive migrations to completely different solutions, we're providing a secure bridge that respects existing investments while enabling future innovation. This isn't about perfect compatibility yet—it's about practical migration paths.

The great Kubernetes ingress transition is happening whether we're ready or not. The question isn't whether you'll need to migrate from ingress-nginx—it's whether you'll choose a practical bridge that's available today or wait years for a solution that might never fully materialize.

Be pragmatic, not idealistic. Your production workloads depend on it.

The NGINX Ingress Provider is available as an experimental feature in Traefik 3.5. For complete setup instructions, visit the setup guide. For configuration examples and the full list of supported annotations, consult the routing configuration documentation. Join the conversation and contribute to the future of practical Kubernetes ingress migration.

]]>

Traefik Proxy 3.5 "Chabichou": A Delicate Masterpiece

Emile Vauge — Thu, 24 Jul 2025 14:55:20 GMT

Traefik Proxy 3.5 has arrived! Codenamed "Chabichou" (the elegant goat cheese from Poitou, France, with its distinctive conical shape and creamy, tangy flavor that develops beautifully with age), this release introduces an exciting Ingress NGINX Provider for seamless ingress-nginx migration, a completely rebuilt React-based dashboard, and full Gateway API v1.3 support, alongside post-quantum cryptography and enhanced ACME capabilities.

Let's unwrap this delicate masterpiece!

Key Features of Traefik Proxy v3.5

Ingress NGINX Provider: Your Migration Path Forward

With ingress-nginx entering maintenance mode and the ingate project still in early stages, the Kubernetes community needed a clear migration path. That's where Traefik 3.5 steps in! We're introducing an experimental Ingress NGINX Provider (#11844) designed to enable ingress-nginx users to migrate seamlessly to Traefik. With it, organizations can continue to use their existing ingress-nginx resources, without modification, directly in Traefik. This transition shouldn't require teams to rewrite manifests, retrain engineers, or disrupt operational workflows.

This isn't about building 100% compatibility with every NGINX annotation—instead, we're focused on providing a practical migration foundation that covers the most common use cases. Our goal is simple: offer you a drop-in replacement for the majority of scenarios, then enable you to progressively migrate to modern standards like Gateway API at your own pace.

This experimental feature represents our commitment to the community's needs during this transition period. It supports essential annotations and core functionality, with the potential for rapid evolution based on community feedback and contributions.

We're excited to see how the community embraces and enhances this foundation! Read more about this in this deep dive article.

Dashboard Renaissance: Modern Interface, Enhanced Dev Experience

Say hello to the future of Traefik management! The dashboard has been completely rebuilt using React (#11674), delivering a sleek, responsive, and intuitive interface that makes managing your proxy configuration a pleasure. Additionally, this provides a better development experience with the ability to use mocks in development mode.

The new React-based dashboard provides enhanced performance, better accessibility, and lays the foundation for exciting features in future releases.

Gateway API v1.3: Leading the Kubernetes Evolution

Traefik continues its leadership in Kubernetes networking with complete support for Gateway API v1.3 (#11719). This latest specification keeps improving this new standard and Traefik remains at the forefront of Kubernetes networking, keeping your infrastructure ahead of the rapidly evolving cloud-native landscape.

Additional Enhancements That Make a Difference

Future-Ready Security with Post-Quantum Cryptography

Prepare your applications for tomorrow's security challenges with the introduction of X25519MLKEM768 for Post-Quantum-Secure TLS (#11731). This cutting-edge cryptographic approach ensures your communications remain secure against future quantum computing threats.

Our sincere appreciation goes to fzoli for implementing this forward-thinking security enhancement that positions Traefik at the front line of cryptographic innovation.

ACME Excellence: Enhanced Certificate Management

Certificate management becomes even more powerful with several key improvements:

OCSP Stapling Support (#8393)—Thanks to alekitto for implementing this performance and security enhancement
HTTP Challenge Delay Configuration (#11643)—Kudos to ldez for adding fine-grained challenge control
ACME Provider HTTP Timeout (#11637)—Appreciation to tkw1536 for improving reliability and customization

Advanced Health Check Capabilities

Health monitoring gets more sophisticated and flexible with unhealthy Interval Configuration (#10610)—Gratitude to sswastik02 for more granular health monitoring control

Kubernetes Integration Improvements

Kubernetes users benefit from enhanced compatibility and consistency: consistent Prefix Matching (#11203)—Our thanks to charlie0129 for aligning Ingress behavior with Kubernetes documentation standards

Middleware and Authentication Enhancements

Improved ForwardAuth Context Handling (#11817)—Special recognition to bengentree for better context cancellation handling
Enhanced Error Middleware Visualization (#11806)—Appreciation to sevensolutions for improved StatusRewrites option display

Plugin Development Evolution

Plugin developers gain more control and safety with the ability to enable unsafe operations through plugin manifests (#11589). Thanks to Rydez for expanding the plugin ecosystem's capabilities.

A Toast to Our Exceptional Community

Like the careful aging process that creates Chabichou's distinctive character, Traefik v3.5 is the result of months of dedication from our incredible community. Every bug report, feature request, code contribution, and documentation improvement has shaped this release into something truly special.

To every developer who opened an issue, every contributor who submitted a pull request, every user who provided feedback, and every community member who helped others—you are the artisans behind Traefik's continued excellence. Your passion and expertise continue to drive innovation in cloud-native networking.

Want to be a part of this awesome community? Join the discussions on our Community Forum or on GitHub where ideas flow as freely as conversations in a French café!

Essential Resources and Next Steps

Download Traefik 3.5: GitHub Releases | Docker Hub
Documentation: Complete Documentation
GitHub Repository
Community Forum
Learn More: Official Website | Blog

]]>

Simplifying Enterprise Connectivity: Traefik Labs Expands Microsoft Partnership with Azure Arc and AKS

Sudeep Goswami — Wed, 14 May 2025 12:32:48 GMT

If you’re steering an enterprise through today’s hybrid and multi-cloud landscape, you’re likely grappling with a paradox: the flexibility of distributed systems fuels innovation, but it also breeds complexity that can stall progress. APIs and AI workloads are the engines of modern business, yet managing them across on-premises data centers, public clouds, and edge locations often feels like navigating a maze.

At Traefik Labs, we’re partnering with Microsoft to transform this challenge into an opportunity, delivering a Kubernetes-native platform that unifies API and AI connectivity via Azure Arc, Azure Kubernetes Service (AKS), and the Azure Marketplace.

Our mission? To simplify your infrastructure, secure your workloads, and accelerate your future.

The Distributed Enterprise Dilemma

The shift to distributed architectures has unlocked unprecedented agility, but it’s also exposed critical pain points that enterprises can’t ignore. Fragmented API management forces teams to wrestle with manual and UI-centric configurations, leading to security blind spots and operational inefficiencies. AI workloads, with their unique demands for specialized routing, caching, and security, strain legacy platforms, causing performance bottlenecks and architectural conflicts. Inconsistent authentication and policy enforcement create compliance risks, while disruptive migrations to modern API management drain resources and impact customer experiences.

These aren’t just technical hurdles—they’re strategic barriers to innovation and growth. As AI becomes a cornerstone of business transformation, enterprises need a connectivity platform that scales responsibly, governs effectively, and evolves without disruption. That’s where our expanded partnership with Microsoft comes in, offering a unified approach to conquer complexity and empower your teams.

Traefik and Microsoft: A Vision for Unified Connectivity

With over 3.3 billion downloads, Traefik Proxy is a cornerstone of cloud-native connectivity. Our collaboration with Microsoft, strengthened through the Azure Arc ISV Partner Program, integrates this expertise with Azure’s ecosystem to deliver seamless API and AI management across any infrastructure. By combining Traefik’s Kubernetes-native simplicity with Azure Arc and AKS, we’re redefining enterprise connectivity.

Let’s explore how.

Azure Arc: Seamless Connectivity Across Any Environment

Azure Arc enables you to manage Kubernetes clusters anywhere—on-premises, in multiple clouds, or at the edge—through a single control plane. Traefik builds on this foundation, providing a cloud-native application proxy that ensures consistent traffic management and governance across all Arc-enabled environments.

Imagine deploying zero-config API routing that eliminates manual setup, reducing operational overhead from day one. Your security policies are centralized and powered by TLS and AI-specific content guard, ensuring compliance without complexity. You’re even able to optimize AI workloads with semantic caching, slashing latency and computational costs.

This isn’t just about connectivity—it’s about freeing your teams to innovate. With Traefik for Azure Arc, you gain a platform that adapts to your infrastructure, whether you’re running Kubernetes in a data center or scaling AI models at the edge. Our declarative, GitOps-driven approach integrates seamlessly with your CI/CD pipelines, making infrastructure management as agile as your development processes.

Azure Marketplace: Streamlined AKS Deployment

Since joining the Azure Marketplace in 2023, Traefik Proxy has empowered AKS users with one-click deployment, integrated billing, and instant scalability. As Microsoft’s recommended ingress controller for AKS baseline architecture, Traefik delivers unmatched compatibility and performance. You can set up advanced traffic management—supporting modern protocols like HTTP/3 and gRPC—in minutes, not hours. GitOps automation ensures your operations are declarative and scalable, while real-time traffic routing based on service health keeps your applications resilient.

For DevOps teams, this means less time configuring infrastructure and more time delivering value. Whether you’re modernizing microservices or launching AI-driven APIs, Traefik on AKS provides a foundation that’s both powerful and intuitive, aligning your Kubernetes strategy with business goals.

A Progressive Journey for Every Enterprise

Whether you’re a retailer streamlining e-commerce APIs, a healthcare provider deploying AI diagnostics, or a SaaS company scaling customer-facing services, your enterprise has unique needs and timelines. That’s why Traefik, integrated with Azure Arc and AKS, offers a three-phase adoption journey that meets you where you are and scales strategically—without disruption or vendor lock-in. This approach ensures immediate wins and long-term success, aligning technology with your business goals.

Phase 1: Application Proxy—Lay the Foundation

For teams adopting Kubernetes or managing microservices, our Application Proxy delivers zero-config service discovery, eliminating manual traffic management. Azure Arc enables deployment across on-premises and cloud clusters, while AKS users benefit from one-click setup via the Azure Marketplace. Real-time traffic routing ensures resilience, and Kubernetes CRDs simplify configuration. This phase slashes setup time, letting you launch new services with minimal resources.

Phase 2: API & AI Gateway—Secure and Optimize

As complexity grows, our API & AI Gateway unifies security and performance. Azure AD integration secures APIs and AI workloads, while semantic caching and content guard optimize AI traffic, cutting latency and ensuring compliance. Advanced traffic shaping—circuit breaking, retries—maintains stability. Azure Arc’s control plane enforces consistent policies across environments, delivering secure, high-performance connectivity for your distributed systems.

Phase 3: API Management—Drive Strategic Value

When APIs and AI are core to your business, our API Management phase empowers you with full lifecycle governance. Self-service developer portals accelerate innovation, unified observability provides actionable insights, and controlled versioning ensures smooth transitions. Azure Arc enables governance across all environments, transforming APIs into revenue-driving assets. This phase positions you to lead with data-driven services.

This journey is your path to agility and growth. By starting with simple routing, scaling to secure AI optimization, and achieving enterprise-grade governance, Traefik and Microsoft empower you to innovate without boundaries.

The Future Is Unified and AI-Ready

AI is reshaping competition, but scaling it requires infrastructure that balances performance, cost, and compliance. Traefik’s AI Gateway, integrated with Azure Arc, delivers intelligent semantic caching, context-aware routing, and robust governance for responsible AI. These innovations position you as a leader in a data-driven world.

Our partnership with Microsoft eliminates silos, accelerates API delivery, and unlocks revenue streams. This is the future of connectivity: unified, scalable, and AI-ready.

Take the Next Step

Ready to simplify your API and AI connectivity? With Traefik and Microsoft, you have a partner to streamline, secure, and scale your workloads. Here’s how to start:

Deploy on AKS: Launch Traefik via the Azure Marketplace for a seamless deployment experience.
Deploy on Azure Arc: Launch Traefik for Azure Arc via the Azure Marketplace and experience the “Any Infrastructure, Any Kubernetes” vision.
Explore Key Use Cases through Azure Jumpstart Drops: Discover how Traefik and Azure Arc redefine Kubernetes management, through bite-sized use-cases modules.
Dive Deep with a Self-guided Hands-on Lab: Walk through the entire three-step journey through a series of self-guided hands-on lab tutorials

At Traefik Labs, we’re passionate about helping you build a future where connectivity is seamless and innovation knows no bounds. Let’s take this journey together.

]]>

5 Smart API Gateway Strategies to Unlock Developer Productivity

Immánuel Fodor — Mon, 12 May 2025 11:09:19 GMT

How forward-thinking platform teams are transforming developer experience through modern API gateway practices

Developers expect speed and autonomy—and organizations demand control and security. The API gateway sits at the heart of this balancing act. The most successful platform teams are rethinking traditional gateway models, using modern strategies to supercharge developer productivity while maintaining ironclad governance, observability, and performance. Here’s how they’re doing it.

The Hidden Cost of Undocumented Infrastructure

Before diving into our strategies, allow me to share a personal experience that many platform leaders can relate to. When taking over leadership roles at several organizations, I've encountered the substantial challenge of undocumented infrastructure at scale.

The pain of uncovering what actually exists in data centers is like navigating through the "fog of war" in a strategy game like Age of Empires—spending significant time, effort, and money just to understand what you're working with. In the game, you start with a dark, foggy world, slowly revealing the map as you move forward. But for me, this wasn't a game—it was our production data centers. Every inch of fog essentially cost us time, money, and sometimes even sleep.

Fog of War in Age of Empires: https://altarofgaming.com/age-of-empires-civilizations/

I've even seen teams terrified of restarting their production gateway—not because of potential downtime, but because nobody knew what was configured there, what rules it was enforcing, or what custom configurations existed only in memory without any documentation or persistence. The entire setup was extremely fragile.

This painful experience taught me something important: we needed to stop firefighting and start thinking like a platform team. I had to build a team that could shift the mindset from reactive operations to proactive engineering.

When it comes to exposing APIs, frustration typically hits two key personas first: the API publisher who builds, owns, and exposes APIs, and the API consumer who uses these services. The gap between them requires governance that doesn't limit but empowers productivity. Here are five proven strategies that can transform your API gateway into a developer productivity powerhouse:

Strategy #1: Implement Zero-Friction API Onboarding

Reducing API integration time from days to minutes isn't just a dream—it's achievable with the right approaches. Here's a surprise, though: when we think about onboarding, we usually think about onboarding developers. But actually, the first persona to face onboarding challenges is the API publisher.

When new team members join or when onboarding a new API into API management, it often takes days, involves tickets, and relies on tribal knowledge. This is where the whole system can break down. By focusing on streamlining the publisher experience first, we create a foundation for developer productivity.

How Traefik Labs makes this possible:

Declarative configuration that allows for version-controlled API definitions
UI vs. Cluster ownership balance - whatever you create from the UI becomes a Custom Resource Definition (CRD) that you can manage through GitOps, making onboarding APIs repeatable and protecting configurations from accidental modification
Managed subscriptions to automate consumer onboarding, potentially eliminating the need for consumers to log into developer portals

It's easy to focus solely on the developer experience—but the API consumer is just as critical. Whether it's a partner integrating your APIs or an internal team building on top of your services, they need instant clarity, autonomy, and confidence. Yet, too often, consumers are left waiting on documentation, meetings, or manual approvals. The real transformation happens when consumers can discover, evaluate, and subscribe to APIs without delays. By removing these bottlenecks, organizations can accelerate time-to-value for every new integration.

Self-service API portal that includes:
- Comprehensive API specifications
- Self-service subscription options with flexible plans
- Try-out features for testing
- AI-assisted client code generation through the Alfred AI assistant

The goal is to achieve onboarding without meetings and waiting times—reducing API integration from days to minutes. This combination dramatically reduces the friction that typically slows down API integration and adoption for both publishers and consumers.

Strategy #2: Create Intelligent Observability Feedback Loops

Combining observability with your API gateway provides developers with contextual insights that dramatically reduce debugging time. Your API gateway serves as the gatekeeper of your infrastructure—securing, monitoring, and exposing APIs—making it essential to have visibility into what's happening at runtime.

Traefik Labs' approach includes:

OpenTelemetry integration that provides metrics, traces, and logs in a unified format to the tools you're most familiar with, whether self-hosted or cloud-based SaaS offerings
Observability-as-a-Service capabilities through our partnership with Treblle—with a simple plugin installation, you get instant insights into API behavior, compliance issues, and security problems based on observed traffic
Response debugging tools that provide granular visibility into API behavior

The real power comes from creating feedback loops that correlate deployment events with actual output metrics, traces, and logs. When you configure everything as code, put it in Git, and apply changes through GitOps pipelines, you can quickly identify the cause of unexpected issues. For example, if you suddenly see 500 errors, you can correlate them with recent PRs, which appear as annotations on your monitoring dashboard. This allows you to revert problematic changes quickly, mitigate incidents, and save the business from costly outages.

These tools transform debugging from reactive to proactive, allowing developers to move faster while resolving incidents before they escalate. The result? Significant reductions in time to recover from outages and overall cost savings for the business.

Strategy #3: Deploy Runtime Governance That Empowers Rather Than Restricts

Runtime governance needs to shift left while still empowering developers. Although governance puts checks in place, these should provide safe guardrails within which teams can move quickly—not restrictive barriers that slow innovation.

Traefik Labs enables:

Static analysis of your configuration in GitOps pipelines to catch configuration errors early and fix them before reaching production
Impact analysis that shows not just what changed but who will be affected by the change—for example, determining if a rate limit or quota configuration modification will impact more users than anticipated
Pull request workflows are enhanced with these semantic outputs that help reviewers understand both the change and its impact
Open Policy Agent (OPA) policies on CRDs for flexible, powerful governance without requiring a separate OPA installation—it's built directly into the gateway
OpenAPI Specification (OAS) validation to eliminate shadow and zombie APIs, which according to Treblle's latest report, account for 35% of API traffic

Traefik API Management allows you to set a flag to block every endpoint outside what's described in the OpenAPI specification. When enabled, all undocumented methods become inaccessible immediately. For legacy systems with undocumented APIs, our partnership with Treblle allows you to build OpenAPI specifications from observed traffic, transforming previously undocumented endpoints into documented ones.

# Also works with API Versions
---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: invoice-api
spec:
  openApiSpec:
    path: /openapi.yaml
    validateRequestMethodAndPath: true  # one simple setting, much less trouble

This approach shifts governance left in the development cycle, preventing issues before they become problems while preserving developer autonomy. It's like having a Swiss army knife for governance—you're not limited to what the tool offers but have an engine you can fine-tune to create safeguards that enable developers to move faster.

Strategy #4: Tame LLMs and AI by Turning Them Into Managed APIs

As AI becomes central to development workflows, new challenges are emerging. The AI landscape is evolving at an unprecedented pace—according to Stanford University's latest study, AI is becoming more powerful each day and is being embedded into everyday life with exponential growth in areas like medical devices. Organizations are going all-in on AI, with various models available worldwide.

With AI endpoints proliferating (Hugging Face alone hosts over 1.6 million models), companies and developer teams are experimenting extensively. This creates a fundamental need for API management and governance specifically tailored to AI workloads.

Traefik Labs' AI Gateway solution includes:

Centralized secret management that can turn any AI endpoint into a managed API—you no longer need to expose your OpenAI API key to all your mobile applications deployed to the public
Semantic caching that recognizes when prompts express the same meaning in different natural language sentences, freeing up costly GPU cycles while dramatically improving latency and reducing costs
Content firewalls that protect information flowing to and from AI models, with the ability to block requests/responses or mask sensitive data through a powerful rule-based system using Microsoft's open-source Presidio engine for natural language processing

Our demo shows this in action with a "Traffic Airlines Feedback Service" that uses JWT authentication, OPA middleware for authorization, and content guards to cleanse PII data flowing through the system. We also demonstrate semantic caching with our chat example, where semantically similar queries returned cached results with dramatically improved response times.

These capabilities allow organizations to harness the power of AI while maintaining control over how it's accessed and utilized—balancing innovation with responsible AI practices.

Strategy #5: Build Resilient API Ecosystems

Configuring API networks that reduce developer on-call burden and improve system reliability is essential for sustainable operations. With AI inferencing, additional challenges emerge around versioning, scalability, cost optimization, and handling sensitive data responsibly.

Traefik Labs focuses on:

High availability configurations to ensure consistent service
Flexible API versioning that decouples exposed API versions from model versions, allowing teams to experiment with different models without breaking client applications
The shared responsibility model for API and ML model versioning

Traefik Hub API Management offers an exceptionally flexible versioning system that can match on host, HTTP method, query parameters, headers, IP, authenticated user, and many other request attributes. This puts power in your hands to manage the lifecycle of APIs and AI workloads with less chaos and fewer incidents.

For example, your API might expose a "/v2" endpoint while the model running behind it could be at version 2.2 or 2.3. This decoupling allows you to experiment with and improve models without disrupting clients. With contracts and versioning present as code, you provide a consistent view to clients while enabling experimentation within your organization.

This approach creates resilient systems that can evolve while maintaining compatibility and reducing maintenance overhead—essential for managing both traditional APIs and increasingly important AI workloads.

Putting It All Together: From Theory to Practice

These strategies aren't just theoretical, they represent practical approaches that forward-thinking organizations can implement today. Traefik Labs demonstrates a complete infrastructure-as-code environment with Terraform, deploying a K3d cluster with Argo CD, Traefik, KServe for AI inferencing, and various other tools—all fully configurable through GitOps workflows.

We demo two key services:

A Traefik Airlines Feedback Service that uses JWT middleware, OPA policies, and content guards to provide secure, compliant AI-powered sentiment analysis
A Chat Service that implemented semantic caching to dramatically improve performance and reduce costs when handling similar queries

Unlocking What’s Next: Developer Productivity Starts at the Gateway

By transforming your API gateway from a simple traffic management solution into a comprehensive platform for developer productivity, you can accelerate innovation while maintaining the control and visibility your organization needs.

Whether you're managing a handful of APIs or thousands, or working to integrate AI capabilities responsibly into your applications, these strategies can help you create a more productive, secure, and resilient API ecosystem that drives business value.

Ready to unlock developer productivity through your API gateway? Learn more about how Traefik Labs can help transform your approach to API management and developer experience in 2025 and beyond.

]]>

Traefik Proxy 3.4 “Chaource” Is Ready to Serve

Emile Vauge — Tue, 06 May 2025 11:51:53 GMT

Traefik Proxy 3.4 is here, and it’s greater than ever! Codenamed “Chaource” 🧀 (a creamy delight from Champagne, France, has been crafted since Roman times), this release is packed with zesty new features, smoother operations, and a sprinkle of magic to keep your infrastructure running like a well-oiled fondue pot. Let’s cut through the rind and dig into the goodness!

Key New Features

Distributed Rate-Limiting with Redis

“Sharing is caring” just got a whole new meaning! Traefik’s new distributed rate-limiting (#10211 by longquan0104) lets you enforce traffic rules globally across instances using Redis. No more rogue services hogging bandwidth—this feature ensures fair play, even in a multi-node setup. Perfect for stopping API abuse or managing traffic spikes without breaking a sweat.

Here is an example to deploy a rateLimit middleware in Kubernetes:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
   name: test-ratelimit
spec:
   rateLimit:
      # ...
      redis:
         secret: mysecret

---
apiVersion: v1
kind: Secret
metadata:
   name: mysecret
   namespace: default
data:
   username: dXNlcm5hbWU=
   password: cGFzc3dvcmQ=

P2C (Power of Two Choices) Load-Balancing

Meet the Power of Two Choices (P2C) load balancer—your new traffic whisperer. Instead of blindly picking a backend, P2C intelligently selects two random candidates and routes requests to the least busy one (initiated by ifross89 in #10534). Under heavy traffic, P2C avoids overloading a single server “by accident”. This results in fewer bottlenecks and a steadier, more predictable load distribution.

Enabling P2C is easy. In your service’s load-balancer config, set the method to p2c instead of the default. For example:

http:
  services:
    my-service:
      loadBalancer:
        servers:
          - url: "http://10.0.0.1"
          - url: "http://10.0.0.2"
          - url: "http://10.0.0.3"
          - url: "http://10.0.0.4"
        method: "p2c"

Custom Server URL with Labels

A long awaited feature is part of this release, Chaource lets you define server URLs via labels on Docker, Swarm, ECS, Consul & Nomad (#11374 by yelvert). This feature is needed in case you don’t want to use the container’s IP but a custom URL instead.

Here is a Docker compose example deploying an app and setting custom server urls:

version: '3.8'
services:
  main:
    image: lorem/ipsum:latest
    deploy:
      replicas: 3
      labels:
        - "traefik.enable=true"

        - "traefik.http.routers.service1.entrypoints=https"
        - "traefik.http.routers.service1.rule=Host(`foobar1.example.com`)"
        - "traefik.http.routers.service1.service=foobar1"
        - "traefik.http.routers.service1.tls=true"
        - "traefik.http.services.service1.loadbalancer.server.url=http://foobar1:80"
        
        - "traefik.http.routers.service2.entrypoints=https"
        - "traefik.http.routers.service2.rule=Host(`foobar2.example.com`)"
        - "traefik.http.routers.service2.service=foobar2"
        - "traefik.http.routers.service2.tls=true"
        - "traefik.http.services.service2.loadbalancer.server.url=http://foobar2:80"

        - "traefik.docker.network=traefik"

More Delicacies

ACME Enhancements: Introduced new options acme.profile and acme.emailAddresses to customize certificate requests (#11597 by ldez). See the ACME documentation for details.
Kubernetes Improvements: Improved CEL validation for CRDs (#11311 by mloiseleur), ingress statuses were added for ClusterIP and NodePort Service Type (#11100 by mlec1) and we can now get a root CA to be added through config maps (#11475 by Nelwhix). Refer to the Kubernetes Ingress documentation.
Gateway API: It’s now possible to set rule priority in Gateway API TLSRoute (#11443 by augustozanellato)
Middlewares: An option preserveRequestMethod has been added to the ForwardAuth middleware to preserve the original HTTP method and to send the request body to the authorization server (#11473 by an09mous). See the ForwardAuth middleware documentation. The ErrorPage middleware now supports rewriting status codes dynamically (#11520 by sevensolutions). Check out the ErrorPage middleware documentation.
Sticky Sessions: You can now specify a cookie domain for sticky sessions (#11556 by jleal52). Refer to the Sticky Sessions documentation.
TLS Enhancements: Added options to disable TLS session tickets and to add extra CA certificates via Kubernetes ConfigMaps (#11609 by avdhoot). See the TLS documentation.
UDP Routing: Traefik can now route UDP traffic with systemd socket activation (#11022 by tsiid). Check out the UDP documentation.
Web UI: The dashboard now supports an auto theme option, switching between light and dark modes based on your system settings (#11455 by zizzfizzix). Refer to the Dashboard documentation.
Under the hood: experimental maps and slices have been replaced by stdlib (#11350 by Juneezee)

A Big Slice of Gratitude

Chaource wouldn’t be the same without our amazing community! To everyone who filed bugs, brainstormed features, or geeked out with us on GitHub—thank you! 🥂

Got questions? Compliments? Cheese puns? Share them in our Community Forum!

Useful Links

Traefik 3.4 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Who's in Charge? The Shared Responsibility Model for API and AI/ML Model Versioning

Sudeep Goswami — Thu, 17 Apr 2025 13:52:14 GMT

Your ML model just broke in production. Was it the API? The model? Or the versioning gap between them?

As organizations scale their AI operations, the traditional approach of handling versioning at a single layer is breaking down. This limitation has led forward-thinking enterprises to adopt what we call the "Shared Responsibility Model" for versioning, distributing responsibilities between API management platforms and ML serving platforms.

The Growing Complexity of AI at Scale

As machine learning transitions from experimental projects to business-critical applications, organizations face increasing complexity in managing both the APIs exposing AI capabilities and the underlying ML models themselves. Effectively deploying and managing these solutions isn't just a data science puzzle; it's a complex challenge involving engineering, IT operations, and business strategy, with Deloitte noting that getting AI into production remains difficult[¹]. This challenge stems from the unique dual nature of ML systems: they are both software services defined by API contracts and statistical models requiring frequent updates, each with distinct lifecycle requirements.

Getting this right extends far beyond technical elegance; it directly impacts customer experience, operational efficiency, and the ability to innovate rapidly in an increasingly AI-driven marketplace.

With 78% of organizations reporting AI use in 2024, up significantly from 55% the previous year, the need for effective versioning strategies has never been more critical.

The Scale and API Management Challenge

One major factor amplifying this complexity is the sheer proliferation of AI models. Platforms like Hugging Face now host over 1.4 million model repositories (as of April 2025), reflecting an explosion of options available for teams to test and deploy. This rapid expansion underscores the versioning challenge.

It also leads directly to a critical related insight: The more AI you deploy at scale, the more APIs you have. And that creates an API management problem. We're witnessing a "Cambrian explosion" of AI APIs, as organizations increasingly expose AI capabilities as modular services. Managing this API sprawl, including shadow APIs created outside governance, is crucial; failing to do so can thwart AI initiatives due to data issues, security risks, and integration failures. Furthermore, AI APIs present unique management needs around aspects like token-based rate limiting, prompt handling, and multi-model routing.

The Shared Responsibility Model Explained

The Shared Responsibility Model distinguishes between client-facing API contracts and underlying model implementations, creating a clear boundary of concerns that addresses the unique challenges of AI systems at scale.

API Management Layer Responsibilities:

Contract Stability: Maintains consistent interfaces that clients can rely on regardless of backend changes
Access Control & Security: Implements authentication, authorization, and data protection at the API boundary
Traffic Management: Handles rate limiting, throttling, and request distribution
Observability: Provides API-level metrics, logging, and monitoring
Documentation & Developer Experience: Offers consistent, up-to-date API specifications and documentation
Versioning Strategy: Manages API versions and deprecation policies

ML Serving Layer Responsibilities:

Model Lifecycle Management: Handles model training, validation, deployment, and retirement
Model-Specific Versioning: Manages rapid iteration of model weights, architectures, and parameters
Inferencing Infrastructure: Optimizes compute resources for model serving
Model Monitoring: Tracks performance, accuracy, and cost metrics
Canary Deployments: Implements gradual rollout of new model versions with automated rollback
Feature Management: Maintains repositories of model input variables (features) and ensures consistent transformations across training and serving

By clearly delineating these responsibilities, organizations can achieve a balance between stability and innovation. The API layer provides a consistent contract that shields consumers from the complexity and frequent changes occurring in the ML layer, while the ML layer retains the flexibility needed for rapid experimentation and improvement. This separation of concerns allows each layer to follow its own appropriate versioning cadence and practices.

Bridging MLOps and APIOps for End-to-End Scalability

Effectively managing the ML layer requires robust MLOps practices. However, since most AI/ML capabilities are consumed via APIs, focusing only on model lifecycle management isn't enough for true scalability. This is where APIOps becomes essential – applying DevOps principles to the entire API lifecycle (design, development, testing, deployment, monitoring, retirement)[²].

The relationship between these two operational domains is crucial:

MLOps focuses on model development, training, and deployment
APIOps manages how these models are exposed to consumers

These systems must work in harmony. Even the most efficient MLOps pipeline will create bottlenecks if your API management can't keep pace. When your data science team develops a new fraud detection model version, the API exposing that capability must be ready to handle the change without disrupting consumers.

As AI adoption accelerates, the number of AI-powered APIs within an organization multiplies rapidly. Without robust APIOps practices ensuring consistency and automation in API management, you won't realize the full agility benefits promised by your MLOps investments.

The Kubernetes-Native Advantage: Bridging API and Model Versioning

Kubernetes provides the ideal foundation for implementing the Shared Responsibility Model and coordinating MLOps and APIOps in practice. Its declarative nature, robust service discovery, and native resource definitions allow different specialized tools to manage their respective layers while integrating seamlessly. Here's how it bridges API and ML model versioning:

API Versioning (The Contract): An API management platform like Traefik Hub leverages Kubernetes Gateway API or Ingress resources to define and manage the stable API contracts exposed to consumers. It can handle different API versions concurrently – for example, routing requests based on the URL path (/v1/predict vs. /v2/predict), headers (API-Version: 1 vs. API-Version: 2), or more. Each API version defined in the gateway routes to a specific, stable Kubernetes Service name. This ensures clients always interact with a consistent API contract, regardless of the underlying model changes, aligning with standard API versioning techniques documented for platforms like Traefik Hub.

ML Model Versioning (The Implementation): Behind that stable Kubernetes Service endpoint managed by the API gateway, an ML serving platform like KServe manages the actual model deployments using its own Custom Resource Definitions (CRDs), such as the InferenceService. When a new ML model version needs to be rolled out (e.g., fraud-detection-v2.1 replacing v2.0), KServe can manage this transparently to the API consumer. Using its CRD, you can specify that only a small percentage of traffic (e.g., 10% via the canaryTrafficPercent field) should initially hit the new model version (v2.1), while the stable v2.0 handles the rest. KServe manages this traffic splitting internally through its integration with Kubernetes networking capabilities.

The AI Gateway Advantage: Between the API Management and ML Serving layers, a specialized AI Gateway addresses unique challenges of AI-specific workloads. Traefik's AI Gateway provides crucial capabilities that enhance security, performance, and governance—ranging from semantic caching that optimizes response times to content protection that ensures compliance. This middle layer bridges the gap between stable API contracts and dynamic model implementations, providing AI-specific optimizations that neither the API layer nor the ML layer can deliver alone.

Connecting Theory to Practice: This demonstrates the Shared Responsibility Model in action on Kubernetes. Traefik Hub handles the external API contract and its versions (/v1, /v2), directing traffic to the appropriate stable backend service endpoint. The AI Gateway provides critical AI-specific optimizations and protections, while KServe manages the ML model versions served by that endpoint, handling canary rollouts (v2.0 vs. v2.1) without disrupting the API contract defined by the Traefik Hub API Management layer. Kubernetes acts as the orchestration layer, enabling these specialized tools to manage their respective responsibilities cohesively.

Business Impact: Why This Matters

Getting this layered, Kubernetes-native approach right directly impacts the bottom line:

Faster Time-to-Value: By separating concerns and streamlining workflows across both MLOps and APIOps, businesses can significantly accelerate innovation. AI, managed effectively, can cut product development timelines by up to 50%, while MLOps practices can slash model deployment times by 30-50%[³], allowing faster response to market needs.

Enhanced Efficiency & Productivity: CEOs are already seeing tangible results, with 56% reporting efficiency gains from GenAI. Organizations effectively leveraging AI can achieve significantly higher productivity growth.

Improved Stability & Trust: The model allows underlying AI capabilities to be updated without breaking crucial client integrations or compromising backward compatibility. Robust management also helps mitigate risks associated with AI, such as inaccuracy and security vulnerabilities.

Future-Proofing Your AI Strategy

The convergence of API management and AI delivery is fundamentally reshaping how organizations innovate and compete. Successfully scaling AI requires more than just adopting algorithms; it demands a strategic approach to managing the interplay between APIs and models. Adopting a clear strategy like the Shared Responsibility Model is essential for navigating this complexity and unlocking the full potential of your AI investments.

This is where Traefik Hub, as a Kubernetes-native API management solution, plays a pivotal role. Built for the modern stack, it provides the sophisticated routing, security, governance, and visibility needed to expertly manage the crucial API management layer – facilitating the robust APIOps practices required to govern the "Cambrian explosion" of AI APIs within your Kubernetes environment.

The inclusion of specialized AI Gateway capabilities extends this value proposition further:

Semantic Caching: Unlock Scalable, Cost-Effective AI: When AI systems scale from experimentation to production, computational costs and latency can become major hurdles. Traditional caching methods often fall short, especially for resource-heavy large language models. Traefik's AI Gateway addresses these challenges with Semantic Caching, which uses advanced embedding techniques to understand the meaning behind queries, enabling intelligent reuse of results for similar requests.

This approach delivers tangible benefits:

Dramatically reduces response times from seconds to milliseconds for semantically similar queries
Eliminates redundant, costly computations for common request patterns
Optimizes resource allocation by serving cached responses for similar queries while dedicating computing power to truly novel requests
Integrates seamlessly with popular vector databases in cloud-native environments

Content Guard: Enabling Secure, Compliant AI: As AI powers increasingly critical business functions, ensuring proper governance becomes essential. Content Guard provides superior protection by safeguarding sensitive data and ensuring both inputs and outputs meet ethical and regulatory standards. Unlike basic solutions, it leverages contextual natural language processing for highly accurate detection across data types.

With over 35 predefined PII recognizers and customizable rules, Content Guard ensures:

Comprehensive detection and de-identification of sensitive information
Consistent policy enforcement across distributed AI deployments
Compliance with evolving regulatory requirements
A foundation for responsible AI that builds user trust

Together, these capabilities empower platform teams to provide stable, secure access while ML and application teams innovate rapidly behind the scenes using MLOps.

Taking Action: Next Steps for Your Organization

How is your organization preparing for the operational realities of scaling AI? Here are three concrete steps you can take today:

Assess Your Current State: Evaluate your existing API management and ML serving capabilities against the Shared Responsibility Model. Identify gaps in your versioning strategy that could impede AI scaling.
Build Cross-Functional Alignment: Bring together your API platform teams and ML engineering teams to establish clear boundaries of responsibility and collaborative workflows.
Implement a Pilot Project: Select a non-critical AI service and implement the Shared Responsibility Model using Kubernetes-native tools. Document learnings and establish patterns that can be scaled across your organization.

Ready to learn more? Contact the Traefik sales team for a personalized consultation on how the Shared Responsibility Model can be implemented in your organization using Traefik Hub.

Ultimately, embracing the Shared Responsibility Model, implemented through coordinated MLOps and APIOps practices, fosters better collaboration across diverse teams – engineering, operations, data science, and business – creating a scalable and resilient foundation for delivering AI-driven value. Investing in the right operational architecture today, with robust API management at its core, is investing in your organization's agility and innovation velocity for tomorrow.

References

[¹] Deloitte. "AI in Enterprise Applications: Challenges and Opportunities." https://www2.deloitte.com/us/en/insights/focus/signals-for-strategists/ai-in-enterprise-applications.html

[²] Microsoft Learn. "API Design Best Practices." https://learn.microsoft.com/en-us/azure/architecture/microservices/design/api-design

[³] Cogent Infotech. "MLOps Benefits: Deployment Time." https://www.cogentinfo.com/blog/mlops-benefits-deployment-time/

]]>

Traefik vs. #IngressNightmare: Security By Design in the Age of Critical Vulnerabilities

Sudeep Goswami — Thu, 10 Apr 2025 10:46:09 GMT

The Wake-Up Call Our Industry Needed

When our security team first alerted us about the critical vulnerabilities disclosed in ingress-nginx on March 24th, 2025, we immediately understood the gravity of the situation. CVE-2025-1974, with its devastating 9.8 CVSS rating, represents one of the most significant Kubernetes security events in recent years.

The implications are sobering: any entity within the pod network could potentially seize control of your entire Kubernetes cluster—without credentials or administrative privileges. Let that sink in for a moment.

While the ingress-nginx team deserves credit for rapidly releasing patches (v1.11.5 and v1.12.1), the discovery of more than 6,500 vulnerable clusters in production environments remains deeply concerning. This isn't just another vulnerability—it's a fundamental architectural flaw that demands our industry's immediate attention.

Understanding #IngressNightmare: Beyond the Headlines

To truly appreciate the severity of these vulnerabilities, we need to examine the technical mechanics at play. The core issue stems from how ingress-nginx processes ingress objects through its admission controller.

When validating an ingress object, the controller constructs an NGINX configuration and validates it using the nginx -t command. The Wiz Research team brilliantly uncovered multiple configuration injection vulnerabilities (CVE-2025-24514, CVE-2025-1097, CVE-2025-1098) that allow attackers to inject arbitrary NGINX configuration directives by sending malicious ingress objects to the admission controller.

While these injection points alone don't immediately grant code execution, the researchers discovered that the ssl_engine directive could be exploited to load shared libraries at any point in the configuration. By combining this with a method to upload a shared library to the pod's filesystem, attackers can achieve remote code execution within the ingress-nginx controller's pod.

Most critically, because admission controllers typically run with elevated privileges and unrestricted network access, successful exploitation allows attackers to execute arbitrary code and access all cluster secrets across namespaces—potentially leading to complete cluster compromise.

Why Traefik Remains Secure: Architectural Decisions Matter

As the founder and developer of Traefik, Emile Vauge made fundamental architectural decisions a decade ago that have proven critical for security. These weren't accidental choices—–they were deliberate design principles that we've maintained throughout Traefik's evolution:

No Raw Configuration Templating: Unlike templated proxies, Traefik parses configuration inputs (Ingress, IngressRoute, Middlewares, Custom Resources, or Gateway API resources) into strongly typed Go structs. Without template mechanisms, there's simply no string injection path for attackers to exploit.
Minimal Network Surface Area: Traefik doesn't implement an admission controller or any component with unrestricted network accessibility. It simply reads ingress resources and applies them if correct—if they can't be parsed correctly, they're ignored with minimal error logging. No configuration is ever executed or interpreted.
No Dynamic Library Loading: Traefik is written in Go, producing statically linked executables by default. We deliberately compile without CGO options, making it impossible to execute code not already part of the binary. Unlike NGINX's ssl_engine directive, there's no mechanism to load external shared libraries.

These architectural principles weren't arbitrary—they were security-first decisions that have stood the test of time.

The Industry's Crossroads: Why Wait Years for a Solution?

It's telling that the ingress-nginx team recently announced plans to pivot toward developing InGate—a new Ingress/Gateway API controller written in Go. They've wisely stated that no new features will be added to ingress-nginx during this transition, with the ultimate goal of deprecating it entirely once InGate reaches GA status.

While this represents a thoughtful long-term strategy, the stark reality is that reaching stability with a comparable feature set will likely take years. With only 18 commits to InGate at the time of writing, the journey has barely begun.

The Path Forward: Security Cannot Wait

For organizations running Kubernetes in production today, the message is clear: you shouldn't wait years to address a critical security vulnerability that exists now.

The immediate priority is upgrading existing ingress-nginx deployments to patched versions (v1.11.5 or v1.12.1). However, this is merely a tactical response to a strategic problem.

The more prudent approach is migrating to an ingress controller that's secure by design. At Traefik Labs, we've spent a decade building and refining Traefik Proxy with security as a foundational principle—not an afterthought. Our architectural decisions made ten years ago—choosing a statically linked toolchain, a strongly typed language, and eliminating templating—have proven prescient in light of today's threats.

Conclusion: Design Principles Matter

As leaders in the cloud-native ecosystem, we believe #IngressNightmare offers a valuable lesson for our industry. Security cannot be bolted on—it must be built in from the ground up. The choices we make about programming languages, compilation methods, and configuration approaches have profound security implications that may not become apparent for years.

We're encouraged to see the ingress-nginx team recognizing these principles in their plans for InGate. By embracing Go, static linking, and strong typing, they're acknowledging the same architectural foundations that have kept Traefik secure for the past decade.

For organizations that can't afford to wait for InGate to mature, Traefik Proxy offers a battle-tested, secure-by-design alternative available today. Your security shouldn't be compromised while waiting for tomorrow's solutions.

Sudeep Goswami is the CEO of Traefik Labs, leading the company's global strategy and operations.

Emile Vauge is the Founder and CTO of Traefik Labs. He created Traefik, the popular open-source cloud-native application proxy, which has been downloaded over 3 billion times.

]]>

Traefik Labs’ API Management Solution Now Available on Oracle Cloud Marketplace

Traefik Labs — Thu, 03 Apr 2025 13:01:42 GMT

Read the original post on Oracle: https://blogs.oracle.com/developers/post/traefik-labs-api-management-solution-now-available-on-oracle-cloud-marketplace

We’re excited to announce that Traefik Labs’ API management solution is now available on Oracle Cloud Infrastructure (OCI) Marketplace. This cloud-native, Kubernetes-native platform enables enterprises to manage APIs seamlessly across hybrid and multi-cloud environments while maintaining security and performance requirements of enterprise customers.

With Traefik’s deploy-anywhere flexibility and OCI’s industry-leading security, performance, and cost efficiency, customers gain a comprehensive API management solution. Deployment is quick and seamless—Traefik’s solution can be launched directly from the OCI Marketplace in just a few clicks.

Powerful OCI Integrations to drive business value for OCI customers

Managing APIs across distributed environments is often challenging due to manual configurations that can lead to security inconsistencies, limited visibility, and high costs. The Traefik-OCI integration addresses these issues by combining Traefik’s modern API management capabilities with OCI’s robust ecosystem, reducing complexity, enhancing security, and accelerating innovation.

Key OCI Integrations:

Oracle Cloud Infrastructure (OCI) Kubernetes Engine (OKE)

Traefik seamlessly integrates with OKE, providing a Kubernetes-native API management experience designed for containerized applications.

Deploy Kubernetes-native API management on OKE’s managed clusters
Eliminate API silos with Traefik CRDs and OKE’s multi-cluster management
Autoscale API gateways dynamically with OKE’s flexible node pool architecture

OCI DevOps

This integration enables GitOps-driven automation, ensuring seamless API deployment and management with OCI’s CI/CD pipeline.

Automate API management with end-to-end GitOps-driven CI/CD
Deploy and scale Traefik API Gateways via OCI build pipelines
Sync Git-committed API configurations automatically to production environments
Enhance OCI’s blue-green deployments with Traefik’s zero-downtime routing

OCI Application Performance Monitoring (APM)

Gain deep observability into API traffic and performance with Traefik’s OpenTelemetry support combined with OCI APM.

Gain full-stack observability with Traefik’s OpenTelemetry support integrated into OCI APM
Monitor GenAI traffic patterns alongside traditional API metrics
Troubleshoot faster with detailed OpenTelemetry instrumentation

OCI Logging Analytics

Ensure optimal API security and performance by leveraging Traefik’s structured logs with OCI’s advanced analytics tools.

Stream Traefik’s structured logs into OCI’s analytics platform
Apply OCI’s ML-powered analytics to optimize API performance and security
Gain unified insights across all environments with tool-agnostic log integration

Key Benefits for OCI Customers

End-to-End API Lifecycle Management: Includes a developer portal, interactive API documentation, testing, mocking, monetization, security, governance controls, and analytics
GitOps-Enabled Automation: Version-controlled API definitions ensure consistency, zero-downtime updates, and automated scaling
Seamless OCI Integrations: Works with OKE, OCI DevOps, APM, and Logging Analytics for a smooth developer experience and to maximize the value gained out of OCI platform
Easy Deployment via OCI Marketplace: Deploy Traefik’s solutions easily using Terraform or Helm templates

More Choice, More Flexibility

OCI is committed to providing enterprises with a broad ecosystem of innovative solutions. With Traefik Labs, OCI customers now have the freedom to choose the best API management approach suited for their needs:

Oracle API Gateway: A fully managed, cost-effective API gateway ideal for services built on OCI. Link
Traefik Labs: A deploy-anywhere API lifecycle management solution with deep OCI integrations. It also offers an easy upgrade path from a standard Ingress Controller to a full-featured API management solution. Link

How to Get Started

Browse the OCI Marketplace – Search for "Traefik" and select your preferred option.
Choose Your Deployment Method – Deploy using Terraform or Helm with pre-validated OCI integrations.
Configure Your Environment – Connect to OCI services like OKE, DevOps, APM, and Logging Analytics.
Learn More – Visit this link to request a demo or to explore further.

This joint solution provides organizations with a comprehensive platform for building and managing cloud-native applications and APIs. By combining OCI's enterprise-grade infrastructure with Traefik's advanced API management capabilities, organizations can accelerate their digital transformation while ensuring security, compliance, and operational excellence whether in OCI, other clouds, or on-premises environments.

]]>

AI Gateways: The Missing Piece in Scalable & Responsible AI Inferencing

Sudeep Goswami — Thu, 27 Mar 2025 11:42:57 GMT

As AI solutions evolve from experimental prototypes to enterprise-critical deployments, organizations face mounting challenges in scalability, performance, and responsible delivery. While standard AI gateways offer essential routing, load balancing, and API management, truly scalable and responsible AI inference demands two advanced enhancements: semantic caching—intelligently storing and reusing responses for similar prompts—and content guard that filters data shared with AI models as well as AI-generated content against safety and compliance standards.

Our exploration builds upon foundational gateway functionality to address the unique challenges of enterprise AI deployment, providing organizations with comprehensive solutions for both performance optimization and responsible content delivery—deployable anywhere from centralized data centers to global edge locations.

Why AI Gateways Form Essential Infrastructure

Organizations deploying AI at scale recognize the value of AI gateways as a unified infrastructure layer managing inference requests. Core gateways provide:

Intelligent Routing: Directing requests to appropriate models and endpoints
Load Balancing: Distributing traffic efficiently across infrastructure
Request Management: Handling timeouts, retries, and concurrency control
Observability: Monitoring performance and operational health
API Standardization: Ensuring consistent interfaces across models
Governance Controls: Enforcing organizational policies, access controls, and compliance requirements consistently across all AI interactions

While addressing fundamental challenges of infrastructure fragmentation and API inconsistency, AI deployments that scale to mission-critical status create additional challenges requiring specialized gateway enhancements: the computational overhead of redundant inference and the need for consistent content moderation.

An AI strategy remains incomplete without a robust gateway. Organizations lacking this critical infrastructure component build on fundamentally unstable foundations. Yet even with basic gateway functionality, enterprises still face significant challenges with performance economics and responsible scaling.

Semantic Caching: Unlocking Inference Scalability

Computational costs quickly become a limiting factor when AI systems move from experimentation to production. Traditional horizontal scaling proves economically unsustainable for AI inference, particularly for large language models with significant computational requirements.

Semantic caching emerges as the critical solution to scalability challenges. Unlike traditional caching requiring exact matches, semantic caching leverages advanced embedding techniques to identify the underlying meaning of queries, enabling reuse of previously computed results for semantically similar requests. Semantic caching dramatically transforms the economics of AI deployment:

Reduced Computational Redundancy: Identifying semantic similarity avoids repeating expensive computations for equivalent requests
Dramatic Latency Improvements: Cached responses resolve in milliseconds rather than seconds
Cost-Effective Scaling: Resources focus on novel prompts while common patterns leverage cached results

Application in Financial Services

In financial services, semantic caching delivers exceptional value for customer-facing applications like chatbots and advisory tools. When implemented within AI gateways, organizations can expect:

Significant reduction in inference costs through intelligent response reuse
Response times improving from seconds to milliseconds
Increased capacity to handle peak loads without proportional infrastructure scaling
Consistent performance during high-traffic events like product launches or market volatility

The impact multiplies in distributed edge deployments, allowing organizations to efficiently scale inference capacity without additional hardware costs.

Content Guard: Foundation for Responsible AI Delivery

While performance challenges merely impede AI adoption, governance concerns can terminate projects entirely. The need for governance becomes especially critical when organizations deploy generative AI in customer-facing and high-stakes environments where inappropriate handling of data or outputs creates significant reputational or compliance risks.

Content guard addresses governance concerns by establishing a sophisticated safety layer within AI gateways that protects sensitive information shared with models and evaluates generated content to ensure compliance with ethical guidelines, industry standards, and regulatory requirements. This bidirectional approach safeguards the entire AI interaction flow, from inputs to outputs, creating a robust governance framework for responsible AI deployment.

For organizations with distributed inferencing operations, content guard delivers consistent policy enforcement across every deployment location while adapting to local requirements when necessary.

Application in Healthcare

In healthcare environments, content guard provides critical safeguards for both clinical and patient-facing AI applications. When implemented within AI gateways, healthcare organizations can:

Enforce HIPAA compliance through automated PII detection and redaction
Apply specialized medical safety filters to prevent potentially harmful recommendations
Maintain distinct policy sets for different user interfaces (clinician vs. patient)
Provide comprehensive audit trails documenting all content validations
Reduce manual compliance reviews, accelerating application deployment while improving safety

By providing consistent, documentable enforcement of organizational policies regardless of where inference occurs, content guard transforms AI from a compliance risk into a compliance-enhancing asset for enterprises in highly regulated sectors.

Deployment Flexibility: From Core to Edge

AI gateways represent a logically centralized control plane that excels across diverse deployment scenarios. The lightweight, high-performance architecture enables organizations to maintain consistent policies, interfaces, and behaviors regardless of where AI inference occurs—from centralized data centers to thousands of edge locations.

Deployment flexibility becomes increasingly valuable as AI inference requirements diversify. Organizations now deploy AI gateways to:

Optimize centralized data center operations for cost-efficiency at scale
Support hybrid architectures combining on-premises and cloud resources
Expand AI services to edge locations for reduced latency and data sovereignty
Enable consistent management across heterogeneous environments

Managing these varied deployments demands a code-first approach. As highlighted in our previous blog on a holistic API architecture, the complexity of distributed AI infrastructure makes traditional manual management fundamentally unsustainable.

The code-first operating model transforms AI infrastructure deployment and management through:

Infrastructure as Code: Gateway configurations, routing rules, and policies defined in version-controlled files
Declarative Management: Explicit definition of desired states, eliminating configuration drift
Automated Consistency: Automatic propagation of changes across distributed instances
GitOps Workflows: Changes reviewed, tested, and deployed through established pipelines
Audit and Compliance: Complete history of infrastructure changes and policy updates

For AI gateways operating at the edge, this approach must accommodate additional requirements:

Lightweight Deployment: Efficient operation in resource-constrained edge environments
Stateful Operations: Maintaining critical functions like semantic caching with minimal overhead
Consistent Policies: Enforcing global standards with local adaptations where required
Resilient Operations: Continuing to function during network disruptions

The code-first model maintains consistency across this distributed edge environment while adapting quickly to evolving requirements.

Integrated Gateway Solutions: Enhancing Performance and Compliance Everywhere

The full potential of AI gateways emerges when semantic caching and content guard operate together within a unified framework managed through code. This integration creates an optimized workflow regardless of deployment location:

Optimized AI Inference Workflow Anywhere:

Request Processing: Incoming queries reach the gateway, whether in centralized data centers or edge locations
Intelligent Cache Utilization: Gateway evaluates semantic similarity against contextually appropriate cached queries
Efficient Response: For matches, retrieve cached responses and perform rapid content guard validation
Optimized Inferencing: For novel queries, perform inference on appropriately sized models locally or route as needed
Continuous Learning: Cache validated responses for future similar queries in that environment

Organizations can implement these AI functions across their entire infrastructure ecosystem—from massive centralized clusters to hundreds of distributed locations—creating a unified system that enables consistent management while maintaining operational flexibility.

The result is an integrated system delivering AI that is simultaneously faster, more cost-effective, more reliable, and demonstrably safer—regardless of where it operates in your infrastructure.

Conclusion: Building Future-Proof AI Infrastructure

Organizations that thrive in AI's rapid evolution won't necessarily possess the most advanced models, but rather the most thoughtful infrastructure to deploy them effectively and responsibly—wherever inferencing needs to occur.

AI gateways equipped with semantic caching and content guard, managed through a code-first approach, provide everything enterprises need to scale AI responsibly across any environment. The combined solution ensures high performance, reduced costs, streamlined workflows, and robust compliance—whether deployed in centralized data centers, distributed edge locations, or hybrid architectures spanning both.

Getting Started with Advanced AI Gateway Implementation

Ready to enhance your AI infrastructure? Here are specific next steps:

Assessment: Request our complimentary AI Gateway Readiness Assessment to identify your organization's specific needs
Pilot Implementation: Start with a focused pilot in a high-value use case to demonstrate ROI
Solution Consultation: Schedule a session with our technical team to discuss integration with your existing infrastructure
Strategic Roadmap: Develop a phased implementation plan tailored to your business priorities

Explore our comprehensive AI Gateway solution or contact our solution team to discuss how semantic caching and content guard can transform your enterprise AI deployment across your entire infrastructure.

]]>

Security Alert: How Attackers Can Bypass Next.js Middleware With a Single HTTP Header

José Carlos Chávez — Tue, 25 Mar 2025 15:11:21 GMT

The Critical Vulnerability Every Next.js Developer Should Know About

Did you know that a simple HTTP header manipulation could completely bypass your authentication and authorization systems in Next.js applications? Recently, a critical vulnerability (CVE-2025-29927) was disclosed in the popular Next.js framework, allowing attackers to circumvent middleware execution—including security checks—by leveraging an internal header.

Understanding Next.js Architecture and Why It Matters

According to Next.js documentation:

Middleware [in Next.js] allows you to run code before a request is completed. Then, based on the incoming request, you can modify the response by rewriting, redirecting, modifying the request or response headers, or responding directly.

In practical terms, middleware serves as the gatekeeper for your application, handling critical functions like:

Authentication & Authorization
Rate limiting
Request logging
Response manipulation
Routing

When middleware execution is bypassed, all of these security controls fail simultaneously.

How the Attack Works

As detailed in research by security researchers Allam Rachid (zhero) and Allam Yasser (inzo_), the attack exploits the x-middleware-subrequest header—an internal implementation detail that was never meant to be exposed to users.

The Exploit Method

By supplying a specific value to this header—easily inferred from the target application's structure—attackers can completely bypass Next.js middleware execution. For example, accessing a protected admin section at dashboard/panel/admin would simply require trying requests with headers like:

For modern versions (14.x and 15.x):

x-middleware-subrequest: middleware:middleware:middleware:middleware:middleware

For older versions (11.1.4 through 12.1.x):

x-middleware-subrequest: pages/_middleware
x-middleware-subrequest: pages/dashboard/_middleware
x-middleware-subrequest: pages/dashboard/panel/_middleware

You can find more details about the vulnerability here.

Real-World Impact

This vulnerability has severe implications:

Unauthorized access
Bypassing of authentication and authorization
Exposure of sensitive data
Circumvention of rate limiting and other protective measures
Potential access to internal APIs not meant for public consumption
Denial of Service through cache poisoning

The Response from Next.js

The Next.js team addressed this issue with a combination of patches and workarounds that vary by version:

Next.js 15.x: Fixed in version 15.2.3
Next.js 14.x: Fixed in version 14.2.25
Next.js 11.1.4 through 13.5.6: No direct patches available; the official recommendation is to implement a workaround, though 13.x is a maintained version.

Importantly, the official workaround recommendation from Next.js is to prevent the x-middleware-subrequest header from reaching your application - which can be implemented through various means, including WAFs, API gateways, reverse proxies, or load balancers.

This version-specific approach creates notable challenges for organizations:

Teams face trade-offs between performing emergency upgrades or rapidly deploying header-filtering solutions, both of which can strain resources.
Organizations managing multiple Next.js apps across different versions must tailor their mitigation approach for each one. Organizations with multiple Next.js applications across different versions need different strategies
Older versions lack direct patches, requiring teams to rely on network-level filtering—and if that’s not feasible, they may need to refactor sensitive parts of their codebase to mitigate the risk.
For legacy versions, the responsibility shifts from developers to operations and network teams, as code-based fixes aren't an option.
Non-managed deployments require self-managed defenses, increasing the overhead for teams hosting Next.js independently.

For many organizations, leveraging an API Gateway and a WAF—or similar network-level protection—is essential for ensuring consistent security across their entire application portfolio. Teams that already have these in place are in a stronger position to mitigate such attacks, reinforcing the importance of adopting the GW pattern as a proactive investment in security.

The Operational Advantage: API Gateway Layer Protection vs. Application Upgrades

One of the most compelling yet often overlooked benefits of protection through an API Gateway is the dramatic operational advantage it provides compared to codebase and tooling upgrades. Consider the contrasting approaches:

Upgrades approach:

Upgrades may require product downtime
Requires end to end testing of the product
May be blocked by release freezes or change management policies
Demands coordination across multiple teams (development, QA, operations)
Requires upgrading to Next.js 14.2.25+ or 15.2.3+ depending on your current version
May introduce compatibility issues with other dependencies and a cascade of other dependency updates

API Gateway protection approach:

Allows for rapid response to the vulnerability
Can be deployed independently of the application
Can be tested in DetectOnly mode before enforcing
Is typically reversible with minimal consequences
Targets only the specific vulnerability without touching the application code
Can be managed by operations teams without developer involvement
And most importantly, will bring security governance to your platform

The difference is stark for organizations managing dozens or hundreds of applications. Implementing API Gateway protection at the network edge can shield all vulnerable applications simultaneously—while application upgrades might require weeks of coordinated work across multiple teams and systems.

Patching remains essential for comprehensive security. However, the ability to deploy an immediate API Gateway-based solution creates a critical time buffer that allows for proper planning, testing, and deployment of application upgrades without leaving systems exposed in the interim. A good security strategy is the one that relies on multiple layers of protection.

Immediate Mitigation with Traefik and Coraza: Blocking the Vulnerable Header

If you're unable to upgrade to the fixed versions mentioned above, the official Next.js guidance is to block the x-middleware-subrequest header from reaching your application. Traefik offers a streamlined and less invasive way to implement this protection—regardless of whether you are pursuing a temporary workaround or a long-term mitigation strategy.

By leveraging Traefik with its Coraza WAF extension, teams can enforce this critical safeguard at the network edge, avoiding disruptive changes to application code or infrastructure. This approach allows for rapid, centralized protection that scales easily across multiple services and environments.

http:
  middlewares:
    waf:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - SecRule &REQUEST_HEADERS:x-middleware-subrequest "@gt 0" "id:1000001,phase:1,msg:'Request contains x-middleware-subrequest header',deny,status:403"

The above configuration blocks any request containing the vulnerable header, providing immediate protection while you plan your patching strategy.

Even without Coraza WAF, Traefik's native header manipulation capabilities can implement this protection using the Headers middleware:

http:
  middlewares:
    patch-nextjs-apps:
      headers:
        customRequestHeaders:
          x-middleware-subrequest: "" # Removes the header

The key requirement is blocking this header before it reaches your Next.js application, and Traefik provides multiple flexible approaches to accomplish this critical security task.

Broader Protection: Mitigating High-Profile Attacks with Traefik Coraza WAF

The power of Traefik's WAF extends far beyond mitigating specific vulnerabilities like the Next.js middleware bypass. It provides protection against many of the most common and devastating web application attacks, including:

SQL Injection (SQLi)

SQL injection attacks remain one of the OWASP Top 10 web application vulnerabilities year after year. These attacks can lead to:

Unauthorized access to sensitive database information
Data theft or manipulation
Complete system compromise

Traefik's Coraza WAF can detect and block SQLi attempts with rules like:

http:
  middlewares:
    xss-protection:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - Include @owasp_crs/REQUEST-942-APPLICATION-ATTACK-SQLI.conf

Cross-Site Scripting (XSS)

XSS attacks allow attackers to inject malicious client-side scripts into web pages viewed by other users. These attacks can:

Steal session cookies and user credentials
Deface websites
Redirect users to malicious sites
Perform actions on behalf of authenticated users

Traefik WAF can protect against XSS with rules like:

http:
  middlewares:
    xss-protection:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - Include @owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf
            - Include @owasp_crs/REQUEST-949-BLOCKING-EVALUATION.conf

Defense in Depth: Beyond Patching

This vulnerability reminds us of the importance of defense in depth. Even with the Next.js patch, consider implementing these additional protections:

Use layered protection mechanisms rather than relying solely on Next.js middleware
Follow the principle of least privilege for all application components
Implement an API gateway architecture for additional security layers
Integrate WAF capabilities with the API gateway architecture
Regularly audit your dependencies for known vulnerabilities

Traefik's flexibility allows it to act not just as a simple reverse proxy but as a comprehensive security gateway that can adapt to evolving threats without requiring application changes.

Real-World Impact

Several recent reports such as the 2024 Verizon Data Breach Investigations Report, Kaspersky Incident Response Report 2024, and 2024 Unit 42 Incident Response Report, have found that web application attacks continue to be the most common attack vector in confirmed breaches. Implementing a WAF like Traefik Coraza provides a crucial defense layer against these prevalent threats.

A single deployment of Traefik with properly configured WAF rules can protect entire application ecosystems from a wide range of attacks, often without requiring changes to the underlying applications themselves.

Conclusion

HTTP header-based attacks represent a sophisticated yet increasingly common attack vector. The Next.js vulnerability demonstrates how a single header manipulation can bypass critical security controls, but it's just one of many threats targeting modern web applications.

The integration of Traefik with its Coraza WAF extension provides a powerful first line of defense against not only specialized attacks like the Next.js middleware bypass, but also persistent, high-impact threats like SQL injection and XSS attacks that continue to plague applications across industries.

This operational approach—implementing Traefik protection rather than rushing application changes—represents a fundamental shift in vulnerability response strategy. It allows organizations to implement immediate protection while planning thoughtful, well-tested application updates on normal release cycles. This balance of security and operational stability is increasingly critical as application complexity and the pace of discovered vulnerabilities both continue to accelerate.

For development teams, this serves as a reminder to:

Keep frameworks and libraries updated
Monitor security advisories from your dependencies
Have a clear incident response plan for vulnerabilities
Prioritize operational resilience through non-invasive security controls
Consider gateway-level protections that can shield multiple applications simultaneously
Implement multiple layers of security, with an API Gateway as a critical component

By understanding these risks and implementing proper mitigations with Traefik, you can significantly reduce your attack surface and better protect your applications from the ever-evolving threat landscape while maintaining operational stability.

References

Official Next.js security advisory (GHSA-f82v-jwr5-mffw)
Next.js middleware documentation
CVE-2025-29927 details
OWASP Top 10 Web Application Security Risks
Verizon 2024 Data Breach Investigations Report
Kaspersky Incident Response Report 2024
2024 Unit 42 Incident Response Report
Traefik Coraza WAF Documentation
Next.js and the corrupt middleware: the authorizing artifact - zhero_web_security
Credits: Vulnerability discovered by Allam Rachid (zhero) and Allam Yasser (inzo_).

]]>

7 Critical API Protection Strategies to Fortify Your API Security

Immánuel Fodor — Thu, 20 Mar 2025 09:48:04 GMT

API security isn't just good practice – it's a necessity. As businesses increasingly rely on APIs to power everything from mobile applications to cloud services and IoT devices, these digital interfaces have become the new perimeter – and the new target for sophisticated attackers.

The question isn't if an attack will happen, but when. A recent analysis by Treblle (2024) reveals a startling reality: over half of API requests analyzed in 2023 used no encryption whatsoever, leaving sensitive data exposed to interception. Even more concerning, 52% had no authentication mechanisms in place – essentially leaving the digital front door wide open. Furthermore, 85% of APIs implemented no rate limiting, making them vulnerable to brute force attacks and service disruptions.

These vulnerabilities aren't just theoretical concerns. High-profile breaches like the one experienced by BeyondTrust in late 2023, where API key exposure led to significant organizational damage, demonstrate the real-world consequences of inadequate security. As APIs continue to proliferate – with organizations deploying hundreds or even thousands across their ecosystem – the attack surface expands dramatically.

Compounding the problem is the rise of "zombie" APIs – those APIs that remain accessible but unmaintained. Treblle's analysis found that a staggering 35% of endpoints fell into this category, creating significant blind spots in security postures. Without proper documentation, oversight, or regular updates, these forgotten interfaces represent prime targets for attackers seeking the path of least resistance.

Against this backdrop of escalating threats and costly consequences, organizations need a comprehensive, multilayered approach to API security. Let's explore seven critical protection strategies that can help secure your APIs against emerging threats.

#1: Encrypt Your API Traffic

TLS certificates are free – so use them!

While HTTPS might seem basic, the data shows a shocking reality: 55% of API requests analyzed didn't use any encryption (Treblle, 2024). This means sensitive data is being transmitted in plaintext, vulnerable to interception.

Even if the topic feels a little arbitrary to mention, we should prioritize first and foremost to encrypt the traffic leaving or reaching the gateway.

Protection levels:

Manual certificate management: Suitable for organizations that purchase extended validity certificates from authorities

Automated certificate management: Using solutions like Let's Encrypt with the ACME protocol

Self-hosted private key infrastructure: Managing your own PKI with ACME-compatible tools like HashiCorp Vault

Remember, even internal APIs deserve dataflow encryption. There is no excuse for internal APIs to be left unsecured. Leverage wildcard certificates to protect subdomains that are only resolvable through an internal DNS server.

#2: Enforce Strong Authentication and Authorization

The data from Treblle revealed that 52% of analyzed requests had no form of authentication in 2024. This amount represents a massive vulnerability surface. Even internal infrastructure isn't immune to threats. Recent breaches like the one experienced by BeyondTrust demonstrate that API keys can be compromised with devastating effects.

Unauthenticated API calls should be the exception, a deliberate choice that you don't put any kind of authentication in front of them. Apart from these choices, you should authenticate everything.

The best approach starts with an identity provider as the foundation layer for proper API governance, with related authentication methods like JWT, OIDC, or OAuth2 building on top of it.

#3: Protect APIs From Abuse and Overuse

Avoid service disruptions, protect performance, user trust & SLAs

Despite the fundamental role of rate limiting in API security, a surprising 85% of analyzed APIs had no rate limits in place, leaving them vulnerable to denial-of-service attacks and excessive usage (Treblle, 2024). This widespread oversight exposes systems to severe disruptions, impacting both operational stability and user experience.

There are several layers of protection available to prevent such vulnerabilities. To name a few:

IP Address Filtering: This method restricts access to specific IP ranges, ensuring that only trusted sources can make requests.

Parallel Request Limitation: By controlling the number of concurrent requests from a single client, systems can prevent overloads and maintain optimal performance. Especially useful to fight web crawlers or DoS attacks.

Request Flow Control: Implementing true rate limiting with token bucket algorithms helps manage request rates effectively, thereby preventing abuse.
Distributed Rate Limiting: This approach shares rate limit counters across gateway replicas, ensuring consistent enforcement of limits even in distributed architectures. Every replica sees the same counters, enhancing security through coordinated rate limiting.

By leveraging these layered protections, organizations can significantly level up their security game, reducing the risk of service disruptions and safeguarding user trust.

#4: Block Known Vulnerabilities

Configure a Web Application Firewall (WAF)

A Web Application Firewall (WAF) is crucial for defending against known attack patterns, including SQL injection and cross-site scripting. Beyond protecting sensitive data, a WAF is also essential for maintaining regulatory compliance, such as PCI DSS requirements.

Traefik Labs utilizes an open-source core for its firewall, which has OWASP endorsement, ensuring industry-standard protection against the latest threats. Deploying a WAF is relatively straightforward, often requiring just a few lines of configuration, making it an accessible yet powerful mechanism for enhancing security.

#5: Apply Zero-Trust for Accessing APIs

Avoid data leaks with the least privilege principle

Rather than providing unrestricted access to entire APs, a zero-trust approach restricts access to specific endpoints and methods, enhancing security by enforcing least-privilege access, significantly reducing APIs' attack surface.

By defining granular access rules, you can control exactly what clients can access, even as your APIs evolve. This ensures that newly deployed endpoints are not automatically exposed, minimizing the risk of unauthorized access.

#6: Centralize Secret and Key Management

Protect API Keys for AI/LLM APIs

With the growing reliance on external services, particularly AI and LLM services that require API keys for access, centralizing API key management has become crucial. These keys are powerful – if compromised, they can allow unauthorized users to exhaust tokens, deplete account credits, or even drain linked bank accounts.

The most effective solution is to store credentials centrally and issue short-lived tokens to end users. These tokens can be easily revoked, minimizing the impact of compromised keys. By implementing this gateway pattern, organizations can prevent the direct exposure of valuable API keys, ensuring better security and control.

Centralized API key management not only safeguards sensitive data but also streamlines the administration of access credentials.

#7: Avoid Zombie and Shadow APIs

Only expose what’s documented in your OpenAPI specification

One of the most alarming findings showed that 35% of API endpoints were "zombie" endpoints – accessible but unmaintained and potentially vulnerable (Treblle, 2024).

These unmaintained APIs pose more and more threats to organizations these days. Use OpenAPI specifications to define legitimate endpoints and block everything else. Even for legacy systems without documentation, observability tools can help build specifications from observed traffic. You can block undocumented endpoints with flipping one single configuration value.

Bonus Tip: Implement Comprehensive Observability and Reduce Downtime Dramatically

While not directly a security control, observability provides the visibility necessary to identify potential threats and respond quickly.

Without this, you will not see if request rates spike up, if error rates spike up, or if an IP address bombards your infrastructure. By implementing OpenTelemetry for metrics, tracing, and logs, organizations can identify potential security incidents and correlate them with changes to systems. This visibility is essential for effective incident response.

Taking a Multilayered Approach

No single security measure is sufficient on its own. To effectively safeguard APIs, organizations should adopt a multilayered security strategy, integrating multiple protective mechanisms. Each layer – be it rate limiting, WAF, zero-trust, centralized key management, or observability – provides unique benefits that collectively enhance the security posture.

These measures are designed to work together, reinforcing each other to create a robust defense system. By implementing this comprehensive approach to API security, organizations can significantly reduce their risk profile, protecting their data and systems from emerging threats.

The multilayered strategy ensures a resilient and secure API environment, safeguarding business operations and maintaining user trust.

This article was contributed to DevOps.com by Traefik Labs for KubeCon London 2025.

Ready to see how Traefik Hub can elevate your security strategy? Request a demo today and experience the benefits of a modern, high-performance API Gateway and Management solution.

]]>

Beyond the AI Gateway: Why a Holistic API Architecture and Code-First Operating Model Are Essential

Sudeep Goswami — Wed, 19 Mar 2025 12:41:51 GMT

In our previous exploration of enterprise AI architecture, we examined how an AI Gateway serves as a critical component for managing AI operations.

Today, we're diving deeper into why an AI Gateway alone isn't sufficient—organizations need a comprehensive API architecture supported by modern "API Management as Code" practices to handle the explosive growth of AI inference endpoints.

AI Inference Everywhere: The Edge Computing Revolution

The AI landscape is experiencing a fundamental shift as inference workloads—the deployment of trained models to make real-time predictions—expand from centralized cloud environments to the network edge and everywhere in between. This expansion represents a strategic imperative for organizations seeking competitive advantage through faster, more efficient AI deployments.

Global spending on edge computing reached $228 billion in 2024, marking a 14% increase from 2023, according to IDC's Worldwide Edge Spending Guide.[1] Looking forward, IDC forecasts that edge spending will reach $378 billion by 2028, demonstrating a clear expansion of AI workloads across the entire computing spectrum.

This edge-ward expansion is driven by necessity. Millisecond response times—essential for latency-sensitive applications like computer vision, autonomous control systems, and voice interfaces—fundamentally cannot be met by cloud-based inference. As IDC's Dave McCarthy notes, edge computing is "crucial for reducing latency,"[2] enabling the split-second decisions required in modern systems. Consider a manufacturing robot analyzing visual input: waiting for data to travel to distant servers and back creates an unacceptable bottleneck for operations that demand immediate responses.

As we progress through 2025, this trend continues to accelerate as organizations discover that edge AI isn't merely about technical performance—it's about creating entirely new capabilities that weren't previously possible.

The Cambrian Explosion of AI APIs

As organizations operationalize AI models, they're increasingly exposing these capabilities via APIs. Modern software architecture trends have accelerated this shift toward microservices and AI-as-a-Service architectures, where machine learning models are deployed as independent API endpoints rather than being embedded directly into applications. This architectural approach enables organizations to build, deploy, and manage AI inference capabilities as modular, reusable services that can be consumed on-demand by multiple applications.

In practice, this means that an employee application or customer-facing system might call an internal API endpoint for an ML model (or an external API like OpenAI) to get a prediction or generate text, instead of containing the ML model code itself. This decoupling creates significant benefits: improved scalability, easier maintenance, and more efficient resource utilization. However, it also results in what industry experts have aptly termed a "Cambrian explosion" of API endpoints.

According to Forrester's 2024 API Management Wave Report, "As AI agents become mainstream, APIs will be the primary means of AI-driven commerce and agent-to-agent communication."[3]

Major vendors are rapidly expanding their offerings to make AI inference more accessible across public, private, hybrid cloud environments and increasingly at the edge.

This democratization of inference capabilities is driving an unprecedented surge in inference endpoints that must be managed, secured, and governed.

The Salt Security State of API report 2025 provides compelling evidence of this growth, with 30% of organizations reporting 51-100% growth in APIs over the past year, and 25% experiencing growth exceeding 100%. Currently, 13% of organizations manage over 1,000 APIs, with 53% of those being large organizations with more than 10,000 employees.[4]

This expansion is further fueled by the rise of agentic AI. According to Deloitte's State of GenAI Q4 2024, agentic AI—which autonomously orchestrates workflows and tasks—is gaining significant traction.[5] These autonomous agents depend heavily on APIs to interact seamlessly with various enterprise tools, databases, and third-party services, driving further API demand.

The Management Crisis: When APIs Multiply Faster Than Governance

This rapid proliferation of AI inference endpoints has pushed traditional API management approaches to their breaking point. Organizations that previously managed dozens of APIs are now handling hundreds, and those with hundreds are rapidly approaching the thousands mark. These APIs are typically developed by different teams for different purposes, all running in production with varying levels of oversight. This "API sprawl" creates a governance nightmare where management practices struggle to keep pace with innovation.

Security concerns top the list of challenges. Salt Security's 2025 State of API Security Report reveals alarming statistics: 58% of organizations monitor their APIs less than daily, and only 20% continuously monitor their APIs in real-time. Only 15% are very confident in their API inventory accuracy, while 34% admitted facing security challenges regarding lack of visibility into sensitive data exposure.[4]

The consequences are severe: 99% of organizations have encountered API issues in the past year, and 55% have slowed application rollouts due to API security concerns. Most organizations remain unprepared, with only 10% having an API posture governance strategy in place and 59% still in the planning or basic stages of their API security strategies. Only 6% report having advanced security programs, while 8% have non-existent API security strategies.[4]

Beyond security, operational complexity increases exponentially with API volume. When dozens of microservices each expose multiple AI model endpoints, orchestrating these moving parts becomes nearly impossible with manual approaches. The microservices architecture that makes AI deployment more flexible also creates a substantial governance challenge, as each AI model exposed as an API requires its own monitoring, access controls, rate limiting, and lifecycle management.

Versioning presents another critical challenge amplified by AI workloads. AI models are frequently updated as they're retrained on fresh data or improved with new architectures. These changes must often be reflected in the API interface, creating a versioning challenge that grows with each new endpoint.

Why an AI Gateway Alone Is Not Enough

While an AI Gateway provides essential functionality for managing AI interactions—including unified access, security enforcement, and operational efficiency—it represents just one piece of a larger API management puzzle. The AI Gateway excels at handling AI-specific concerns but needs to be part of a holistic architecture that addresses the entire API lifecycle.

An AI Gateway without comprehensive API management is like having a sophisticated security system for your front door while leaving windows and side entrances unprotected.

It creates a false sense of security and control while leaving critical vulnerabilities unaddressed.

For example, an AI Gateway might effectively manage authentication and rate limiting for AI model access, but without proper versioning, documentation, and lifecycle management across all APIs, organizations still face significant challenges as their AI implementations scale. Governance frameworks often lag behind technical implementations, creating gaps where policies aren't consistently applied or enforced.

Furthermore, as organizations deploy AI inference capabilities everywhere—from cloud to edge and all points in between—they need management solutions that span this distributed architecture—providing consistent controls regardless of where models are deployed or how they're accessed.

The Limitations of Traditional API Management

Established API management platforms provide essential capabilities for controlling and streamlining API usage. These platforms offer API gateways for routing and policy enforcement, authentication integration, rate limiting, monitoring, and developer portals.

However, while these tools have served well in traditional environments, they're showing limitations in the face of AI-driven API proliferation. The sheer quantity of APIs generated by AI initiatives strains manual configuration approaches. Traditional API management often relied on web-based consoles where administrators would register and configure dozens of APIs—now they face hundreds of model endpoints, making click-through management impractical.

Governance consistency presents another challenge. Even with robust API management solutions, ensuring that every new AI endpoint receives appropriate policies relies on process discipline. In fast-moving AI development cycles, teams might bypass governance for quick experiments, creating gaps where not all inference APIs are properly managed.

Performance considerations also influence management decisions. AI inference calls can be latency-sensitive, and adding an API gateway hop introduces overhead. For high-throughput or real-time inference scenarios, teams might be tempted to bypass management layers for speed, creating shadow APIs outside governance frameworks.

As we look toward the rest of 2025 and beyond, it's clear that existing API management approaches remain essential but insufficient alone for the AI-driven API ecosystem. The solution lies in creating a holistic architecture that integrates the AI Gateway with proven API management best practices, implemented through modern code-first approaches.

API Management as Code: The Path Forward

"API Management as Code" represents a paradigm shift in how organizations handle their growing API estates. Rather than configuring API gateways and management settings through web interfaces, this approach defines all API definitions, routing rules, policies, and access controls declaratively in version-controlled files. Changes to APIs occur through code modifications deployed via automated pipelines, not through manual console interactions.

This shift mirrors the evolution we've seen in infrastructure management, where Infrastructure as Code (IaC) has become the standard approach for large-scale deployments. For AI-rich environments, the benefits are particularly compelling:

First, code-driven API management enables scale through automation. With APIs defined as code, organizations can script and templatize the creation of new AI endpoints. When a data science team develops a new model, a CI/CD pipeline can automatically generate the API configuration from a template and deploy it to the gateway. As Microsoft's APIOps best practices note, treating API configurations as code helps teams deploy changes iteratively and handle complexity at scale.[6]

Second, treating API configurations as code brings version control and collaboration benefits. API definitions in Git repositories allow teams to review changes via pull requests, catch potential issues before deployment, and maintain a complete history of API evolution. This approach enables rollbacks when needed and fosters standardization through centralized policy templates that every API must follow before deployment.

Third, code-driven management facilitates multi-environment coordination. Large enterprises typically maintain multiple API gateways across development, testing, and production environments. Managing these through code means deploying consistent configurations everywhere from a single source of truth.

Fourth, API mocking capabilities combined with code-driven management can reduce costs during development and testing. When working with expensive AI services, teams can use mock endpoints to simulate responses instead of calling paid APIs for every test. This approach can significantly reduce development costs while accelerating innovation.

As organizations deploy more AI inference endpoints in 2025 and beyond, API Management as Code will move from advantageous to essential—providing the only viable path to maintain control, security, and agility at scale.

A Unified API Architecture: Bringing It All Together

A truly effective approach combines an AI Gateway with a comprehensive API architecture and code-first management practices. This holistic strategy includes:

An AI Gateway that serves as the specialized entry point for AI workloads, providing model-specific optimizations, security controls, and unified access.
Proven API Management best practices implemented through modern, code-driven approaches. These include complete API lifecycle management (design, documentation, versioning, retirement) reimagined for the scale and complexity of AI workloads.
Centralized identity and access management through an IdP, providing a foundational layer to know your users, govern their access, and enforce consistent security policies across all API endpoints.
API Governance that enforces standards, security policies, and compliance requirements across all endpoints.
API Mocking to facilitate development and testing without disrupting production environments or incurring unnecessary costs when working with expensive AI services.
API Management as Code practices that automate deployment and configuration of all these components.

This unified architecture addresses the complete spectrum of challenges organizations face when scaling AI initiatives. The AI Gateway handles the unique aspects of AI workloads, while the broader API management framework ensures consistency, governance, and developer experience across all digital interfaces.

By implementing this holistic approach, organizations can maintain control over their growing API landscape while enabling the agility needed for rapid AI innovation. Teams can deploy new models quickly and securely, knowing that appropriate controls are automatically applied through code-driven processes.

The Time to Act Is Now: How to Get Started

The AI inference explosion isn't a distant future scenario—it's happening now, and the velocity is only increasing. Organizations that delay implementing a holistic API architecture with code-first management practices aren't just missing an opportunity; they're creating existential business risk.

According to Deloitte's "State of Generative AI in the Enterprise – Q4 2024" survey, organizations are experiencing a strategic shift towards using AI for competitive differentiation, with GenAI becoming integral to core business processes.[5] As AI moves deeper into business-critical functions, companies will increasingly rely on APIs to integrate and operationalize these AI models across applications and platforms.

The survey also reveals mature adoption of GenAI within IT, cybersecurity, operations, marketing, and customer service, indicating widespread integration into existing workflows and software. Such integration inevitably increases the reliance on APIs to facilitate interactions between AI services and core enterprise systems.

Higher-than-expected ROI from advanced GenAI initiatives, particularly in cybersecurity, is motivating organizations to scale deployments. This scaling naturally increases API utilization as enterprises integrate these solutions across more users and processes.

Implementing a modern, holistic API architecture doesn't require a complete overhaul of existing systems. Organizations can begin their journey immediately with these steps:

Assess your current API landscape and identify gaps in governance, security, and scalability
Adopt API Management as Code practices for new AI initiatives first, then expand
Integrate your AI Gateway with comprehensive API management capabilities
Implement a consistent governance framework that spans all APIs, not just AI endpoints
Establish centralized identity and access management as a foundation for API security
Embrace automation and code-driven approaches throughout the API lifecycle

The question isn't whether you need a holistic API architecture—it's how quickly you can implement one before the incoming wave of AI inference APIs overwhelms your current capabilities. The organizations that act decisively now will turn API proliferation from a potential crisis into a strategic advantage, creating the foundation for sustainable AI innovation that delivers real business value while managing risk effectively.

Don't be caught unprepared for the AI inference tsunami. The time to build your unified API architecture is now—before the wave hits.

Footnotes

[1]: "Worldwide Edge Spending Guide," IDC Research, 2024.

[2]: "Global edge computing spending to reach $228 billion in 2024," Back End News.

[3]: "The Forrester Wave™: API Management Solutions, Q4 2024," Forrester Research, October 2024.

[4]: "Salt Security State of API report 2025," Salt Security, 2025.

[5]: "State of Generative AI in the Enterprise – Q4 2024," Deloitte Research, 2024.

[6]: "Automated API deployments using APIOps," Microsoft Azure Architecture Center.

]]>

From Ingress to API Intelligence: Unlocking Traefik v3.x’s Full Potential in NKP v2.14

Sudeep Goswami, Michael Matur — Thu, 06 Mar 2025 14:08:03 GMT

The integration of Traefik v3.x in Nutanix Kubernetes Platform (NKP) v2.14 marks a pivotal advancement in enterprise API capabilities. Far more than just an ingress controller, Traefik v3.x establishes a progressive pathway for organizations to evolve their API strategy—from fundamental application routing to sophisticated API Gateway functionality, AI Gateway services, and comprehensive API lifecycle management. This modular approach allows enterprises to advance at their own pace, adopting capabilities sequentially as their needs mature, without disruptive migrations or platform changes. By meeting organizations exactly where they are on their API journey, Traefik v3.x transforms NKP v2.14 from a container orchestration platform into a complete API innovation ecosystem.

Phase 1: Application Proxy - Building the Foundation

When you need it:

You're deploying microservices on NKP and need reliable traffic routing
Multiple applications require consistent load balancing
Service discovery is becoming manual and cumbersome
Team resources are limited for managing infrastructure complexity

How it works: Traefik v3.x in NKP v2.14 provides out-of-the-box service discovery that automatically detects new services as they're deployed. A retail company migrating from monolithic applications to microservices can leverage Traefik's dynamic configuration to route traffic between customer-facing web applications, inventory systems, and order processing services without manual intervention. When holiday shopping traffic increases, Traefik's load balancing capabilities ensure resources are distributed optimally across services, preventing any single component from becoming a bottleneck.

Key benefits:

Zero-configuration service discovery reduces operational overhead
Real-time traffic routing adjustments based on service health and availability
Simplified configuration through Kubernetes CRDs and annotations
Centralized entry point for all microservices traffic management

Phase 2: API Gateway - Securing and Controlling Access

When you need it:

Security requirements demand centralized authentication and authorization
You need to manage access to APIs across multiple teams or departments
API traffic requires special handling (rate limiting, circuit breaking)
AI workloads need specialized routing and management
Enterprise integration requirements are increasing

How it works: As your API ecosystem grows within NKP, Traefik v3.x transforms into a full-fledged API Gateway. A financial services company can implement OAuth2 authentication for all customer-facing APIs while using different authentication mechanisms for internal services. When they introduce a new AI-powered fraud detection system, Traefik's AI Gateway capabilities allow them to route specific transactions to this service based on risk profiles, while implementing circuit breaking to prevent cascading failures if the AI service experiences high load. Enterprise integrations with existing identity providers and security systems ensure compliance with financial regulations without custom code.

Key benefits:

Centralized security policies for all API traffic
Specialized handling for AI endpoints with appropriate rate limiting and prioritization
Advanced traffic shaping capabilities (circuit breaking, retries, timeouts)
Integration with enterprise security infrastructure
Single control plane for all API traffic

Phase 3: API Management - Full Lifecycle Runtime Governance

When you need it:

APIs have become strategic business assets requiring governance
Developers need self-service portals to discover and consume APIs
Operations teams need comprehensive observability across the API landscape
API versioning and lifecycle management become critical
Business stakeholders need insights into API usage and performance

How it works: At the most mature stage, Traefik v3.x within NKP v2.14 evolves into a complete API Management solution. A healthcare technology company can implement API governance policies ensuring patient data APIs adhere to HIPAA compliance requirements. Their developer portal enables partner companies to discover available APIs, test them in sandbox environments, and generate client libraries. The API observability dashboard gives operations teams real-time insights into performance bottlenecks, while business analysts track API usage to determine which features drive the most value. When new AI-powered diagnostic services are introduced, the existing API lifecycle management ensures proper versioning, documentation, and controlled rollout.

Key benefits:

Comprehensive API governance with policy enforcement
Developer self-service through customizable portals
End-to-end API observability for operations and business stakeholders
Controlled API versioning and deprecation workflows
Unified management across traditional and AI-enabled services

By leveraging these progressive capabilities of Traefik v3.x within NKP v2.14, organizations can evolve their API strategy at their own pace while maintaining a consistent platform and operational model. Whether you're just beginning your microservices journey or managing a sophisticated API ecosystem with AI components, Traefik provides the tools needed at each stage of maturity.

Putting Theory into Practice: Upgrading Your NKP v2.14 Traefik Deployment

Now that you understand the value progression from Application Proxy to API Gateway to full API Management within Traefik v3.x, you might be wondering:

"How do I actually implement these advanced capabilities in my NKP v2.14 environment?"

The good news is that Nutanix and Traefik Labs have designed a straightforward upgrade path that preserves your existing configurations while unlocking these powerful features. The upgrade process is deliberate yet simple, allowing you to evolve your API infrastructure without disruption to your current services.

Let's walk through the specific steps required to transition from the default Traefik OSS version in NKP v2.14 to the full enterprise capabilities:

Create the Traefik Hub Gateway Token: contact the Traefik Sales Team for access, and then login to the Traefik Hub Dashboard https://hub.traefik.io

Create a New Catalog Entry in NKP

kubectl apply -k https://github.com/traefik/nkp-traefiklabs-catalog/hub
TRAEFIK_HUB_TOKEN=XXXX {where XXXX represents the actual token from the previous step}
kubectl create secret generic license --namespace kommander --from-literal=token=$TRAEFIK_HUB_TOKEN

Enable the Traefik Hub Catalog Entry in NKP

Next, enable the Traefik Hub catalog entry to complete the upgrade process. This step integrates Traefik Hub within your NKP cluster, providing advanced API management capabilities. Note: this step could take anywhere from 1 to 5 minutes

Finally, log back in to Traefik Hub, and verify that the upgrade is now complete and your Traefik Hub API Gateway is online.

This transition bridges the conceptual journey we've discussed with concrete implementation steps, allowing you to immediately begin realizing the benefits of advanced API capabilities in your Nutanix Kubernetes environment.

The API-Powered Future Begins with Traefik and NKP

The integration of Traefik v3.x in NKP v2.14 represents more than just a technical upgrade—it's a strategic enabler for organizations looking to harness the full potential of their API ecosystem. By providing a progressive path from basic application routing to sophisticated API management and AI gateway capabilities, Traefik empowers enterprises to evolve at their own pace while maintaining operational consistency.

As we've explored throughout this post, each phase of the Traefik journey addresses specific organizational needs and challenges:

Application Proxy capabilities solve immediate routing and discovery challenges
API and AI Gateway features enhance security and control in complex environments
Full API Management delivers the governance and visibility needed for strategic API and AI programs

With the straightforward upgrade path available in NKP v2.14, organizations can now seamlessly access these capabilities without disruptive changes to their existing infrastructure.

Take the Next Step in Your API Journey

Ready to unlock the full potential of Traefik v3.x in your NKP environment? Here's how to get started:

Assess your current API maturity and identify which Traefik capabilities would deliver the most immediate value
Explore the five essential qualities of a modern API gateway to ensure your infrastructure is future-proof
Implement runtime API governance best practices to enhance security, compliance, and observability
Plan a seamless API gateway migration with a step-by-step guide that you can apply to your specific use case
Schedule a demonstration with our team to see Traefik's advanced features in action within an NKP environment

Don't let your API infrastructure limit your innovation potential. With Traefik v3.x and NKP v2.14, you have all the tools needed to build a scalable, secure, and manageable API ecosystem that can grow with your business—from your first microservice to your most sophisticated AI-powered application.

]]>

How to Size Your Traefik Hub API Gateway Instances

Nicolas Mengin — Tue, 04 Mar 2025 14:14:01 GMT

When onboarding new customers, one common question we face is about the optimal resource configuration to set for a Traefik Hub API Gateway instance. Unfortunately, there is no one-size-fits-all answer to this question. Several factors need consideration, such as the infrastructure type, traffic volume, and how Traefik Hub API Gateway will be utilized.

Let’s explore why these factors are crucial and how to customize the Traefik Hub API Gateway deployment accordingly.

Define the size

From the tests we have done internally, we know a single Traefik Hub API Gateway instance (installed on 8 CPUs and 16 GB of RAM machine) is capable of routing over 72,000 HTTP Requests per Second (RPS) (and more with bigger configurations) if there is no security or operations to do on the requests. With the introduction of the experimental fastProxy option, performance can even reach up to 102,000 RPS with the same configuration. This suggests that when provided you allocate a sufficiently powerful machine, RPS should not be your primary concern in production—at least in theory.

In practice, the maximum RPS depends heavily on the type of traffic being managed. For example, if your Traefik Hub API Gateway instance is handling TLS connections or utilizing middleware (such as header management or rate limiting), the RPS capacity will decrease.

Additionally, the more you want Traefik Hub API Gateway to handle high RPS values, the larger, and the more expensive, the machines are. While these machines offer high performance, they come with high costs and a poor return on investment (ROI) since you likely won't use their full capacity most of the time—but you'll pay for them regardless.

On top of that, a key best practice is to favor scaling out (horizontally adding more smaller instances) over scaling up (vertically increasing the resources of a single instance). This approach not only makes your infrastructure more scalable but also more resilient: you reduce the risk of a Single Point of Failure (SPOF) and ensure that your system can handle traffic spikes more effectively.

For these reasons, we recommend deploying multiple smaller instances rather than a few powerful ones. The exact number and size of instances depend on your infrastructure setup as we will see below.

Define the number

When Traefik Hub API Gateway is deployed within an orchestrator (like Kubernetes), sizing becomes more straightforward. We recommend deploying enough instances (each with 2 vCPUs, and 4GB RAM) to handle up to 3,000 RPS per instance under normal traffic conditions.

The key factor here is observability: in such an environment, monitoring your containers or pods and ensuring they scale up when resource consumption reaches 70% of CPU or RAM capacity, and scaling down when traffic returns to normal.

However, despite Traefik Hub API Gateway being a cloud-native Ingress Controller primarily designed for container environments, it performs admirably on traditional bare-metal infrastructure as well.

In these environments, the challenge is finding the right balance between avoiding a Single Point of Failure (SPOF) and minimizing over-provisioning of instances. Deploying multiple small VMs with limited CPU and memory resources can handle normal traffic loads, but this approach may fall short during sudden traffic spikes if the combined RPS capacity is insufficient.

On the other hand, using a few larger VMs and scaling them up is both time-consuming and operationally intensive compared to simply adding replicas in a Kubernetes deployment. This makes it crucial to anticipate traffic peaks, ensuring each machine is capable of handling unexpected spikes and that scaling occurs promptly.

For bare-metal environments, we recommend deploying medium-sized VMs that can reliably handle around 50% of peak traffic, providing sufficient capacity while maintaining resilience.

Real-Life Example: Traefik Hub API Gateway in Kubernetes

In today’s landscape, securing API access is non-negotiable. Therefore, it's important to configure the Traefik Hub API Gateway in a way that accounts for the additional latency introduced by the TLS handshake and JWT authentication verification.

The use of the header middleware is also common, as it allows you to pass extra information to your backend services. However, adding headers introduces additional latency, which is why we include it in this test.

Finally, gathering metrics for observability is crucial for any production platform. Exposing traffic metrics through OpenTelemetry allows you to monitor and gain insights into system performance, but it too has an impact on Traefik Hub API Gateway’s overall performance.

Let's now apply the sizing recommendations we discussed earlier to two typical use cases with different requirements:

Up to 1,000 RPS per Instance

This scenario represents a common need for exposing APIs. Here, the traffic volume is moderate, with each Traefik Hub API Gateway instance handling up to 1000 RPS.

Instance Sizing: For this scenario, we recommend deploying several small instances (e.g., 1-2 vCPUs, and 1-2 GB RAM). Traefik Hub API Gateway can scale horizontally, allowing you to start with a modest setup and add instances as traffic grows.
Middleware Impact: Even though the traffic load is internal, enabling TLS for secure communication and using JWT Authentication middleware (as well as other middleware or operations on access logs) will introduce some overhead. Injecting headers also adds to the processing time, but at this scale, the latency impact should remain manageable.
Scaling Strategy: Ensure autoscaling triggers when resource usage (CPU, memory) hits 80%, and scale down once traffic normalizes. A few well-sized instances should be enough to handle this load efficiently while maintaining cost-effectiveness.
Recommendation: Start with 2-5 replicas to ensure resiliency, and monitor performance to adjust as needed.

1,000 to 5,000 RPS per Instance

This scenario is geared toward more demanding use cases, such as exposing APIs to external partners, customers, or commercial websites. With traffic ranging from 1000 to 5000 RPS per instance, the need for scalability and performance is more critical.

Instance Sizing: For this higher traffic volume, each Traefik Hub API Gateway instance should have more resources—typically around 2-4 vCPUs and 4-8 GB of RAM per instance. However, instead of using a few large machines, it’s best to deploy multiple medium-sized instances to balance performance and cost.
Middleware Impact: As with internal API exposure, TLS termination and JWT Authentication will still introduce latency, but at this higher traffic rate, the impact is more pronounced. Ensure the resources allocated to each instance are sufficient to handle the added load from security measures and header management. Also, exporting metrics in OpenTelemetry format will increase the processing load, so factor that into resource allocation.
Scaling Strategy: For this scenario, it is essential to configure autoscaling carefully to handle traffic spikes effectively. Ensure your system scales before traffic exceeds 80% of your instance capacity to avoid bottlenecks, and consider pre-emptively adding extra replicas during anticipated peak periods (e.g., during product launches or promotional events).
Recommendation: Start with 4-10 replicas to handle external traffic spikes effectively, ensuring a minimum of 20% buffer capacity for peak periods.

More than 5,000 RPS per Instance

For scenarios where APIs are exposed to high-traffic environments, such as public-facing commercial websites or services with millions of users, a more specialized setup is needed:

Instance Sizing: Deploy large instances with at least 4-8 CPUs and 8-16 GB of RAM. You might also consider using high-performance nodes or dedicated bare-metal instances if the traffic is extremely high.
Scaling Strategy: Each instance should be optimized to handle more than 5,000 RPS. You may need to use both horizontal scaling (increasing instance count) and vertical scaling (boosting instance capacity) to meet traffic demands.
Middleware Impact: At this level, even small latencies introduced by TLS handshakes, JWT, header manipulation, and metrics collection can add up quickly. It’s critical to identify each bottleneck that slows down the traffic (using Tracing) to optimize each layer of the stack for performance, and potentially offload some tasks to specialized systems (e.g., a dedicated authentication service).
Recommendation: Start with at least 10-20 replicas and monitor usage closely to ensure a quick reaction to traffic spikes. For very high loads, consider deploying Traefik Hub API Gateway in a high-availability setup, with global load balancing across multiple regions or data centers.

Conclusion

Properly sizing Traefik Hub API Gateway instances is key to ensuring secured API publication, whether you're dealing with internal or external traffic. By understanding the specific needs of your environment—such as traffic volume, middleware usage, and infrastructure type—you can deploy a cost-effective and scalable Traefik Hub API Gateway configuration that meets your performance goals.

Regardless of the scenario, monitoring resource consumption and scaling based on usage thresholds (like 80% of CPU or RAM) is crucial to avoiding performance bottlenecks. By following these guidelines, you’ll ensure your Traefik Hub API Gateway deployment is both resilient and optimized for your specific use case.

Useful Links

Traefik API Gateway Documentation
Traefik API Gateway Webpage
Our Community Forum

]]>

The AI Gateway Imperative: Why Your Enterprise AI Strategy is Incomplete Without it

Sudeep Goswami — Mon, 10 Feb 2025 11:02:17 GMT

The enterprise AI landscape has reached a pivotal moment. According to Deloitte's latest research, 52% of organizations are prioritizing automated AI agents, while 45% are focused on multiagent systems.

Even though the uptake of these technologies is impressive, given how new they are, we're rapidly moving beyond basic automation to autonomous AI agents that can plan, decide, and act independently. These "digital workers" are transforming enterprise operations (adoption has surged to 78% of companies surveyed, according to McKinsey), but they're also introducing unprecedented complexity and risk.

In this article, we’ll explore this newfound complexity and risk, as well as how to tame them.

The Reality of Enterprise AI Usage Today

Enterprise AI adoption is accelerating, but it’s happening in silos. Development teams deploy AI-powered chatbots using their preferred providers, HR leverages AI-driven resume screening tools, and marketing automates content generation with yet another set of AI services. Each department operates independently—selecting its own vendors, enforcing separate security controls (if at all), and managing AI usage in isolation.

This fragmented approach forces organizations to navigate a growing maze of AI integrations, each with different authentication methods, governance frameworks, and cost structures. A single company might have marketing using GPT-4, customer service relying on Claude, and HR leveraging Gemini—each through separate interfaces, with inconsistent security policies and compliance measures.

As AI investments continue to rise—67% of organizations plan to increase spending (McKinsey), and 78% expect further growth in 2025 (Deloitte)—this lack of cohesion is becoming increasingly unsustainable.

The Hidden Risks of Ungated AI

The fragmented, decentralized AI adoption exposes organizations to serious risks. Without a unified mechanism to manage and monitor AI interactions, companies face mounting challenges.

Compliance Challenges
As the #1 barrier to GenAI deployment, regulatory compliance isn't just about checking boxes. When AI systems handle sensitive customer data or make automated decisions affecting individuals, a single compliance misstep can lead to substantial fines and damaged reputation.
Security Vulnerabilities
The fact that 35% of organizations cite "mistakes/errors with real-world consequences" as their top concern (Deloitte) isn't surprising. Imagine an AI-powered trading system making unauthorized transactions, or a customer service AI exposing sensitive information—these aren't hypothetical risks but real scenarios playing out in unprepared organizations.
Cost Management
The statistic that 78% of organizations plan to increase AI spending in 2025 (Deloitte) takes on new meaning when you consider uncontrolled usage. Without proper oversight, departments can unknowingly rack up massive bills through inefficient API calls or redundant model usage.

The Strategic Role of an AI Gateway

An AI gateway serves as your enterprise's command center for AI operations, fundamentally transforming how organizations manage their AI infrastructure. At its core, it functions as a unified control point that abstracts away the complexity of multiple AI providers and models. This allows development teams to focus on what truly matters—building valuable applications—rather than getting bogged down in managing complex integrations.

Security and governance are equally crucial aspects of the gateway's role. It acts as a centralized checkpoint ensuring all AI interactions adhere to enterprise policies—a critical function given that 70% of organizations need at least 12 months to resolve governance challenges (Deloitte). A centralized approach means security policies can be consistently enforced across all AI interactions, regardless of which team or department initiates them.

Additionally, as AI usage scales across the enterprise, the gateway's intelligent traffic management becomes increasingly vital. Through sophisticated caching and load-balancing capabilities, it optimizes resource utilization and controls costs while maintaining high performance. This means organizations can scale their AI operations confidently, knowing they have the infrastructure to support growing demand without compromising on efficiency or reliability.

Accelerating Enterprise-Wide Adoption

The path to widespread AI adoption in enterprises faces significant hurdles.

Current data shows that less than 40% of the workforce has access to GenAI tools (Deloitte), and even among those with access, adoption rates remain surprisingly low.

This limited penetration stems from multiple challenges.

Many employees feel overwhelmed by the complexity of AI tools.Others worry about inadvertently violating company policies. Some teams struggle to justify the investment when they can't properly track and measure AI's impact on their productivity.

An AI Gateway provides the critical infrastructure foundation that enables widespread AI adoption. It offers essential backend capabilities that organizations can build upon to create consistent, reliable, and secure AI experiences across the enterprise. This standardized infrastructure means teams can focus on creating solutions that address their specific business needs, which are supported by enterprise-grade security and governance controls.

Moreover, the Gateway's rich observability capabilities enable organizations to track usage patterns, measure ROI, and identify departments that might need additional training or support. This data-driven approach helps organizations understand where adoption is lagging and why, allowing them to make targeted interventions. For example, if certain teams show low usage rates, the organization can provide specialized training or showcase relevant use cases from successful departments.

By providing a robust foundation for AI applications, ensuring consistent security controls, and offering visibility into AI usage, the Gateway transforms AI from a specialized tool for technical teams into an enterprise-wide capability that can serve every employee's needs.

Competitive Advantages of AI Gateway Implementation

Organizations that implement AI gateways gain significant strategic advantages that extend far beyond basic infrastructure benefits. While an AI gateway may appear to be just another technical implementation, it fundamentally transforms how enterprises operate with AI technology.

Time-to-market acceleration becomes a key differentiator. Development teams can launch new AI initiatives in days rather than months by leveraging pre-built integrations and standardized security controls. Instead of each team reinventing the wheel for API connections, authentication, and compliance checks, they can focus on creating business value.

Operational costs decrease substantially through intelligent resource optimization. The gateway's ability to route traffic efficiently, cache responses, and load-balance across providers means organizations typically see 30-40% reduction in their AI operational costs (Deloitte). Teams can experiment with different AI models without worrying about unexpected cost spikes or resource waste. These cost benefits are being realized across industries—McKinsey's research shows that organizations are seeing material benefits from AI deployment, with notable cost decreases particularly in human resources and service operations.

Security and compliance posture strengthens considerably across the organization. Rather than managing security policies for each AI application individually, organizations can enforce consistent controls at the gateway level. Centralizing security not only reduces risk but also streamlines audit processes and regulatory reporting.

Technology adaptation becomes seamless as the AI landscape evolves. When new AI models emerge or existing providers enhance their capabilities, organizations can quickly integrate these advances without disrupting existing applications. Enterprises are then better able to stay current with AI innovation while maintaining operational stability.

So in short, AI gateways transform scattered AI initiatives into a cohesive enterprise capability. It's analogous to the difference between having a fleet of autonomous vehicles with no traffic management system versus having a sophisticated control tower that ensures safety, efficiency, and coordination. The AI gateway becomes the foundation that turns isolated AI experiments into a true AI-powered enterprise.

The Time to Act Is Now

With market pressures accelerating AI adoption, that stark Deloitte finding bears repeating—70% of organizations lose a whole year struggling with AI governance and risk management.

This extended implementation timeline means organizations must act now, as the window for gaining a competitive advantage through proper infrastructure is closing. Organizations that delay implementing an AI gateway risk falling behind more agile competitors who can deploy and scale AI initiatives faster and more securely. McKinsey's findings reinforce this urgency, showing that high-performing organizations are already using AI in three or more business functions, while others average just two—creating a widening competitive gap.

However, implementing an AI gateway and governance doesn't have to be a lengthy process. Modern gateway solutions leverage GitOps practices and declarative configurations, allowing organizations to get up and running in minutes rather than months. Teams can define their entire AI infrastructure as code, version control their configurations, and automate deployments. This makes the journey from decision to implementation remarkably swift.

The question isn't whether you need an AI gateway but how quickly you can realize its benefits.

In the race to harness AI's potential, having the right infrastructure isn't just an advantage—it's a prerequisite for success. And with today's tools and practices, that infrastructure is more accessible than ever.

Sources:

Deloitte’s State of Generative AI in the Enterprise Report Q4 2024

McKinsey State of AI in early 2024

Useful Links

Traefik AI Gateway Documentation
Traefik AI Gateway Webpage
Our Community Forum

]]>

Traefik Proxy v3.3 - Nectaire of the Gods

Emile Vauge — Thu, 09 Jan 2025 11:46:21 GMT

Traefik v3.3, codenamed saint-nectaire (one of the best cheeses you can find in France, made in the volcanic area Auvergne), is now generally available after two months of dedicated development and two release candidates. This release focuses on two main tracks: observability and documentation.

Observability has become essential for platform engineers to quickly detect outages and recover services in today's complex, cloud-native environments. Traefik Proxy v3.3 once again pushes the boundaries by adding state-of-the-art features with a deeper OpenTelemetry integration as well as a much more granular observability configuration. This makes Traefik Proxy one of the most flexible and powerful telemetry data providers, giving you instant and complete knowledge of all your ingress traffic.

Additionally, documentation is one of the most important (and underrated) parts of an open-source project. It’s the main entry point 😉 to your product. It should onboard newcomers to quickly learn and test while providing a clear and complete description of every option and use case to more advanced users.

Back in 2024, we started a major project to reorganize and revamp the Traefik Proxy documentation. This is quite a big journey, and we are happy to hit the first milestone today with a brand-new reference documentation.

Ready to dive into v3.3? Let’s start with OpenTelemetry.

OpenTelemetry Support for Logs and Access Logs

Since its very first release, Traefik Proxy made observability one of its core features, supporting many vendors for metrics and tracing analysis. Building upon the foundation laid in Traefik Proxy v3.0 with OpenTelemetry integration, v3.3 goes even further by adding support for logs and access logs.

This feature is tagged “experimental,” as the official underlying library is still being finalized.

To unlock it, simply add this to your configuration:

experimental:
  otlpLogs: true

Then let’s enable logs and access logs globally to the OpenTelemetry configuration:

metrics:
  otlp: {}
tracing:
  otlp: {}
log:
  otlp: {}
accesslog:
  otlp: {}

The OpenTelemetry logger exporter will send logs to the collector using HTTPS by default to https://localhost:4318/v1/logs. You can customize the exporter by referring to the logs and access logs documentation.

Flexible and Granular Observability

Until today, you could enable observability features globally on Traefik Proxy, which means, either you had metrics (or tracing) for all your routes or none. However, there are many use cases where you don’t want to have tracing for all your applications. Or you might just need to enable these features temporarily.

That’s why we have been working on a new mechanism to have more granular control over observability. In v3.3, you now have a way to define default behaviors and you can enable or disable tracing, metrics or access logs per router.

The default router observability configuration is inherited from the attached entryPoints and a router defining its own observability configuration will override those options.

Let’s take an example where you want to enable tracing by default for an entryPoint:

You still need to configure the tracing globally:

tracing:
  otlp: {}

Tracing is enabled by default on entryPoints, so you shouldn’t need to change anything in your configuration:

entryPoints:
  foo:
    address: ':80'

This is equivalent to:

entryPoints:
  foo:
    address: ':80'
    observability: 
      tracing: true

Every router attached to this entryPoint will have tracing enabled. If you want to disable a specific router, that’s pretty simple:

## Dynamic file configuration
http:
  routers:
    my-router:
      rule: "Path(`/foo`)"
      service: service-foo
      observability:
        tracing: false

Or if you use Kubernetes Ingresses, just add this annotation to the corresponding ingress:

traefik.ingress.kubernetes.io/router.observability.tracing: false

Let’s take another example where you want to disable tracing by default for an entryPoint:

entryPoints:
  foo:
    address: ':80'
    observability:
      tracing: false

And then if you want to enable tracing temporarily while troubleshooting a service:

## Dynamic file configuration
http:
  routers:
    my-router:
      rule: "Path(`/foo`)"
      service: service-foo
      observability:
        tracing: true

Or within an ingress:

traefik.ingress.kubernetes.io/router.observability.tracing: true

This simple and powerful granular control applies to tracing, metrics, and access logs.

New Reference Documentation

As mentioned before, great documentation is key for open source projects. We've been working hard to improve Traefik Proxy's docs, and we have defined new guidelines as follows:

Organize the documentation for 3 personas: beginners, advanced ops, and/or advanced developers
Describe high level use cases without listing every option possible in the first sections of the documentation (Getting Started, Setup, Expose, Observe, Migrate, etc)
Move all option descriptions to a well organized Reference section

Today we're excited to launch our brand new reference documentation as a first milestone of this global revamp.

The reference documentation used to have this structure:

├── Static Configuration
│   ├── Overview
│   ├── CLI
│   ├── Environment Variables
│   ├── File
└── Dynamic Configuration
    ├── Consul Catalog 
    └── …

We reorganized this section to provide a much more comprehensive description of all options available:

├── Install Configuration
│   ├── Boot Environment
│   ├── Configuration Discovery
│   │   ├── Overview
│   │   ├── Kubernetes
│   │   │   ├── Kubernetes Gateway API
│   │   │   ├── Kubernetes CRD
│   │   │   └── Kubernetes Ingress
│   │   ├── Docker
│   │   ├── Swarm
│   │   ├── Hashicorp
│   │   │   ├── Consul Catalog
│   │   │   ├── Consul
│   │   │   └── Nomad
│   │   ├── KV Stores
│   │   │   ├── Consul
│   │   │   ├── ETCD
│   │   │   ├── Redis
│   │   │   └── ZooKeeper
│   │   └── Others
│   │       ├── ECS
│   │       ├── File
│   │       └── HTTP
│   ├── Entrypoints
│   ├── API & Dashboard
│   ├── Observability
│   │   ├── Healthcheck
│   │   ├── Logs and Accesslogs
│   │   ├── Metrics
│   │   └── Tracing
│   └── TLS
│       ├── Certificate Resolvers
│       │   ├── Overview
│       │   ├── ACME
│       │   └── Tailscale
│       └── SPIFFE
└── Dynamic Configuration
    ├── Consul Catalog
    └── …

We hope this will help beginners and advanced users to quickly find and learn everything they want to about Traefik Proxy. The next step? Revamp high-level sections!

Other Improvements

On top of those great new features, Traefik Proxy v3.3 brings options to control ACME (Let’s Encrypt) propagation checks (#11241 by ldez). We also added configuration dump support from the API endpoint to make troubleshooting sessions easier.

Skirtan1 made the IngressRoute kind optional (#11177) and BZValoche added Kubernetes serving endpoint status support to detect and mark servers as “fenced” when stickiness is enabled. Regarding sticky cookies, you can now make the path configurable (#11166 by IIpragmaII).

Host headers in the HTTP provider can now be set (#11237 by nikonhub), and you can preserve the Location header in the ForwardAuth middleware (#11318 by Nelwhix).

Another great addition brought by michelheusschen allows you to only calculate basic auth hashes once for concurrent requests (#11143).

You can now send the request body to the authorization server with the ForwardAuth middleware (#11097 by kyo-ke).

Additionally, it is now possible to make Traefik abort the startup in case a plugin load fails (#11228 by bmagic) and finally, the API & Dashboard base path can now be configured.

Moving Forward

Traefik Proxy v3.3 is a significant release that focuses on observability and documentation. With the addition of logs and access logs to its existing support for metrics and traces, Traefik Proxy now provides even deeper insights into your ingress traffic. Additionally, the new granular observability configuration allows you to enable or disable tracing, metrics, or access logs on a per-router basis, giving you more control and flexibility over the data you collect. And the improvements to the documentation, including the new reference documentation, make it easier for users to learn about and use Traefik Proxy.

Overall, v3.3 is a powerful release that makes Traefik Proxy an even more valuable tool for platform engineers and developers.

We are immensely thankful to the contributors who are shaping this great project. Join the vibrant Traefik community, share your feedback, and help us shape the future of cloud-native networking.

Useful Links

Traefik 3.3 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

How to Keep Your Services Secure With Traefik’s Rate Limiting

Harold Ozouf — Tue, 10 Dec 2024 18:34:52 GMT

The internet can be a challenging environment for running applications.

When you expose a service to the public internet, it's crucial to assess the risks involved. Malicious actors may try to misuse your resources or even bring your service down.

You can't just make your service publicly available without protection and expect things to go smoothly. They won’t. That’s why safeguarding your service is essential. There are many threats to address. Configuring your server properly is a good start, but you also need to protect your application from harmful actions. Our Web Application Firewall (WAF) can assist with that.

However, today we're focusing on a different threat: attempts to overwhelm your resources or disrupt your service with an excessive number of requests. This is where rate limiting comes in.

What is Rate Limiting?

Rate limiting is the process of limiting the flow of requests that reach your servers. Think of it like a funnel, where a large pipe of water narrows into a smaller one, flowing at a much more manageable rate before reaching its destination:

In this analogy, each water molecule represents an HTTP request. A pipe of a certain size limits how many molecules can pass through at once. When the pipe narrows, fewer molecules can flow through, reducing the flow rate.

This illustrates how a rate limiter works to control the flow of requests. Additionally, we can fine-tune the control by limiting traffic based on characteristics like IP address, user, or other request details. You might also want to allow brief bursts of traffic without blocking them entirely.

When discussing rate limiting, the following algorithms are commonly mentioned:

Token Bucket: The one used by Traefik, which we'll focus on below.
Leaky Bucket: A first-in, first-out queue that releases traffic at a steady rate. It's less flexible than Token Bucket for handling traffic bursts.
Generic Cell Rate Algorithm (GCRA): Similar to Leaky Bucket but ensures packets follow a set timing interval instead of draining at a fixed rate.

Two other types of algorithms often come up:

Sliding Window
Fixed Window

Unlike rate limiting, which regulates the flow and timing of requests, these two algorithms count the total number of requests within a specific period. They're better suited for enforcing strict usage quotas than managing request flow.

Token Bucket

The Token Bucket algorithm controls the flow of requests by using a metaphorical bucket that holds tokens. Tokens are generated at a constant rate and added to the bucket, which has a fixed capacity.

When a request arrives, the system checks if there are enough tokens in the bucket. Each request consumes one token. If enough tokens are available, the request proceeds, and a token is removed. If there aren’t enough tokens, the request is either delayed until more tokens are available or blocked if the delay would be too long.

The bucket can't hold more tokens than its maximum capacity, so once it's full, any new tokens are discarded. This prevents tokens from accumulating indefinitely and allows the system to handle short bursts of high traffic, as long as the bucket has enough capacity.

When the bucket is low or empty, the system slows down, giving time for more tokens to be generated. The token generation rate and the bucket size are key factors that determine how the system performs.

The behavior of the Token Bucket algorithm is determined by two key factors:

Bucket size: This defines how many requests can be processed simultaneously.
Rate: This controls how frequently new request opportunities become available as tokens are generated.

Rate Limiting in Traefik

Traefik allows you to define a ratelimit middleware that can be applied to your routers. Assigning this middleware ensures that the flow of incoming requests doesn’t exceed the configured rate.

💡

Reusing the same middleware on multiple routers does not mean they share the same bucket. Each router gets its own instance of the middleware. Shared buckets and distributed rate limiting for enterprise use cases are available through Traefik Hub.

This middleware uses the Token Bucket algorithm described earlier. The bucket size is controlled by the burst parameter, while the rate is set by the average and period values. The following example configures a rate limiter with a rate of 100 requests per second and a bucket size of 200:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: my-rate-limit
spec:
  rateLimit:
    average: 100
    period: 1s
    burst: 200

This configuration means a new request can pass every 10ms. If you receive fewer than 1 request every 10ms, the bucket will fill up, eventually allowing a burst of up to 200 simultaneous requests.

Grouping Requests by Source

Grouping requests by their source and applying rate limiting is an effective way to prevent a small number of clients from overwhelming a system. Instead of limiting the system as a whole, this technique enforces rate limits per individual source, ensuring balanced resource usage.

For example, you can group requests by their IP address and apply rate limits to each IP. This prevents a single IP from sending too many requests, which could degrade the system’s performance. By limiting requests at the source level, the system gains more granular control, preventing any one source from monopolizing resources while still allowing others to access the system at normal rates.

The ratelimit middleware allows requests to be grouped by different sourceCriteria, such as:

ipStrategy: The client’s IP address (default strategy)
requestHost: The client’s hostname
requestHeader: The value of a specific request header

The ipStrategy relies on the de-facto standard X-Forwarded-For header to determine the client’s IP address. When a client connects directly to a server, its IP address is sent to the server. But if a client connection passes through any proxies, the server only sees the final proxy's IP address, which is often of little use. So, to provide a more-useful client IP address to the server, the X-Forwarded-For request header is used. When a proxy receives a request it adds itself to this header.

The X-Forwarded-For header can be a useful tool, but it also poses security risks if not handled properly. Since each proxy in a network chain adds its own IP address to this header, a malicious client could manipulate it by inserting fake IP addresses. This can mislead the server into believing that the request originated from a different location, allowing attackers to potentially spoof their IP address and bypass security measures that rely on client IP verification.

To minimize this risk, it’s essential to only trust proxies under your control or those known to handle requests securely.

In Traefik, this is managed through the --entryPoints.web.forwardedHeaders.trustedIPs setting. If a request originates from an untrusted proxy, the X-Forwarded-For header will be unset. This ensures that only IP addresses from trusted sources are considered, preventing IP spoofing and maintaining the integrity of the real client IP address.

When you are in the situation where you have proxies before Traefik, you will need to configure at least of these options:

depth: Specify from right to left, the nth IP address to use. This is needed to avoid spoofing.
excludedIPs: Specify a list of IP addresses to exclude from X-Forwarded-For header.

For example, if the X-Forwarded-For header contains the value 10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1, a depth of 3 will use 11.0.0.1 as the client's IP address, and a depth of 3 with an excluded IP of 12.0.0.1 will use 10.0.0.1.

For this example, the middleware could be configured like:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-ratelimit-with-ip-strategy
spec:
  rateLimit:
    average: 100
    period: 1s
    burst: 200
    sourceCriterion:
      ipStrategy:
        depth: 3

Conclusion

Rate limiting is essential to protect your services against overwhelming traffic, but for enterprises, the challenge often goes beyond just managing individual servers. With distributed rate limiting through Traefik Hub, you can scale your traffic management across multiple gateway replicas, ensuring consistent protection and control no matter how distributed your infrastructure becomes.

By leveraging shared buckets across routers and applying limits at both global and granular levels, Traefik Hub helps enterprises maintain fairness, prevent resource monopolization, and optimize performance. This distributed approach offers robust protection against high-traffic events, making it ideal for large-scale operations where traffic spikes and resource demands are common.

While rate limiting is an essential tool in your security arsenal, it should be part of a broader strategy that includes other measures like Web Application Firewalls (WAF), active monitoring, and proper server configuration. By combining these layers, you can create a robust defense system that keeps your services running smoothly and securely in the face of various internet challenges.

As you implement rate limiting in your Traefik setup, consider your specific use case, traffic patterns, and resource constraints to fine-tune the settings for optimal performance. Regular monitoring and adjustments will help ensure that your rate-limiting strategy continues to meet your needs as your service grows and evolves.

💡

Ready to safeguard your services with flexible, enterprise-grade distributed rate limiting? Experience it firsthand with Traefik Hub. Sign up for a 14-day free trial or speak to our experts today to discover how Traefik can elevate your infrastructure's resilience.

]]>

Taming The Wild West of LLMs with Traefik AI Gateway

Emile Vauge — Tue, 03 Dec 2024 15:13:12 GMT

Within the last ten years, deep learning has moved from a research area to wild industry adoption thanks to the recent generative AI gold rush. This has led to the arrival of Large Language Models (LLMs) into mission-critical infrastructure at an unprecedented pace. Organizations are rapidly discovering that different LLMs excel at different tasks – from OpenAI, Claude or Gemini for generic content creation and problem-solving, up to domain-specific models for highly specialized tasks. This multi-model approach has become essential for maximizing AI's business value.

However, such a gold rush has created a familiar pattern in tech history: rapid adoption leading to fragmented infrastructure and technical debt. Each AI vendor has developed their own proprietary APIs, authentication methods, and SDKs, creating a "Wild West" environment without any governance and where complexity is king. As organizations scale their AI initiatives, this uncontrolled growth threatens to undermine the very benefits these technologies promise.

The Enterprise AI Challenge Landscape

Now that GenAI is being integrated into production environments, the challenges of large-scale AI deployment is becoming more apparent. Let’s dive into the most critical ones.

The number one problem is integration complexity. Development teams struggle with managing multiple SDK implementations, each with its own quirks and requirements. This complexity slows down development and creates maintenance nightmares as teams juggle different API versions and implementation patterns. Additionally, integrations with those different stacks lead to being vendor-locked as there is no straightforward way to migrate from one model to another.

Once integrated with those SDKs, companies end up facing even more issues in production. Security being the most visible. API keys and credentials scattered across various systems create significant security risks. Without centralized control, organizations struggle to maintain secure access patterns and protect sensitive credentials from exposure.

The lack of standardized governance leads to inconsistent policy enforcement across different AI services. Without uniform and centralized authentication, authorization or rate limiting rules, compliance issues and resource waste cannot be avoided.

Finally, this fragmentation also leads to major observability black holes, with no way to optimize costs, monitor performance, or detect issues effectively.

The benefits of GenAI do not come for free and barriers are very real. That’s why at Traefik Labs, we made the decision to tackle those problems, with our unique expertise in ingress and API management.

Enter Traefik AI Gateway

Traefik Labs addresses these new challenges with Traefik AI Gateway, a powerful solution that unifies and streamlines LLM management at the API level. By transforming any AI endpoint into a secure, governable API, Traefik AI Gateway allows enterprises to connect to multiple LLMs through a unified AI API, simplifying integration and centralizing control.

Traefik AI Gateway offers effortless integration with various popular LLMs, eliminating the need for multiple SDKs and client integrations. It ensures centralized security and credential management, minimizing risk and facilitating consistent policy enforcement. It prevents vendor lock-in, enabling easy switching between LLM providers without changing client applications. It supports OpenTelemetry for comprehensive observability across LLM usage, empowering data-driven operations and performance optimizations. Furthermore, Traefik AI Gateway offers unified governance through centralized policy enforcement, ensuring compliance with industry regulations.

Traefik AI Gateway is natively integrated to Traefik Hub and lets organizations gain access to enterprise-grade API capabilities that bring LLMs connectivity to industry standards with advanced access controls, security at scale and premium integrations. It means that AI apps can now benefit from the most advanced APIs features like rate limiting by plan, enterprise access control with identity providers or API keys, deep troubleshooting thanks to traffic debugging, etc.

It’s extremely simple to deploy Traefik AI Gateway, here is a quick overview on how to set it up. You need to have Traefik Hub Gateway up and running to enable Traefik AI Gateway, head over here if that’s not already the case.

First, upgrade the CRDs and your hub deployment enabling the AI gateway feature:

kubectl apply --server-side --force-conflicts \
-k https://github.com/traefik/traefik-helm-chart/traefik/crds/

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set "additionalArguments={--hub.experimental.aigateway}" \
     traefik/traefik

Then, define & apply an AIService resource with any of the supported AI providers, for example OpenAI:

apiVersion: hub.traefik.io/v1alpha1
kind: AIService
metadata:
  name: ai-openai
  namespace: traefik
spec:
  openai:
    token: YOUR_OPENAI_TOKEN
    model: gpt-4o

Finally, attach the ai-openai AIService to an IngressRoute as a TraefikService:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ai-test
  namespace: traefik
spec:
  routes:
    - kind: Rule
      match: Host(`ai.localhost`)
      services:
        - kind: TraefikService
          name: traefik-ai-openai@ai-gateway-service

As simple as that! Now you can request OpenAI on http://ai.localhost:

curl -d '{
    "messages": [
        {
            "role": "user",
            "content": "tell me a joke"
        }
    ]
}' http://ai.localhost

{
  "id": "chatcmpl-AYGYJuSVWlp6gRFVvO9ClTN36eRkQ",
  "object": "chat.completion",
  "created": 1732730143,
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Why don't scientists trust atoms?\n\nBecause they make up everything!"
      },
      "finish_reason": "stop",
      "content_filter_results": {
        "hate": {
          "filtered": false
        },
        "self_harm": {
          "filtered": false
        },
        "sexual": {
          "filtered": false
        },
        "violence": {
          "filtered": false
        },
        "jailbreak": {
          "filtered": false,
          "detected": false
        },
        "profanity": {
          "filtered": false,
          "detected": false
        }
      }
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 12,
    "total_tokens": 23,
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0
    },
    "completion_tokens_details": {
      "audio_tokens": 0,
      "reasoning_tokens": 0
    }
  },
  "system_fingerprint": "fp_831e067d82"
}

Traefik AI Gateway implements the Open Telemetry GenAI semantic conventions which provides metrics (gen_ai.client.token.usage and gen_ai.client.operation.duration) designed to monitor generative AI client applications thanks to specific attributes like token type or model. AI apps present specific challenges compared to traditional software systems and Traefik AI Gateway provides unique LLM observability capabilities to ensure platform teams have the best situation awareness.

Traefik AI Gateway is a natural evolution of Traefik's established expertise in Ingress & API management. Built on top of one of the most widely deployed open-source ingress controllers & API Gateways, Traefik Hub brings its battle-tested experience in managing incoming traffic to the emerging challenge of egress AI traffic.

Shaping the Future of AI Gateways

As enterprises continue to scale their AI initiatives, the need for structured, governed access to LLM services becomes increasingly critical. Traefik AI Gateway provides immediate solutions to today's challenges while laying the groundwork for future developments.

Traefik Labs is deeply invested in the Gateway API effort, the next evolution of Kubernetes Ingress, and is actively working on developing specifications for LLM routing within the Gateway API framework. This ongoing commitment to open standards ensures that Traefik AI Gateway will continue to evolve with enterprise needs.

Organizations looking to bring order to their AI infrastructure should consider Traefik AI Gateway as a strategic investment in their AI strategy. With its combination of production-proven expertise in the ingress & API management space, Traefik is uniquely positioned to help enterprises tame the Wild West of LLMs and build a foundation for sustainable AI growth.

Useful Links

Traefik AI Gateway Documentation
Traefik AI Gateway Website
Our Community Forum

]]>

BigBasket Modernizes Kubernetes Traffic Management with Traefik's API Gateway

Marie Ponseel — Tue, 26 Nov 2024 12:18:08 GMT

Overview

BigBasket, India’s largest online grocer, faced growing challenges in managing traffic across their Kubernetes infrastructure. As their e-commerce platform scaled, their microservices infrastructure became more complex and increasingly difficult to manage, leading to concerns over scalability and operational efficiency. Sushant Gulati, Senior Engineering Manager for Cloud, DevOps, and Platform, alongside Sayantan Bhattacharjee, Principal DevOps Engineer, led the effort to find a robust API Gateway solution to simplify microservice traffic management, improve day-2 operations, and future-proof their infrastructure.

Challenge

Before Traefik, BigBasket’s network architecture relied on NodePort services and AWS ALB Target Groups (TG). While this worked at a smaller scale, this solution quickly reached its limits in terms of scalability and manageability. Each worker node was attached to every ALB target group, causing API throttling due to the excessive volume of register and deregister calls. As traffic grew, the system became unstable, eventually reaching AWS API limits.

In addition to stability issues, scaling became difficult as microservices and hosts multiplied. Infrastructure management was increasingly time-consuming and complex. To address these challenges, they needed a solution capable of simplifying traffic routing while accommodating their growing needs.

Solution

After evaluating several API Gateway solutions, BigBasket’s team chose Traefik for its simplicity, scalability, and performance.

During their evaluation, BigBasket found that other solutions involved complex configuration and required numerous annotations to meet their custom needs, resulting in a high administrative burden. In contrast, Traefik’s use of Custom Resource Definitions (CRDs) made configuration much simpler and easier to manage and customize. Traefik provided a more intuitive solution that significantly reduced management effort while meeting their scalability and performance requirements.

Key factors in selecting Traefik included:

Ease of Management: Traefik’s simpler configuration through CRDs allowed the team to manage their infrastructure with minimal effort, providing much-needed operational breathing room.
Distributed Architecture: Traefik’s API Gateway offered a highly available, scalable, and secure architecture, making it a reliable choice for BigBasket.
Advanced Traffic Management: Traefik’s API Gateway efficiently handled various routing scenarios, such as header-based, host-based, cookie-based, and path-based routing.
High Performance: Traefik's performance, particularly its ability to handle increased traffic loads, was a critical requirement for BigBasket.

“With Traefik, deployment was a breeze, our success criteria were satisfied, and making this major change in our architecture was worry-free knowing we had enterprise support.” Sayantan Bhattacharjee, Principal DevOps Engineer at BigBasket

Implementation

BigBasket deployed Traefik’s API Gateway as their primary solution across five Kubernetes clusters, all running in AWS. The new Traefik architecture became the foundation of their network, handling every transaction across their e-commerce platform. All their critical services are now routed through Traefik’s API Gateway, making it an indispensable component of their infrastructure.

BigBasket architecture with Traefik

The team significantly simplified their routing configuration by eliminating the need for multiple AWS target groups and application load balancers (ALBs). This not only reduced complexity but also allowed them to automate much of their setup and significantly improve the management of their architecture.

Traefik’s API Gateway also enabled BigBasket to manage five different versions of the same service within a single cluster, supporting their testing needs without requiring additional tools or investments.

“Traefik is simple to configure, doesn’t have component sprawl, and has significantly reduced our service onboarding and route management efforts.” Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

Visibility into traffic management has improved substantially. With Traefik, the team gained deeper insights into traffic flow, making it easier to monitor and troubleshoot issues. The ability to track everything in real-time has been transformative and allowed them to efficiently manage and optimize their system. Everyone involved at BigBasket found Traefik simple to manage, resulting in smoother day-to-day operations.

Grafana dashboard showing Traefik’s request count

Additionally, the support from Traefik Labs played a crucial role in their success. The team praised Traefik's proactive approach, noting how helpful their experts were during setup and in answering ongoing questions. Regular meetings with the Traefik Labs team ensured that BigBasket received continuous guidance and assistance in resolving any challenges.

“The support from Traefik has been very helpful, the team is very knowledgeable, forthcoming, and open to discussions.” Sayantan Bhattacharjee, Principal DevOps Engineer at BigBasket

Results

Since adopting Traefik’s API Gateway, BigBasket has experienced the following improvements:

Scalability without limitations: Traefik has enabled BigBasket to scale seamlessly alongside their growing business The team has already increased their Traefik adoption by 50% within the first 12-months to support the expansion of their infrastructure needs. Had they remained on their previous solution, they would not have been able to scale to this extent.
High availability and reliability: Over the past year and a half, BigBasket experienced no stability issues in their clusters, even during periods of significant traffic spikes, thanks to Traefik’s robust and stable environment.
Simplified Traffic Management: Traefik eliminated the need for managing thousands of AWS Target Groups and ALBs, significantly reducing complexity and administrative overhead. This allowed BigBasket to handle a 2x increase in internal traffic without performance degradation.
Reduced Complexity: By consolidating traffic management through Traefik’s API Gateway, BigBasket reduced the number of components needed to manage their system, freeing up engineering resources for more strategic tasks.
Enhanced Visibility: Traefik’s built-in metrics and dashboard provided BigBasket with real-time insights into their traffic flow. This increased visibility, made troubleshooting easier, and allowed the team to respond quickly to system changes. The ability to monitor traffic at a granular level—down to requests-per-second (RPS)—has been critical in optimizing system performance and ensuring smooth operations.

“Traefik has exceeded our expectations; it has done a lot more for BigBasket than what is normally expected out of an API Gateway.” Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

Key Takeaways

Traefik’s API Gateway has transformed BigBasket’s approach to managing Kubernetes traffic, delivering significant improvements in efficiency, scalability, and system stability. Traefik’s distributed architecture, automation, and performance have allowed BigBasket’s team to future-proof their infrastructure. Key outcomes include:

50% increase in Traefik usage, to support the expansion of their infrastructure.
2x increase in east-west traffic, handled seamlessly without impacting performance.
Elimination of 7,000 NodePort configurations, drastically reducing the operational complexity.
Mitigated AWS API throttling, resulting in fewer errors and operational headaches.

Sushant and his team at BigBasket continue to expand their use of Traefik, confident that it provides the flexibility, visibility, and stability needed to support their business at scale.

“Traefik has been a game-changer for us. Its ease of configuration, scalability, and seamless support have exceeded our expectations. Traefik is a strategic tool that has transformed how we manage traffic across our infrastructure. I would highly recommend Traefik to any organization looking to modernize their Kubernetes environment.” — Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

]]>

How Traefik Labs is Pioneering the Kubernetes Gateway API Revolution

Emile Vauge — Tue, 29 Oct 2024 15:00:01 GMT

Since Kubernetes’ inception, the Ingress specification has been a reliable solution for managing inbound HTTP/S traffic, helping to expose services externally. As a leader in the Cloud-Native industry, Traefik was one of the first Ingress Controllers on the market and played a key role in the adoption of this API.

However, as organizations increasingly scale up their Kubernetes deployments and require more flexibility, the limitations of Ingress become apparent. Enter Kubernetes Gateway API, a new specification designed to address those limitations, introducing a more advanced and extensible model. It is no surprise that Traefik is playing a key role in this specification effort being one of the most complete Gateway API implementations on the market.

This article explores why the Gateway API is set to replace the traditional Ingress specification and how Traefik Labs drives this joint effort.

Exposing Workloads in Kubernetes: A Brief History

When Kubernetes was first introduced, it had different networking concepts: Services, Load Balancers and Ingress. Kubernetes Services allowed applications within a cluster to communicate, load balancers were used to provision cloud provider LBs, but when it came to exposing workloads externally, we used (and are still using) Ingress, a resource designed to handle HTTP/S routing to services. Ingress was a good foundation, but it soon became clear that it had limitations. It relied on provider-specific implementations with countless specific annotations, lacked flexibility, and offered only basic HTTP traffic management. Mastering the Ingress API covers only about 20% of the knowledge needed to address your needs, the next 80% is the “annotation hell”:

kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: https://example1.com/
    nginx.ingress.kubernetes.io/session-cookie-name: route
    nginx.ingress.kubernetes.io/too-many-annotations: "true"

This left many teams needing more advanced solutions to handle complex networking scenarios. That’s why we introduced the IngressRoute resource with Traefik v2.0 in 2019. The idea was to solve the annotation hell with a much more powerful structured CRD, but this was at the cost of vendor neutrality as it was only available for Traefik.

It was clear that Kubernetes needed a standardized way to manage inbound networking more elegantly.

Gateway API Comes to the Rescue

Kubernetes Gateway API is the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs. It has been incubating since 2019 and used to be called Service API at first. It introduces new concepts that simplify and standardize how Kubernetes clusters manage traffic, both internally and externally. Gateway API is designed to be vendor-neutral, layered and flexible at its core, and future-proof, addressing many of the shortcomings of Ingress resources.

Role-Based Resources

Gateway API introduces a set of resources that decouple networking responsibilities: GatewayClass resources which define the gateway controller and Gateway resources represent entry points into the cluster, can be managed by the operators of the cluster.

As for Developers, they will specifically use HTTPRoute or TLSRoute resources which define how traffic coming via the gateway maps to the services.

This separation of responsibilities makes it easier to define clear boundaries between different roles, allowing developers, network admins, and operations teams to work seamlessly together.

Layered Flexibility

Unlike the Ingress resource, which focused on HTTP/S traffic, Gateway API supports routing for multiple protocols like HTTP/S but also TCP, UDP and gRPC. This makes it an ideal choice to handle a diverse set of traffic types.

Another powerful concept is how policies can be applied at various layers—global, gateway-specific, or route-specific—allowing for a fine-grained control over how traffic is managed. Thanks to the Policy Attachment pattern, it is even possible to augment the behavior of an object that can't be described within the spec of that object.

Expressiveness & Vendor Neutrality

One of the key goals of Gateway API is to provide a consistent API across different environments. The core features set comes with header-based matching, traffic weighting, and many other capabilities that were only possible in Ingress through custom annotations. This allows developers to define routing rules and policies without being tied to a specific vendor’s implementation (through annotations). Whether you use Traefik, NGINX, or another provider, Gateway API aims to offer a powerful yet consistent experience.

Extensibility

Gateway API has been designed with extensibility in mind. Custom resources can be “augmented” at basically any layer of the API with Metaresources or ExtensionRef. This enables precise customization at the right levels within the API structure.

As you can see, Gateway API is much more powerful and flexible than Ingress. Most common exposing patterns can be tackled out of the box, and more complex needs can be addressed with extension mechanisms. This is a breath of fresh air for anyone who has been dealing with dozens or hundreds of Ingresses in production, familiar with the vendor-centric annotation hell.

Traefik Labs, Leader in the Gateway API Effort

Traefik Labs has always been a leader in the Kubernetes industry, being one of the very first Ingress Controllers back in 2016. Today is no different, Traefik is strongly involved in the Kubernetes Gateway API effort, playing a significant role in its development and adoption. Traefik was amongst the first controllers to adopt and support the Gateway API first alphas, recognizing its potential to simplify Kubernetes inbound traffic. Since then, Traefik Labs has been actively involved in the Kubernetes SIG-Network community, contributing to the development and refinement of the Gateway API specification. This involvement has allowed Traefik to influence the direction of the API, pushing for features that enhance user experience, security, and flexibility.

Traefik has been integrating Gateway API 1.0 since v3.0 and is continuously investing to provide the latest version available. Today, Traefik v3.2 brings support to fresh new Gateway API 1.2 adding GRPCRoute, TLSRoute, as well as more features on the HTTPRoute like h2c (HTTP/2 without TLS) and WebSocket support for backends, port matching in rules, response header modification, etc.Traefik Labs is committed in the long term to continually investing in the support and enhancement of Gateway API, ensuring that the community will always benefit from the latest progress in that space.

Conclusion

Gateway API is not just an improved Ingress resource. It’s basically a full replacement of Ingress, addressing all major flaws of this (too) basic specification and bringing advanced features to cover every need in cloud-native connectivity. By standardizing how inbound and east-west traffic is managed, Gateway API is making Kubernetes networking easier and more accessible for everyone. Its extensibility and flexibility will enable companies to tackle more complex workload exposure use cases, without sacrificing vendor neutrality.

As a pioneer of the Cloud-Native ecosystem, Traefik Labs has been at the forefront of designing the future of Ingress. This journey started back in 2019 and five years were needed to complete this giant step to the first GA of Gateway API. Today, as one of the most complete Gateway API implementations in the industry, we are extremely proud of the work accomplished. Seeing the impressive ongoing activity on the project, there is no doubt that Gateway API is only at the beginning of this joint effort.

Useful Links

Gateway API documentation
Traefik 3.2 release notes on Github
Getting started with Kubernetes Gateway API and Traefik

]]>

Traefik Proxy v3.2 - A Munster Release

Emile Vauge — Tue, 29 Oct 2024 14:58:03 GMT

After 3 months of intensive development and two release candidates, we are thrilled to announce the general availability of Traefik v3.2. With Halloween just around the corner, we wanted a munster codename for this new version 👹. Let’s dig deeper into the key changes.

As Traefik Labs continues to lead the Kubernetes Gateway API effort, it is no surprise that the new Gateway API v1.2 is supported out of the box in this latest Traefik release. We will explore the latest additions brought by the most recent development.

And that’s not all! The team has been working very hard on building an alternative HTTP reverse proxy engine for quite some time now, and the day has finally come to make it available to everyone. This Fast Proxy engine offers a whopping ~ 50% performance boost compared to the standard engine. Yes, that means approximately 50% more requests per second 💥

The clue's in the name—we're calling it a 'Munster' release for a (spooktacular) reason! Let’s dive in!

Performance Breakthrough

The Traefik team has been working on writing a complete new HTTP/1 engine. It took a long time to converge and we are extremely excited to announce that the new Fast Proxy engine is available for everyone in experimental.

experimental:
  fastProxy: {}

The Fast Proxy engine is a high-performance reverse proxy designed to enhance the performance of routing based on a zero allocation pipeline. This new engine significantly improves performance, boasting a remarkable 50% increase in speed compared to the standard engine.

Here are the results of an analysis conducted on our benchmark platform using the standard vs. the Fast Proxy Engine.

The difference between both engines is mind-blowing, you can now expect around 50% increase in throughput and 50% decrease in latency handled by Fast Proxy Engine through this new high performance proxy engine.

Kubernetes Gateway API v1.2

Traefik Labs has always been a key player in Kubernetes inbound traffic management. Gateway API, seen as the next generation for Ingress, is probably one of the most active Kubernetes projects. Traefik was amongst the first controllers to adopt and support the Gateway API first alpha release, and since v3.0, Traefik has continuously integrated updates. Today, Traefik v3.2 introduces support for the newly released Gateway API 1.2, and brings new additions like GRPCRoute and TLSRoute, as well as additional HTTPRoute features such as h2c (HTTP/2 without TLS), WebSocket support for backends, response header modifications, and more.

Let’s start with GRPCRoute, which is a type for specifying routing behavior of gRPC requests. gRPC is a widely adopted RPC framework popular across the industry, within Kubernetes itself. Therefore, a specific type to route this protocol provides a more granular configuration.

Similarly to HTTPRoute, the specification let you define hostnames (a list of hostnames to match against the Host header of the gRPC request), matches (conditions used for matching gRPC requests), filters (to process or alter the request) and backendRefs (defines the backend services to which matching requests should be sent). Here is an example of a GRPCRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
  name: grpc-route
spec:
  parentRefs:
  - name: traefik-gateway
  hostnames:
  - "example.com"
  rules:
  - filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: my-header
            value: foo
  - matches:
    - method:
        service: com.example.User
        method: Login
    backendRefs:
    - name: login-v1
      port: 50051
  - matches:
    - headers:
      - type: Exact
        name: magic
        value: foo
      method:
        service: com.example.Foo
        method: DoFoo
    backendRefs:
    - name: foo-v1
      port: 50051
      weight: 90
    - name: foo-v2
      port: 50051
      weight: 10

TLSRoute support has also been added to Traefik in v3.2’s experimental channel, to enable multiplexing TLS connections via SNI.

apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: tls-route
spec:
  hostnames:
  - example.com
  parentRefs:
  - name: traefik-gateway
  rules:
    - backendRefs:
        - name: backend-tls
          kind: Service
          port: 443

Introduced with Gateway API v1.2, Backend Protocol (through the appProtocol field) can now be set to specify the underlying protocol when a Route's backend references a Kubernetes Service. Currently, you can choose between http, https, kubernetes.io/h2c for HTTP/2 without TLS and kubernetes.io/ws for websocket over HTTP.

apiVersion: v1
kind: Service
metadata:
  name: test
spec:
  selector:
    app: test
  ports:
  - protocol: TCP
    appProtocol: kubernetes.io/h2c
    port: 8080
    targetPort: 8080

The ResponseHeaderModifier filter is now supported since Traedfik v3.2. It allows to setsetting a custom header for all responses being sent.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-response-header
spec:
  hostnames:
    - response.header.example
  rules:
    - backendRefs:
      - name: example-service-beta
        weight: 50
        port: 80
        # set a custom header for all responses being sent from the
        beta build of the backend server.
        filters:
           - type: ResponseHeaderModifier
             responseHeaderModifier:
               add:
                 name: build
                 value: beta
      - name: example-service-stable
        weight: 50
        port: 80

To learn more about how Traefik supports Gateway API, we encourage you to visit the Traefik Documentation.

Other Improvements

In addition to these exciting new features, Traefik v3.2 brings several updates to ACME certificate management (Let’s Encrypt), including the ability to use different email addresses in certificate resolvers (#11019 by Emrio), the support for custom CA certificates in certificate resolvers (#10816 by ldez), and a 30-days certificatesDuration threshold (#10970 by luker983).

BasicAuth support has been added to the Docker and Swarm endpoints (#10776 by 985492783).

Middlewares have been improved with IPv6 subnet support in ipStrategy for RateLimit and InFlightReq (#9747 by michal-kralik), an option to log the user in the ForwardAuth middleware (#10833 by GaleHuang), a new compression encodings option (#10943 by wollomatic), and the ability to mirror the full body in the mirroring (#11032 by MatteoPaier).

OpenTelemetry metrics now allow you to configure service.name for scenario with several Traefik instances (#10917 by cmartell-at-ocp), while access logs now capture trace IDs and EntryPoint span IDs (#10921 by weijiany).

A configurable maximum request header size option has also been added, allowing users to specify the maximum size for HTTP request headers beyond the default 1MB limit (#10995 by lucasrod16).

For detailed information on all changes, please refer to the full release notes on GitHub.

We extend our gratitude to all contributors for their efforts and dedication, which has made Traefik 3.2 an exceptional release. Thank you for driving Traefik forward!

Moving Forward

Traefik Proxy v3.2 is a major leap forward in performance, functionality, and security. This release introduces a groundbreaking Fast Proxy engine, which delivers unparalleled speed and efficiency. As a key contributor to the Kubernetes Gateway API project, Traefik Labs ensures cutting-edge support for the latest Gateway API v1.2 specifications. Along with many other enhancements, Traefik 3.2 empowers developers and organizations to effortlessly build and manage modern, cloud-native applications with unprecedented efficiency.

We encourage you to explore the new features and capabilities of Traefik 3.2 and experience the impact they can make in your infrastructure. Join the vibrant Traefik community, share your feedback, and help us shape the future of cloud-native networking.

Useful Links

Traefik 3.2 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Deepening The Synergy: HashiCorp and Traefik Labs’ Journey Toward a Seamless Cloud-Native Stack

Emile Vauge — Mon, 14 Oct 2024 15:28:44 GMT

Cloud-native infrastructure is evolving rapidly, with agility, security, and scalability increasingly becoming key table stakes. As such, organizations are constantly seeking integrated solutions to streamline their operations, and HashiCorp and Traefik Labs are one of the best examples of a deep integration between two stacks. What began with early integrations, with Traefik Proxy and Hashicorp Consul, has matured into a deep symbiosis of technologies with the addition of Traefik Enterprise, Traefik Hub, Hashicorp Vault and HashiCorp Nomad. Traefik Labs being recognized as the “2022 HashiCorp Integration Partner of the Year” showcases this strong unity that helps enterprises architect modern cloud-native infrastructures with ease.

The Early Days

The foundation of this partnership was laid with the integration of HashiCorp Consul into Traefik Proxy, Traefik Labs’ open-source reverse proxy and ingress controller. As early as Traefik’s inception back in 2015, the team at Traefik Labs was embracing service discovery and dynamic configuration patterns. Obviously, Consul from HashiCorp, was a natural fit.

At its core, Consul is designed to handle service discovery, health checks, and distributed system coordination across environments, making it the perfect complement to Traefik’s dynamic routing capabilities. Consul is integrated as a KV store and as a service catalog within Traefik, offering the maximum flexibility to fit companies’ needs.

As enterprises adopted cloud-native infrastructures, this combination of Traefik’s efficient ingress management and Consul’s robust service discovery made the duo indispensable in production environments.

Growing the Partnership

The expansion of both companies’ business offerings created the opportunity to deepen this collaboration, particularly with the introduction of Traefik Enterprise and HashiCorp’s Vault and Nomad into the mix.

Traefik Enterprise & HashiCorp Vault: Security has become an increasingly critical concern for organizations as they shift to microservices-based architectures. Vault, HashiCorp’s flagship solution for secrets management, integrates seamlessly with Traefik Enterprise, enabling enterprises to securely manage sensitive data. Companies can leverage Vault PKI as a TLS certificate resolver in Traefik Enterprise or can unlock the potential of a fully distributed ACME (Let’s Encrypt) setup. This is particularly essential in highly dynamic production environments, where breaches of sensitive data can be catastrophic.

Traefik Enterprise & HashiCorp Nomad: HashiCorp’s Nomad, a highly efficient workload orchestrator, became another pivotal piece in the collaboration. Designed to work across cloud and on-premises environments, Nomad excels at scheduling and managing containerized services at scale. Paired with Traefik Enterprise, this combination ensures that workloads are automatically discovered and routed without downtime or manual updates.

Nomad’s orchestration, when aligned with Traefik’s smart routing, delivers a streamlined solution for managing microservices, making deployment and scaling more seamless than ever before. This integration is particularly valuable for organizations looking for flexible, multi-cloud deployment options while maintaining high performance and operational efficiency.

Together, these integrations create a secure, highly automated, and flexible platform that enables enterprises to deploy, scale, and manage applications across any environment. With Traefik Enterprise and HashiCorp’s Vault and Nomad, organizations can achieve greater levels of automation while maintaining security, simplifying operations, and minimizing human intervention.

Maturing the Partnership

Traefik Hub & HashiCorp Vault: The maturity of the HashiCorp-Traefik Labs partnership reached a new level with the introduction of Traefik Hub, the industry's first Kubernetes-native API Management solution for publishing, securing, and managing APIs. Traefik Hub represents the most advanced solution in Traefik Labs’ portfolio, providing end-to-end API management that supports complex environments at scale. Built on the foundations of Traefik Enterprise’s proven routing technology, Traefik Hub API Gateway integrates deeply with Vault to offer an automated TLS certificate PKI for all APIs handled by Traefik Hub. The Hub & Vault combo provides the most advanced and secured API Gateway solution on the market.

Traefik Proxy/Hub & HashiCorp Nomad: The Nomad integration into Traefik Proxy and Traefik Hub continues to be improved and the latest Traefik v3.2 release brings a new watch option to the Nomad provider which refreshes the configuration on a per-event basis instead of the less effective pull mechanism.

Finally, both companies have been working hand in hand to publish a workload modernization integration guide with Traefik Enterprise to embed an API Gateway into Hashicorp Nomad clusters running Consul and Vault. The idea was to create a definitive guide for operation, platform and security teams to deploy a modern platform to seamlessly and securely publish APIs.

The impact? Outstanding success stories from clients across the world, highlighted recently by BT Group:

“We adopted Traefik OSS to help build a new container platform on HashiCorp Nomad in AWS, following the closure of one of our on-prem data centers. We needed something that integrated smoothly with Consul and had minimal overhead while handling spikes in demand during major events. Traefik's seamless integration with Consul made it easy to implement, and it’s been so reliable that we rarely have to think about it. As our platform grew, we needed to support more use cases around security and wanted operational support, which led us to adopt Traefik Hub. Being able to secure our APIs quickly by adding a few Consul tags has been a huge time-saver—our teams can focus on development without worrying about complex configuration.” ⎯ Tom Davies, Principal Software Engineer, BT Group

Conclusion

The deep integration of HashiCorp and Traefik Labs stacks has been transformative for enterprises navigating the complexities of cloud-native environments. With the synergy between Traefik Enterprise, Vault, and Nomad already providing robust solutions for secure microservices orchestration, the introduction of Traefik Hub represents a leap forward in API management.

This partnership continues to evolve, driving innovation in cloud-native technologies and creating tools that allow enterprises to thrive in an era where security, automation, and flexibility are key. As this collaboration deepens, the future of cloud-native infrastructure looks more agile, resilient, and secure than ever before. Visit our Traefik & HashiCorp page to learn more.

]]>

Implementing Runtime API Governance in Traefik Hub

Immánuel Fodor — Wed, 18 Sep 2024 03:41:00 GMT

In a recent blog post, we discussed the top five policies for runtime API governance. Now, let’s look at how Traefik Hub implements these policies. Traefik Hub is a Kubernetes-native API management solution built with scalability, flexibility, and simplicity in mind. Traefik Hub helps route traffic to services in dynamic, microservices-oriented environments.

In the following sections, we will provide examples of configuration implementation in Traefik Hub to support the standards we have discussed. But note that our examples are not exhaustive - Traefik Hub supports many more capabilities and configuration options!

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments

As a Kubernetes-native solution, Traefik Hub supports declarative resource configuration. Traefik Hub configuration can be done through Kubernetes resources like Ingress and Service, and Traefik Custom Resource Definitions (CRDs) like IngressRoutes, Middleware, API, APIPortal, APIAccess, APIVersion, and APIRateLimit. Configuration can be version-controlled in Git and automatically applied to the Kubernetes cluster using GitOps tools like ArgoCD and Flux.

For example, defining external traffic routing to a service involves creating an API CRD and exposing the API with an IngressRoute. The following resource definition declares an API called hello-api, and provides the path to its OpenAPI description.

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
name: hello-api
namespace: apps
spec:
openApiSpec:
path: /openapi.yaml

Next, specify an IngressRoute CRD with routes to connect incoming requests to services that can handle them. The following resource definition declares a router that matches a request with the host api.example.com to the hello-api-service.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: hello-api-ingress
namespace: apps
annotations:
hub.traefik.io/api: hello-api # Name of the API object
spec:
routes:
- match: Host(`api.example.com`)
kind: Rule
services:
- name: hello-api-service
port: 8080

Developers can save configurations to a Git repository, and GitOps operators ensure that these configurations are consistently applied across environments. The diagram below illustrates this process.

The declarative nature of Traefik configuration, combined with GitOps workflows, supports advanced and progressive deployment capabilities. Teams can follow blue-green deployments and canary releases using GitOps operators like ArgoCD and FluxCD. If any issues are detected, changes can be rolled back to a stable version by reverting the changes in the Git repository.

TIP: For an in-depth treatment on how to do load-balancing in Traefik and support advanced deployment techniques, see Traefik’s Advanced Load Balancing course.

Policy 2: Make comprehensive and correct API documentation available to users

Traefik Hub API management includes an API Developer Portal to make API documentation accessible to developers and other API users. In the portal, users can explore available API endpoints, understand their usage, and test them in real time.

The APIPortal CRD creates and configures the portal, generating a web interface for browsing the API documentation. The visibility of API documentation depends on user groups. An API will be visible to an API consumer if they belong to a specified group with access to the API, as configured by the APIAccess CRD.

You can create an API developer portal by applying an APIPortal resource as follows:

---
apiVersion: hub.traefik.io/v1alpha1
kind: APIPortal
metadata:
name: my-portal
namespace: default
spec:
title: "My Portal" # The title for the Portal
description: "API documentations" # A short description of the Portal
trustedUrls:
- "https://portal.example.com"
ui:
logoUrl: https://traefik.io/favicon.png # URL to a picture used as logo

To expose the portal, create an IngressRoute resource, as demonstrated below:

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: my-portal-ingress
namespace: traefik-hub
annotations:
hub.traefik.io/api-portal: my-portal@default # Reference the APIPortal
spec:
routes:
- match: Host(`portal.example.com`)
kind: Rule
services:
- name: apiportal
port: 9903

Policy 3: Secure all API traffic from malicious and unauthorized access

With Traefik Hub, you can ensure secure API access using protocols such as Oauth2, OpenID Connect (OIDC), API keys, and JSON Web Tokens (JWTs). Additionally, Traefik integrates with identity providers such as Keycloak and Okta. Traefik also integrates with any other IdP that supports the OIDC protocol to handle user identities and authorize access to the APIs and API portals.

To protect APIs from DDoS attacks or excessive requests from a single client, you can implement rate limits by defining an APIRateLimit resource. Traefik allows you to do this for all APIs or specific user groups. The following declarative config defines a rate limit of 100 requests per minute.

apiVersion: hub.traefik.io/v1alpha1
kind: APIRateLimit
metadata:
name: my-rate-limit
namespace: apps
spec:
# Rate limit configuration, this config allows 100 requests/minute.
limit: 100 # 100 requests
period: 1m # One minute
groups:
- support
apiSelector:
matchLabels:
module: crm

Using Traefik’s Distributed Rate Limit (which is based on the Token Bucket algorithm), you can limit requests over time across your entire cluster rather than just an individual proxy.

In addition, Traefik Proxy v3 includes the Coraza Web Application Firewall (WAF) integrated as a plugin. Coraza is an open-source OWASP project and a high-performance WAF that supports Modsecurity's seclang language and OWASP core rule sets. Enabling this involves two simple steps:

- First, updating Traefik’s static configuration that loads the plugin as follows:

experimental:
plugins:
coraza:
moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
version: v0.2.2

- Then, updating the WAF CRD as follows:

---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: coraza-waf-block-admin-path
namespace: apps
spec:
plugin:
coraza:
directives:
- SecRuleEngine On
- SecDebugLog /dev/stdout
- SecDebugLogLevel 9
- SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"

Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes

Traefik Hub helps you understand the impact of API configuration changes by performing static checks on manifest files. It includes a Static Analyzer tool that allows you to check your Traefik Hub manifest files for consistency and quality. The Static Analyzer can lint manifest files and generate differential reports. It can be used as a CLI tool or as part of a CI/CD pipeline. The linter can detect issues such as:

Childless resources
Duplicate resources and resource references
Unknown operation sets
Orphan resources
Invalid resource references
Invalid regular expressions
Duplicate releases of a given API
Invalid resource definitions
Invalid selector definitions

The following is an example of the linting error message displayed on the PR that introduces a change.

The Static Analyzer tool can also run a diff to generate change reports. For example, a report like the following makes it clear that the rate limit was changed. It shows the user group and APIs affected by the change. If that was not the developer’s intention, the developer can take remedial action.

To aid API evolution, Traefik Hub also supports several API versioning schemes. Teams can define and use an API version based on host, URI path, HTTP method, query parameters, media/content type headers, custom headers, client IP, auth user (basic/forward auth), JWT claims, and more. Traefik even allows teams to mix different versioning schemes based on logical expressions.

Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery

Traefik Hub provides metrics and tracing information using the OpenTelemetry (OTel) format, to help teams monitor and gain insights into the traffic flowing through their services and APIs. Traefik Hub has the most comprehensive OTel support in the industry, with over 20 metrics and 15 labels. Additionally, it supports vendor-specific metric systems such as Prometheus, Datadog, InfluxDB, and StatsD.

For instance, to set up the OTel metrics, first get the configuration values for the Traefik Hub Helm release:

helm get values traefik-hub -n traefik | tail -n +2 > values.yaml

Then, modify and apply the configuration values in values.yaml.

metrics:
# Disable Prometheus (enabled by default)
prometheus: null
# Enable providing OTel metrics
otlp:
enabled: true
http:
enabled: true
endpoint: http://myotlpcollector:4318/v1/metrics
# Enable providing OTel traces
tracing:
enabled: true
http:
enabled: true
endpoint: http://myotlpcollector:4318/v1/traces

By integrating with monitoring systems, you can configure alerts and notifications for critical metrics and performance indicators. Traefik can also be connected to pre-built Grafana dashboards to visualize a wide range of API metrics. This allows you to correlate deployment events with incident management, enhancing your ability to monitor and respond to issues effectively and improve MTTR. For more details on Traefik Hub’s integration with Grafana, see this webinar.

Additionally, Traefik provides a built-in dashboard that offers a real-time view of the current status of routes, services, middleware, and other components.

GitOps Implementation Example

The first policy we highlighted at the beginning of this article was about providing a developer-friendly, traceable, and easy-to-rollback approach to API deployment. We also mentioned in Standard 1.2 that GitOps was an implementation of this approach. In the Traefik Hub GitOps Tutorial project in GitHub, we provide a hands-on tutorial to demonstrate the power of GitOps for API configuration deployment and change management. It also demonstrates Traefik Hub’s integration with API monitoring and observability visualization tools. The tutorial uses the following tools:

Aspect	Tool
API Management	Traefik Hub
Software platform	Kubernetes cluster, running locally in kind or k3d
GitOps Deployment Tool	FluxCD
Git repository	GitHub
Metrics visualization	Grafana
API monitoring and alerting	Prometheus

To work on this project, request a Traefik Hub trial here. Then follow the instructions on the README page, clone the repository, and set up kind and FluxCD. In the tutorial, FluxCD watches your fork of the GitHub repository for changes. When a change to the master branch is detected, it deploys the latest version of the configuration file to the cluster.

Summary

In the part 1, we introduced five essential policies for effective runtime API governance:

A developer-friendly API deployment process
Comprehensive and correct API documentation available to users
Secure both ingress and egress API traffic,
Clear API change impact analysis
Proactive API monitoring and deep observability.

By establishing standards based on these policies, organizations can enforce runtime API governance more effectively. The primary objective of improving runtime API governance is to improve software delivery performance (as measured by the DORA metrics - DR, LTC, CFR, and MTTR). It also aims to enhance the API developer experience for both API producers and API consumers.

To successfully implement these runtime governance policies, it's crucial to leverage a modern API management platform, such as Traefik Hub, which emphasizes dynamic change management and runtime API governance at its core. As a Kubernetes native API management solution, Traefik Hub is optimized for end-to-end API delivery automation which improves operational efficiency. To learn more, get a personalized demo of Traefik Hub today.

]]>

Top 5 Policies for Runtime API Governance to Ensure Your APIs are Consistent, Compliant, and Secure

Ikenna Nwaiwu — Wed, 18 Sep 2024 03:15:03 GMT

API governance provides a framework for defining policies and standards to ensure APIs are consistent, compliant, and secure. However, a common misconception about API governance is that it is solely about API design concerns, such as following a specified pagination pattern or ensuring API descriptions adhere to specific linting rules. While these design-time considerations are important, they’re only part of the picture. API governance is not just about defining design standards; it’s the backbone of ensuring that APIs are consistent, compliant, and secure across their entire lifecycle.

Overlooking runtime governance—like monitoring API performance, managing access controls, and enforcing security protocols—can lead to catastrophic failures in production. For instance, imagine a beautifully designed API that lacked proper runtime governance. Without robust security measures, attackers could exploit it, leading to data breaches and significant financial loss. Or consider an API with no runtime monitoring; a sudden surge in traffic could cause unanticipated outages, bringing down critical services and eroding user trust. Effective API governance addresses design and runtime concerns, ensuring that APIs meet standards during development and perform reliably and securely in the real world.

API management teams are responsible for defining and enforcing runtime API governance. Runtime API governance is the structured framework that manages and enforces policies for API deployment, security, observability, and lifecycle during their operational phase. It ensures APIs are deployed in a traceable and secure manner, with all configurations version-controlled and changes managed through a standardized process. This governance also mandates that live APIs be centrally documented, monitored for performance, and secured via gateways and firewalls. Overall, API governance provides a controlled environment to maintain API integrity, security, and efficiency in real-time operations.

Given how crucial API governance is to an organization’s API strategy's success, let's explore the topic further. In this post, we discuss five policies we recommend for API teams. They are listed below.

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments
Policy 2: Make comprehensive, correct API documentation available to users
Policy 3: Secure ingress and egress API traffic from malicious and unauthorized access
Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes
Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery.

Each policy has one or more actionable standards that teams should adopt to meet these criteria.

Before we dive in, here is a note on the terminology we use in this post. By policy, we mean a high-level statement that requires compliance with one or more associated standards. A policy focuses on results and not implementation. On the other hand, by standards, we mean detailed mandatory, recommended, and optional rules used to gauge compliance with a given policy. Standards are specific and measurable, using capitalized keywords from IETF RFC-2119 (‘MUST’, ‘MUST NOT’, ‘SHOULD’, ‘SHOULD NOT’, ‘MAY’ etc.) to indicate the level of requirement compliance.

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments

The DORA metrics, Deployment Frequency (DF), Lead Time for Change (LTC), Change Failure Rate (CFR), and Mean Time to Recovery (MTTR), are widely used to assess a software team's performance. However, there's now an increased emphasis on developer experience and productivity. The State of Developer Experience Report 2024 revealed that 69% of developers are losing 20% or more of their time due to inefficiencies at work. The survey also explored the key areas that engineering leaders believe will enhance developer productivity and satisfaction. One of the top five areas identified was improving collaboration tooling.

When making changes to your API, it's crucial to have a deployment process that allows for fast deployment, easy experimentation, and the ability to roll back failed deployments and restore service quickly. Such a deployment process requires an automated deployment system to minimize manual errors. A deployment process like this is a key enabler for reducing LTC and improving software delivery performance. Additionally, the deployment process should facilitate seamless collaboration between development and operations teams and provide robust access controls to determine who can make changes. Furthermore, the process should also maintain an auditable history for traceability and accountability.

Here are three standards that can help realize this policy.

Governance Standard 1.1: All API descriptions and configuration MUST be version-controlled in Git

Managing API descriptions (like OpenAPI) and API configuration files (like Kubernetes Ingress manifests) in Git enables a workflow that provides a clear audit trail, giving a history of what was changed, when, and who changed it. This traceability makes it easier to troubleshoot changes and roll back a change that introduces a failure with a single Git revert command. It also enables developers to edit the files using editors, CLIs, and any preferred tool.

Governance Metric: The ratio of the organization's APIs whose configuration is under version control.

Governance Standard 1.2: All API configuration changes MUST follow a GitOps approach

GitOps has emerged as the modern way to manage application and infrastructure deployments. As part of the GitOps approach, the application state is managed in Git repositories, which act as the source of truth for the system's desired state. Application state is stored using declarative descriptions in formats like YAML and JSON. When the configuration files are updated, continuous delivery tools synchronize the application's live state with the desired state.

A GitOps approach allows for the fast delivery of API changes. Automated governance checks in the deployment pipeline enable organizations to achieve fast LTC and improved API consistency. Pull requests can be a lightweight governance technique to promote collaboration on changes. Teams following the GitOps approach disallow configuration changes from the application UI, eliminating error-prone TicketOps and ClickOps-based API deployments.

Governance Metric:

Number of API configurations managed with GitOps.
API config change review speed: The average time it takes from opening a config change PR to completing a review on it.

Governance Standard 1.3: The CI/CD pipeline MUST act as the primary change agent, driving all changes, feedback, and collaboration through automated processes, with no manual steps outside the pipeline.

A CI/CD pipeline should include all the necessary steps to validate and deploy working software to production. These steps involve compiling and packaging source code, conducting unit, integration, and acceptance tests, and deploying to progressively higher environments. In this way, the pipeline acts as both a central falsification mechanism and a primary change agent, ensuring that all changes, feedback, and collaboration are processed through automated workflows, with no manual intervention. If a new software change (that is, a commit) presents the hypothesis that “this change complies with standards and does not break working software,” then the CI/CD pipeline serves as a falsifiability test to evaluate that hypothesis.

By automating collaboration patterns, the pipeline ensures that feedback loops are consistent and transparent across teams. In this way, the CI/CD pipeline becomes not only an objective evaluator of changes but also a facilitator of cross-team feedback and collaboration. No manual steps should occur outside of the pipeline, ensuring that all changes and collaboration are fully automated and trackable. By automating previously manual processes, the pipeline evolves into an automated knowledge base that outlines the steps required to validate working software.

Such a pipeline has a profound effect on the team’s culture. Changing the pipeline changes the culture. Whenever a new standard is introduced, the team must first ask, “Is it possible to build this into the pipeline to solidify our expectations of collaboration from others?” The development team can refer to the pipeline logs when a build fails. The platform and infrastructure teams can use data produced by the pipeline to verify whether the software has been successfully deployed. The security team can review the security scan and testing results from the pipeline to validate a release candidate, and so on.

Governance Metric: % of changes validated through the pipeline vs. not

Policy 2: Make comprehensive and correct API documentation available to users

Large enterprises often struggle with API sprawl, which refers to the uncontrolled proliferation of APIs. API sprawl usually occurs when there is a lack of proper documentation and centralized API governance. Simply put, the organization may not know where all its APIs are located or what they are used for. API sprawl hinders integration and the exploitation of new business opportunities and poses a security risk. This risk can stem from Shadow APIs, which are used without the knowledge of the API governance teams and exist outside the established governance process. Additionally, the security risk can result from Zombie APIs that are no longer actively maintained, supported, or monitored but are still functional and accessible to users.

To combat API sprawl, a comprehensive API discovery mechanism should be implemented. API discovery is about making it easy for anyone in the organization to find any API they need. When people can locate the available APIs within an organization and understand what they do, it helps promote API consistency and reuse and reduces duplication of effort.

API documentation documents (that is, API descriptions like OpenAPI) also have to be comprehensive and accurate. Nothing produces an annoying user experience, and reduces trust in API providers, like available but inaccurate API documentation. Therefore it is important that runtime platforms are able to validate that API documentation matches the runtime traffic.

Governance Standard 2.1: Edge technologies SHOULD detect undocumented APIs

One way to manage API sprawl is to have policies and controls in place to detect undocumented and unmanaged APIs. Detecting APIs can be done by monitoring traffic at the edge. This detection can be one of the first steps for organizations suffering from API sprawl. From there, the organization can collate the documentation and store it in the internal API catalog, which leads to the next standard.

Apart from detecting undocumented APIs, runtime API platforms can also validate the correctness of API documentation by comparing it to run time traffic. This runtime validation mitigates API drift and is a security measure that identifies unexpected and potentially malicious API behaiviour.

Governance Metric:

Number of unmanaged APIs detected by edge controls.
Number of API description document defects

Governance Standard 2.2: All live APIs MUST be documented in a central API catalog

As part of your API deployment pipeline, API documentation should be stored centrally in an internal API catalog or registry. Additionally, a selection of these APIs can be made available to partners or consumers through an external developer portal. The catalog should have a browseable UI for users, but it should also provide API registry features. That is, it should provide the API descriptions for programmatic access by machines.

Governance Metric: The number of APIs in the API catalog.

Governance Standard 2.3: The API management platform SHOULD support self-service onboarding

Along with making APIs discoverable, users should be able to request and obtain access to an API quickly without jumping through administrative hoops. Self-service onboarding of APIs by obtaining an API key through a developer portal is one of the runtime requirements for a great developer experience for API users.

Governance Metric: Number of APIs documented in a self-service developer portal.

Policy 3: Secure ingress and egress API traffic from malicious and unauthorized access

The rapid growth of APIs and API traffic has exponentially expanded the attack surface for malicious actors. In its State of API Security Report 2024, Salt Security states that 95% of respondents in the research experienced security problems in production APIs, with 23% experiencing a breach. 25% of respondents also stated that one of their biggest concerns with their company’s API program was that it didn’t adequately address runtime or production API security.

It is crucial to protect APIs and invest in API runtime protection to guard against threats resulting from authentication weaknesses, sensitive data exposure, and account takeover or misuse. Public APIs can especially face a host of attack vectors—from SQL and code injection to cross-site scripting (XSS) and other OWASP Top 10 API vulnerabilities. They can also face bot attacks ranging from data scraping and intellectual property theft to account takeover and distributed denial of service (DDoS).

While organizations need to secure the APIs they build, they also consume APIs from third parties. These third-party APIs include APIs of external LLMs, CRM systems, payment processing platforms, messaging APIs, and more. However, the unsafe consumption of APIs is an OWASP API Security Top 10 threat. It arises when an application, such as an API, consumes external, third-party APIs and blindly trusts the data they return without adequately validating or sanitizing the data it receives. This exposes the application to SQL injection, code injection attacks, or the exposure of sensitive information.

Here are a few standards you can implement to achieve this policy in practice.

Governance Standard 3.1: All governed APIs SHOULD be exposed through an API gateway configured with appropriate security controls

API gateways provide control points for runtime API governance, enabling governance teams to enforce common runtime standards such as authentication and authorization, granular API access, rate limiting and throttling, input validation, and monitoring. API gateways can also integrate with an identity provider (IdP) to provide user authentication and authorization.

Governance Metric:

The number of governed APIs not exposed through the gateway
The number of APIs with insufficient authentication

Governance Standard 3.2: All public APIs MUST be protected by a WAF

A web application firewall (WAF) actively scans and filters incoming HTTP traffic. It blocks malicious API requests in real-time, typically using a set of threat signatures—pre-defined rules or patterns that identify malicious activity. In addition to maintaining a set of threat signatures, some WAFs use machine learning and behavioral analysis for pre-emptive protection, minimizing the need to update signatures when a new threat is identified continuously. WAFs may integrate with a global network of sensors to provide real-time threat intelligence on emerging threat patterns. WAFs can also provide detailed audit logs and reports required to meet various industry compliance requirements, such as PCI DSS, HIPAA, and GDPR.

Governance Metric: The number of public APIs unprotected by a WAF.

Governance Standard 3.3: Third-party API traffic MUST be validated

While organizations may put a lot of thought into securing the APIs they provide externally (such as encrypting the traffic and routing it through a gateway), they may not pay as much attention to the security of the APIs they consume. That is, API requests to third parties. Developers tend to trust the data from third-party APIs they consume, especially if they are provided by well-known companies. As such, they may adopt weaker security standards for API consumption.

A way to mitigate the security risk from third-party API traffic is to ensure third-party API traffic is encrypted, validated, and sanitized. Organizations can proxy the traffic through an API gateway that acts as an egress API gateway (also called a consumption gateway or reverse gateway). In the egress gateway, teams can use strong API contracts from the third party to validate that the traffic complies with the types, formats, and constraints defined in the contract. The egress gateway can also be a control point for implementing robust error handling and preventing the exposure of sensitive data. This egress gateway pattern is illustrated in the figure below.

Governance Metric: The number of unvalidated third-party APIs.

Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes

Change impact analysis involves identifying the potential consequences of a change, the components affected, and the possible risks associated with the change. It helps developers identify areas that can be affected by a change so that changes do not introduce system stability, performance, and security issues.

A way to analyze the impact of an API configuration deployment is to perform a static code analysis of the deployment artifacts. This analysis can help build a representation of a change and visualize its impact and any dependencies that may also be affected. For example, a static change analysis check may show that a rate limit change may affect not just one API but more. It may also show that changing a rate limit for a user group affects multiple groups. Such insights help reduce CFR.

Another way to analyze the impact of an API change is by detecting breaking changes, which can disrupt API users. A breaking change is a non-backward compatible change that forces API consumers to change their code to ensure their API integrations keep working.

Let’s look at two standards an organization can use to implement this runtime governance policy.

Governance Standard 4.1: All API configuration changes must pass static checks

One advantage of GitOps is that API configuration is stored in version control. This allows the team to run a static analysis on API configuration in the CI/CD pipeline. The analysis ensures that configuration files are valid and prevents any API misconfiguration from arising. It also helps the dev team understand the impact and scope of their changes on different APIs and API consumer groups.

Governance Metric: The ratio of the organization’s APIs whose configuration is under version control and change impact analysis checks.

Governance Standard 4.2: All APIs must follow the recommended versioning scheme

API versioning helps providers evolve an API and add new versions without breaking existing integrations. Many different versioning schemes exist for REST APIs (such as using the URI path, query parameters, custom headers, content type, and more). Your runtime infrastructure must support the versioning mechanisms that best fit your requirements. As an API provider, defining and adopting a consistent versioning mechanism for your APIs gives users a clear migration path to a new version and helps you manage the lifecycle of the API. Be sure to run breaking change checks before deployments to ensure no breaking changes are introduced.

Governance Metric: The number of APIs that do not follow the versioning scheme.

Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery

API monitoring involves tracking key metrics of API performance, such as response times and error rates. API Observability goes beyond monitoring a known set of API performance metrics. Observability allows API publishers to troubleshoot novel problems and understand what is happening inside an application. Observability is based on the ability of applications and infrastructure to emit telemetry signals: metrics, logs, and traces (MLT), which API publishers can analyze. API publishers can run distributed traces using telemetry, observing requests as they flow through a distributed system. This tracing gives them visibility into a system’s health and enables debugging the behavior of the production system.

API observability should be based on open standards for instrumenting, generating, collecting, and exporting telemetry data. Open observability standards like OpenTelemetry guarantee compatibility with various backend API analytics platforms and prevent vendor lock-in. They provide a way to decouple your observability strategy from your gateway strategy. Open standards also provide teams with a common language for describing cross-application interactions and troubleshooting.

Governance Standard 5.1: Teams MUST provide dashboards and configure alerts to monitor API request rate, error rate, and request latency

Having runtime visibility into an API’s performance is essential not only for understanding its usage but also for incident mitigation. API publishers should capture API metrics on uptime, requests per minute, latency, and errors per minute. They should also set up monitoring dashboards and proactive alerts to make API performance information available for the teams that support them.

Governance Metric: The number of APIs not set up for monitoring.

Governance Standard 5.2: APIs SHOULD use OpenTelemetry to publish telemetry data

OpenTelemetry is a widely supported open-source observability framework that has become the de facto standard for cloud-native applications. OpenTelemetry is built on rules and conventions known as the OpenTelemetry Protocol (OTLP). This protocol dictates how components related to OpenTelemetry exchange telemetry data between a source system that generates the data and its destination. The OTLP data model consists of traces, which describe the flow of execution; metrics, which offer statistical information on application performance; and logs, which provide detailed information on the application's state.

Governance Metric: The ratio of APIs instrumented with OpenTelemetry.

Governance Standard 5.3: API deployments SHOULD emit telemetry data

Application performance telemetry can be correlated with API deployment events to quickly visualize and investigate any API performance problems resulting from new deployments. Deployment event correlation with API runtime metrics is a powerful incident mitigation technique. This event correlation lets teams see if a deployment introduces problems and roll back the change.

Governance Metric: The ratio of API deployment pipelines that publish telemetry data.

Let's now explore how these runtime API governance policies are implemented in practice with Traefik Hub in Part 2 of this blog post.

]]>

Stop SQLi and XSS Attacks Easily with Traefik's WAF Integration

Immánuel Fodor — Tue, 03 Sep 2024 18:25:03 GMT

In today’s digital landscape, web applications and APIs are integral to virtually every aspect of modern life. From the familiar world of online shopping and social media to the behind-the-scenes functionality of mobile apps, government websites, and even entertainment platforms, APIs serve as the backbone of online interactions.

However, this ubiquity also makes them prime targets for cyber threats, and securing web applications and the flow of data that keeps our world running is a critical concern for enterprises. Malicious actors constantly search for vulnerabilities to exploit, aiming to steal sensitive data, disrupt operations, or even inject malware into unsuspecting systems. The exponentially increasing threat landscape demands a robust defense strategy, and that's where Web Application Firewalls (WAFs) come into play.

The Role of WAFs in API Security

WAFs are the first line of defense for your web-based applications and their APIs. They stand guard, meticulously examining every incoming request, ensuring only authorized and legitimate data passes through. WAFs prevent attackers from submitting malicious or unwanted data to applications as well as from exfiltrating sensitive data. Here's how they safeguard your business from a variety of threats:

Known attack detection: WAFs recognize the signatures of common attack strategies employed by cybercriminals. These include attempts to exploit vulnerabilities like SQL injection (SQLi), Cross-Site Scripting (XSS), file inclusions, code injection, session hijacking, and even metadata or error leaks. By identifying these known threats, WAFs can effectively block them before they can cause you any damage.
Beyond the obvious: Malicious activity isn't always so straightforward. WAFs can also detect abnormal or malformed requests deviating from regular user behavior. These red flags could indicate a more sophisticated attack, allowing the WAF to take preventative measures.

This level of protection provided by WAFs is crucial. But it doesn't stop there. WAFs offer additional functionalities to enhance security further:

Virtual patching: Sometimes, vulnerabilities in your system might be identified before a permanent fix is available. WAFs can provide temporary protection by blocking exploits targeting those vulnerabilities, effectively plugging the hole until a proper patch can be deployed.
Blocking malicious bots: The internet is rampant with automated bots. While some bots serve legitimate purposes, others can be malicious, launching denial-of-service attacks or scraping valuable data. WAFs can help filter out these malicious bots, keeping your APIs safe from unwanted traffic.
Audit logging: Staying informed about potential security breaches is critical. WAFs provide valuable audit logs detailing matched rules and anomaly scores, allowing your security team to analyze potential threats and take appropriate action.

Beyond Technology: WAF as a Legal Necessity

In addition to protecting against cyber threats, WAFs are critical in helping enterprises meet various security standards and regulatory requirements. Many industries are governed by strict regulations that mandate specific security controls to protect sensitive data. Some of the most common standards that require or strongly recommend the use of WAFs include:

Payment Card Industry Data Security Standard v4: Under PCI DSS 4.0, organizations must deploy an automated technical solution for public-facing web applications that continually detects and prevents web-based attacks. This is articulated in Requirement 6.4.2, which explicitly states that WAFs are no longer just a recommended practice but a must for compliance.
Federal Risk and Authorization Management Program: FedRAMP requires stringent security controls for cloud service providers working with US federal agencies, including protections that WAFs can enhance.
Health Insurance Portability and Accountability Act: While HIPAA does not explicitly require WAFs in the healthcare sector, it mandates the protection of electronic protected health information (ePHI), which can be supported by using WAFs to prevent unauthorized access and attacks.
General Data Protection Regulation: Although GDPR requires appropriate technical and organizational measures to protect personal data, it does not explicitly mention WAFs. However, organizations should use them as part of their overall security posture to reduce the risk of data breaches that could lead to significant fines under GDPR.

To learn more about WAFs’ importance, check our detailed article: Why does WAF matter in API security?

However, keep reading if you’re curious about how Traefik can help you effortlessly mitigate these threats while complying with security standards.

A Free and Open-Source WAF in Traefik Proxy

The release of Traefik Proxy v3 marked a significant leap forward in enhancing application security through the introduction of two groundbreaking features: WASM-based plugins and the Coraza Web Application Firewall (WAF).

WASM (WebAssembly), a binary instruction format designed for efficient execution in web browsers, has found its way into the world of proxy servers. Traefik v3 embraces this technology, enabling developers to compile code in virtually any language into a portable format that runs seamlessly within the proxy environment. This innovation opened the door for a new breed of plugins, offering greater flexibility.

Capitalizing on this breakthrough, Traefik v3 introduced its first WASM-based plugin, the Coraza Web Application Firewall. Coraza is an open-source, Go-based WAF that has quickly become a favored choice for security-conscious enterprises. It stands out for its performance and endorsement by OWASP, a globally recognized authority on web application security.

Coraza is built on best practices and standards that are widely recognized in the cybersecurity community:

It understands and can enforce rules written in ModSecurity’s Security Language (seclang). This compatibility is a significant advantage for organizations that are already familiar with ModSecurity, as they can easily transition to using Coraza without needing to rewrite existing security rules.
Coraza comes equipped to enforce the OWASP Core Rule Set (CRS), a set of generic attack detection rules that protect against a wide range of threats. The CRS helps automate threat mitigation by blocking common attack vectors such as SQL injection, cross-site scripting (XSS), and other OWASP Top 10 vulnerabilities, saving valuable time and resources.

The WASM-based Coraza WAF in Traefik is best suited for:

One of the most compelling reasons to adopt a plugin-based WAF is to simply meet compliance requirements. A plugin-based solution like Traefik Proxy simplifies the process of implementing and managing security controls.
Furthermore, a plugin-based WAF allows for granular protection. You can selectively apply WAF rules to specific pages or applications, such as protecting sensitive administrative areas like /admin while leaving less critical pages unfiltered. This approach optimizes performance and reduces the risk of false positives.
A plugin-based WAF is particularly beneficial for organizations with internal applications that experience low traffic volumes but may still contain sensitive information. With Traefik Proxy, you can easily deploy and manage WAF protection for these applications without compromising performance or budget.
Finally, filling in the cybersecurity skills gap is a significant challenge for many organizations. Attracting and retaining skilled security personnel is expensive and time-consuming. A plugin-based WAF like Coraza in Traefik Proxy can help address this issue by providing user-friendly configuration and baked-in rule sets based on best practices.

A 2023 study from the Information Systems Security Association (ISSA) indicates that 71% of companies report being severely affected by the shortage of skilled cybersecurity professionals. A plugin-based WAF can help bridge this gap by simplifying the management of API security.

Let’s illustrate how simple configuring the Coraza WAF WASM plugin is! Add this to your Traefik’s static configuration to load the plugin:

experimental:
  plugins:
    coraza:
      moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
      version: v0.2.2

Then, leverage the WAF plugin in a middleware configuration. Using the Kubernetes CRD provider as an example:

---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: coraza-waf-block-admin-path
  namespace: apps
spec:
  plugin:
    coraza:
      directives:
       - SecRuleEngine On
       - SecDebugLog /dev/stdout
       - SecDebugLogLevel 9
       - SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"

Then accessing /admin will return a 403 error as specified by the configuration rule, but everything else will return 200 as there is no matching rule. You’ll see its power even more when combining it with IP address checking, header-based filtering, or chaining any other middleware or plugin available for Traefik Proxy.

The plugin-based WAF offers a versatile solution for a variety of use cases. Its ease of use but still powerful capabilities make it an essential tool for organizations looking to secure their web applications in a cost-effective and manageable way. But if needs evolve and you need to juice out as much filtering from your WAF as possible, you’ll need a bigger wall.

A Performance-Optimized WAF for High-Volume APIs in Traefik Hub

Security needs become more complex as companies evolve and scale their digital infrastructure. While traditional solutions like application proxies serve many essential functions, they often fall short regarding advanced security requirements like advanced authentication and authorization. This is particularly true for organizations that rely on WAFs to protect their web applications and APIs. For them, transitioning from a simple application proxy to a more robust API Gateway, like Traefik Hub, can offer significant advantages, including enhanced security and vastly improved performance.

Traefik Hub represents the next step in the evolution of serving APIs, offering a comprehensive platform that goes beyond the capabilities of an application proxy. As a modern API Gateway and API Management solution, Traefik Hub is designed to handle the complex needs of today’s enterprises, providing advanced features such as its native WAF integration, which directly addresses the performance limitations many organizations face when using WAF plugins in a proxy setup.

The Need for a High-Performance WAF

One of the key challenges with using a WASM-based WAF plugin in Traefik Proxy is the performance overhead associated with the plugin. While WASM offers a flexible and portable way to deploy WAFs, it introduces certain inefficiencies that can impact the overall performance of the application:

Boot-time overhead: Since WASM offers a secure runtime environment for plugins, the WASM binary associated with the WAF needs to be loaded at boot time for all middleware configurations. This loading process can delay application startup, especially in environments with complex configurations or high traffic demands.
Run-time overhead: Once loaded, the WASM-based WAF analyzes each incoming request and outgoing response in real time. This continuous analysis introduces latency, which can slow down the request processing and reduce the application's overall efficiency. The performance overhead can become a significant bottleneck with high traffic volumes, limiting scalability and increasing operational costs.

To address these performance challenges, Traefik Hub offers a native WAF integration that eliminates the inefficiencies associated with WASM-based plugins. The native integration in Traefik Hub results in a WAF that is 23 times faster than its WASM-based counterpart in Traefik Proxy, providing a significant performance boost for your applications. You’ve read it correctly, not 23%, but 23 times!

The key to this performance improvement lies in the architecture of the native WAF integration in Traefik Hub. Unlike WASM-based plugins, the Coraza WAF codebase is natively integrated into Traefik Hub at compile time. Since no WASM runtime is involved in running the WAF, it eliminates the boot-time and run-time overhead associated with the plugin-based solution, resulting in faster startup times and more efficient request processing.

Benefits of a Native WAF Integration

The native integration of the Coraza WAF in Traefik Hub offers several significant benefits:

Further enhanced security: The native WAF integration provides all the robust security features expected from Coraza, including protection against OWASP Top 10 vulnerabilities—but now with additional benefits. There’s no need to download the plugin during boot time, reducing the dependency on external components. This is perfect for air-gapped environments where external network access is restricted and enhances overall security by reducing the attack/risk surface.
Simplified management and configuration: With native integration, managing security configurations becomes more straightforward, as there’s no need for static plugin configuration. Instead, security rules and configurations are handled only within the dynamic configuration.
Improved performance and scalability: The native WAF integration allows for faster request processing, leading to better scalability. The same number of API Gateway replicas can handle more requests, improving resource utilization. This means you can run leaner infrastructure, serving the same or larger traffic with fewer resources, leading to lower operational costs.

Now, let’s see some proof of the claimed 23x performance improvement!

Performance Benchmark: WASM-Based vs. Native WAF

We conducted a performance benchmark to compare the efficiency of the native WAF integration in Traefik Hub against the WASM-based WAF plugin used in Traefik Proxy.

The benchmark results highlight the performance advantages of the native WAF in Traefik Hub:

WASM-Based WAF: In our test environment, the WASM-based WAF managed to handle 29.04 requests per second.
Native WAF: Under the same conditions, the native WAF processed a remarkable 683.34 requests per second.

This represents a 23.53x performance improvement when using the Coraza WAF natively in Traefik Hub!

Comparison between WAF integrations based on requests per second.

These metrics were obtained in a controlled environment using a local k3s cluster with just one gateway replica. While this setup provides a clear comparison of the two WAF approaches under identical conditions, it’s important to recognize that this is just one snapshot of performance as it is based on a simplified scenario and doesn't fully capture the capabilities of either the WAF or the API Gateway in a complex environment. The actual observed performance could vary depending on several factors:

Scaling: In production, the API Gateway and WAF would probably be deployed with multiple replicas to handle higher traffic volumes in production. The native WAF's performance benefits would likely become even more pronounced in such scaled environments.
Environment complexity: The nature of the applications, the complexity of the enforced security rules, and the overall architecture will all influence the WAF's performance.
Resource allocation: Differences in CPU, memory, and other resource allocations between environments can also impact the measured throughput.

While the native WAF integration offers superior performance, it’s also essential to consider the trade-offs involved in running a WAF, even a high-performance one. Adding a WAF to your traffic processing pipeline introduces some overhead due to the need to inspect and potentially filter every request and response.

Our tests showed that disabling the WAF altogether could result in at least 4 times higher request throughput in Traefik Hub. This is an important consideration for organizations that must balance security with performance. For some applications or routes with lower security requirements, disabling the WAF might be acceptable to achieve higher performance.

However, for critical applications or sensitive data, the performance overhead introduced by a WAF is typically a necessary trade-off to ensure robust security. The key is to use the flexibility offered by Traefik Hub to apply the WAF selectively, protecting the most vulnerable parts of your infrastructure while maximizing performance where possible.

Migration from WASM-Based WAF to Native WAF

If you’re currently using the WASM-based WAF in Traefik Proxy, migrating to the Native WAF in Traefik Hub is straightforward. Just like other transitions within the Traefik ecosystem, this migration is designed to be seamless, allowing you to enhance your security setup without disrupting your existing configurations or workflows.

Traefik Hub has been built with user experience in mind, ensuring that the migration is as painless as possible:

Remove static configuration: One key difference in the migration process is removing the static configuration required by the WASM-based WAF (the first block in the code example shown previously). The Native WAF in Traefik Hub does not require any static configuration, making the setup simpler and less prone to errors.
Retain existing middleware configuration: The rest of your WAF middleware configuration will remain the same, so you won’t need to make any changes to your existing setup!

The migration process is quick and efficient, allowing you to immediately take advantage of the highly improved performance after upgrading from Traefik Proxy to Traefik Hub.

For more information on the WAF configuration options, check out the official Traefik Hub documentation.

Wrapping It Up

If you’re looking for tools to enhance your web application security within the Traefik ecosystem, the availability of Coraza WAF as a WASM plugin represents a significant step forward. It’s free and open-source, and you can start experimenting with it today within Traefik Proxy with optional commercial support.

When you feel you want more, Traefik Hub is equipped to handle the most demanding security requirements, providing both protection and performance at scale with its native WAF integration. You can easily migrate from plugin-based to native WAF through a smooth transition with further simplified configuration, keeping all your security defenses in place while optimizing your resource utilization with an outstanding 23x performance improvement. Visit our WAF solution page to learn more.

Ready to see how Traefik Hub can elevate your security strategy? Request a demo today and experience the benefits of a modern, high-performance API Gateway and Management solution.

]]>

From Ingress to API Management: How Traefik OSS Grows with You

Immánuel Fodor — Fri, 23 Aug 2024 22:00:10 GMT

In the ever-evolving landscape of cloud-native technologies, managing and securing microservices and APIs effectively is crucial for businesses of all sizes. But as your API ecosystem grows and matures, so do your needs.

This blog will guide you through the seamless migration journey across Traefik Labs' comprehensive product portfolio, highlighting the effortless upgrade process that bridges it together. Starting from the foundational Traefik Proxy, you can now elevate your API operations to new heights with the Traefik Hub API Gateway and API Management tools.

And while we’re going to get deep in the weeds, I hope you’ll see the beauty in its simplicity—upgrading Traefik is essentially a replacement of the running binary, making it a non-disruptive process.

Start With Us and Grow With Us

Traefik Labs helps DevOps and Platform engineers throughout their cloud-native operations journeys, from a simple reverse proxy implementation to more advanced API lifecycle management capabilities.

Key API Gateway & API Management capabilities that can be added to Traefik OSS.

Traefik Hub API Gateway and API Management are built on the open-source Traefik Proxy. They include all the capabilities and characteristics of Proxy and extend it with specialized features targeting APIs. Each level-up takes just a few seconds and is an additive extension, nothing is taken away. Existing configurations and the UX remain the same. Everything you appreciate about Traefik Proxy today is carried through into each upgrade stage.

Since Proxy can run in many different environments, our API Gateway and Management can also publish and secure APIs from every Kubernetes flavor, big cloud providers, Hashicorp Nomad, Docker Swarm, and many more. And all these solutions are based on declarative configuration and GitOps. Leverage labels, middlewares, plugins, and CRDs declaratively, no matter which product you run.

Let’s explore the three journey steps in detail with short hands-on labs, starting with the origins and significance of Traefik Proxy.

Traefik Proxy: Born in The Trenches

The Traefik story began in 2015, addressing a real-world production challenge encountered by Emile Vauge, the founder of Traefik Labs. Deploying thousands of microservices demanded a solution that transcended the limitations of existing reverse proxies at that time. Traditional solutions lacked the automation, service discovery, and scalability required for their environment. This led to the birth of Traefik Proxy, a solution designed from the ground up for the cloud-native world.

Traefik Proxy's focus on ease of use and powerful features quickly propelled it to global recognition. With over 3.3 billion downloads and a place among the top 15 most downloaded images on Docker Hub, Traefik has become a cornerstone of cloud-native infrastructure. This widespread adoption is a testament to its core strengths:

Simplicity: Declarative configuration and GitOps integration streamline management and ensure infrastructure-as-code best practices.
Automation: Automatic service discovery and certificate management eliminates manual configuration burdens by automatically detecting and configuring routes to newly deployed services, reducing administrative overhead.
Scalability: Traefik scales effortlessly alongside your infrastructure, handling even the most demanding workloads.

Its versatility allows deployment on various cloud environments, including all Kubernetes distributions and other container orchestrators like Hashicorp Nomad or Docker Swarm, as well as bare metal and virtual machines, making it a comprehensive solution for all possible scenarios. Traefik serves as the gatekeeper of infrastructure, ensuring security, observability, and efficiency while scaling up to meet evolving needs.

Unsurprisingly, Traefik Proxy is a trusted solution for organizations of all sizes. Financial institutions, major retailers, and countless service providers leverage Traefik to manage mission-critical production workloads. It’s an ideal choice for:

Users prioritizing best-of-breed integrations: Traefik Proxy integrates seamlessly with a vast ecosystem of tools and technologies, allowing you to leverage your existing infrastructure.
Environments requiring advanced network routing: Traefik Proxy functions as a versatile application proxy, handling tasks like ingress control, reverse proxying, and load balancing. It also supports content caching and circuit breaking to enhance performance and resilience.
Organizations seeking a free and open-source solution: Traefik Proxy is a fully open-source project allowing complete transparency and customization. However, you can also get commercial support with 24/7 access to the Traefik Labs team.

Now that it’s clear what Traefik Proxy is and what it can do for you, let’s get our hands dirty and explore the journey of an API in the Traefik ecosystem.

Journey Lab 1: Deploy a Weather Service with Traefik Proxy

In this blog post, apart from covering the obvious upgrade steps from Proxy to API Gateway then to API Management, we’ll follow the life of a simple Weather API for illustration purposes.

The first step with the open-source Traefik Proxy is where everybody gets familiar with the Traefik ecosystem. Let’s deploy a Proxy to simulate the Kubernetes-based infrastructure of our imaginary weather forecasting service provider company.

helm repo add --force-update traefik https://traefik.github.io/charts
helm install traefik -n traefik --create-namespace --wait \
  --set ingressClass.enabled=false \
  --set ingressRoute.dashboard.enabled=true \
  --set ingressRoute.dashboard.matchRule='Host(`dashboard.docker.localhost`)' \
  --set ingressRoute.dashboard.entryPoints={web} \
  --set ports.web.nodePort=30000 \
  --set ports.websecure.nodePort=30001 \
   traefik/traefik

💡

Did you know that Traefik comes pre-packaged as the default Ingress Controller for Rancher's popular Kubernetes distribution? So, if you want to replay the commands in k3d, you first need to create a cluster with the default Traefik Ingress Controller turned off:

k3d cluster create traefik-hub --port 80:80@loadbalancer \
--port 443:443@loadbalancer --port 8000:8000@loadbalancer \
--k3s-arg "--disable=traefik@server:0"

Once it's installed, we can access the local Proxy dashboard: http://dashboard.docker.localhost/

Traefik Proxy Dashboard

Now, it’s time to deploy the backend of the Weather API using a simple JSON server written in Go and expose it with an IngressRoute object:

kubectl create namespace apps
kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/weather-app.yaml
kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-no-auth.yaml

The Traefik-native IngressRoute is chosen here for simplicity. Still, Traefik Proxy can also expose applications using the old-standard Kubernetes Ingress or Kubernetes’ new Gateway API HTTPRoute objects. In fact, we have first-class production-ready Kubernetes Gateway API support starting from Proxy v3.1!

The Weather API can be accessed now; let’s try it using curl:

$ curl http://walkthrough.docker.localhost/no-auth
{
  "public": [
    { "id": 1, "city": "GopherCity", "weather": "Moderate rain" },
    { "id": 2, "city": "City of Gophers", "weather": "Sunny" },
    { "id": 3, "city": "GopherRocks", "weather": "Cloudy" }
  ]
}

Let’s not keep the API unsecured; protect it quickly with Basic Authentication.

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-basic-auth.yaml
# To generate the password, we used `htpasswd -nb foo bar | openssl base64`
# The endpoint path has also moved to /basic-auth

Basic Authentication was widely used in the early days of the web. However, it also has a security risk: credentials can be visible to any observer when using HTTP. Additionally, it uses hard-coded credentials, potentially giving more authorization than required for a specific use case.

Nowadays, those issues are addressed when using JSON Web Tokens (JWT). A JWT can be cryptographically verified; it detaches authentication from user credentials and has an issue and expiration date.

Even in this simple use case, this is how we reach the need for a sophisticated API gateway that has enterprise-level authentication and authorization, among many other new capabilities, on top of Traefik Proxy.

Traefik Hub API Gateway: Level Up Your API Security

While the exact percentage may vary depending on the data source and methodology, recent reports consistently show that API traffic makes up most of the web traffic. Imperva found that 71% of internet traffic in 2023 was API calls. The API Security and Management Report by Cloudflare indicates that APIs accounted for over 57% of dynamic internet traffic processed by their platform in the past year. And according to Akamai, 83% of web traffic consisted of API calls.

The growth in API usage is driven by factors like digital transformation, cloud migration, and the rise of the microservice architecture. As APIs proliferate within organizations, the need for an API Gateway becomes increasingly critical, particularly as centralizing common functionalities to streamline operations and enhance security becomes a priority.

Traefik Hub API Gateway offers a natural evolution from Traefik Proxy, addressing this need by offering a range of enterprise-grade features specifically targeting the use case of exposing and securing APIs. Building upon the core strengths of Proxy, our API gateway equips you with a comprehensive suite of features designed to safeguard your APIs, including centralized authentication and authorization, distributed security features, and various vendor integrations.

Centralized security and control: Imagine managing authentication and authorization across your entire API portfolio from a single location. Traefik Hub API Gateway empowers you to do just that. It supports a wide range of industry-standard protocols, like OIDC and OAuth token introspection, allowing you to define and enforce access policies for all your APIs centrally. This eliminates the need for repetitive implementation and configuration on the microservice level and ensures consistent security practices across your deployments.
Distributed security features: Traefik Hub API Gateway takes security a step further by offering features like distributed rate limiting and distributed TLS certificates. Rate limiting ensures that no single API backend becomes overwhelmed by excessive traffic spikes. Rate-limiting quotas are distributed across your Gateway instances, providing a robust defense against denial-of-service attacks. Additionally, API Gateway integrates seamlessly with security tools like Let's Encrypt, allowing you to provision and manage TLS certificates for your APIs effortlessly, even in a distributed environment.
Enhanced vendor integrations: The modern API landscape often involves a complex ecosystem of tools and services. Traefik Hub API Gateway recognizes this by integrating with leading security vendors like HashiCorp Vault and Azure Key Vault. This enables you to centralize your secrets management and leverage existing security infrastructure within your organization.
Native Web Application Firewall (WAF): In our API Gateway, the recently announced Coraza WAF integration is implemented natively as part of the Gateway’s code base. The native integration provides more than 23 times (Yes, you read that right!) performance improvement over the plugin-based WAF available in Proxy.
Open-source with enterprise benefits: Traefik Hub API Gateway retains the open-source core of Traefik Proxy, allowing you to benefit from continuous innovation from the whole open community. However, API Gateway also unlocks additional enterprise-grade features on top of Proxy without requiring excessive migration.

Upgrading from Traefik Proxy to API Gateway is a smooth and straightforward process. The seamless upgrade path protects your investment in existing configurations and ensures minimal disruption to your ongoing operations. All your current configurations can stay in place, and the API Gateway continues to honor them without any change. While building upon the foundation you've established with Traefik Proxy, Traefik Hub API Gateway empowers you to manage access centrally, enforce security policies, and leverage industry-standard best practices.

Journey Lab 2: Upgrade Traefik Proxy to Traefik Hub API Gateway

Let’s continue the upgrade journey in this next lab, leveling up the Weather API game from Proxy to API Gateway capabilities. The upgrade is as easy as setting up a license key and executing a simple Helm chart upgrade. Fill out this form to get a trial or production license key.

Now, open a terminal and save the license as a Kubernetes secret:

kubectl create secret generic license \
  --namespace traefik \
  --from-literal=token=$TRAEFIK_HUB_TOKEN

Then, upgrade Traefik Proxy to Traefik Hub using the same Helm chart we used previously since Traefik Hub API Gateway is 100% compatible with Traefik Proxy v3 and up. All settings are retained; the only command you need to execute just replaces the running binary with a new one:

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set hub.token=license \
  --set image.registry=ghcr.io \
  --set image.repository=traefik/traefik-hub \
  --set image.tag=v3.2.0 \
   traefik/traefik

The upgrade finishes in seconds, and the dashboard is still reachable (http://dashboard.docker.localhost/), but it now shows the Traefik Hub API Gateway logo in the top left corner.

Traefik Hub API Gateway Dashboard

You can also confirm that Basic Auth is still here if you execute the same curl commands against the Weather API. All other configurations are also kept and served.

Let's secure the weather API with something more professional, such as an API Key, a new middleware to which the API Gateway gives access (among many other things).

First, we generate the hash of our API key: “Let's use API Key with Traefik Hub”. Using the key hash in the configuration ensures that the key itself is not leaked, even if it is accidentally pushed to a public Git repository.

htpasswd -nbs "" "Let's use API Key with Traefik Hub" | cut -c 2-

We can now put it into the API Key middleware and apply it:

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-apikey.yaml
# The API path has also been changed to reflect the use case better, now it’s /api-key

Test it with the following commands:

# This call is not authorized => 401
curl -I http://walkthrough.docker.localhost/api-key
# Let's set API Key
export API_KEY=$(echo -n "Let's use API Key with Traefik Hub" | base64)
# This call with the token is allowed => 200
curl -I -H "Authorization: Bearer $API_KEY" \
  http://walkthrough.docker.localhost/api-key

The API is now secured using an API key. The API gateway also allows you to handle users with an identity provider using OIDC or OAuth, apply distributed rate-limiting, and implement other enterprise-grade configurations.

But what if we want to cover internal and external use cases and protect the Weather API on the HTTP verb level to apply further governance? Or test a new API version with part of the production traffic? And what about publishing all the managed APIs to a developer portal for easy discovery and reduced time to consumption?

You’ll need Traefik Hub with API Management features enabled to do this.

Traefik Hub API Management: Granular Control and Scalability

As your API ecosystem scales beyond security needs, managing APIs effectively becomes crucial. Building upon the robust features of Traefik Hub API Gateway, Traefik Hub API Management offers a powerful solution for comprehensive API lifecycle management, enabling you to govern your APIs, streamline development workflows, and cater to diverse API stakeholder needs.

API governance for complete control: Traefik Hub API Management provides a centralized platform for discovering, registering, and managing your entire API portfolio. Features like versioning and Kubernetes-native API labels and selectors allow you to organize your APIs logically and track their evolution over time. Additionally, API Management integrates with linters, which can automatically detect and flag potential errors within your API configurations before they impact production.
Advanced API security: API Management takes API security to the next level by offering granular access control mechanisms. You can define fine-grained permissions for individual users or groups, ensuring only authorized users can access specific API resources and functionalities. Furthermore, API Management integrates with industry-standard authentication protocols, allowing you to leverage existing identity and access management (IAM) solutions within your organization.
API monitoring and observability: Deep insights into API health and performance are crucial for maintaining a reliable and scalable API landscape. Traefik Hub API Management integrates seamlessly with OpenTelemetry, providing comprehensive metrics and tracing capabilities. This allows you to gain real-time visibility into API traffic patterns, identify potential bottlenecks, proactively address any performance issues, and troubleshoot effectively.
API Developer Portal: Create one or more dedicated API Developer Portals easily! The portal is a central hub for developers to discover, understand, and interact with your APIs. The portal can house API documentation, interactive consoles for testing purposes, and clear instructions on integrating with your APIs. You can also fully customize it to your liking. By providing a user-friendly developer experience, API Management fosters adoption and streamlines the development process for your APIs' internal and external consumers.

As with the previous steps, upgrading from Traefik Proxy or API Gateway to API Management is a similarly simple process. It involves a license upgrade and a Helm chart update, minimizing disruption to ongoing operations. API Management inherits all the functionalities of both Proxy and Gateway, ensuring a smooth transition and continued access to their features. API Management also treats configuration through declarative CRDs as a first-class citizen, but also offers a user-friendly UI for managing API configurations and their current state.

Journey Lab 3: Upgrade Traefik Hub API Gateway to API Management

In our last lab, we assume that the license of the weather provider company has been updated with enabling API Management capabilities. After that, we enable the API Management feature on the Traefik Hub deployment using the same Helm chart:

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set hub.apimanagement.enabled=true \
  traefik/traefik

The upgrade was as easy as executing this one command. The local dashboard is still reachable on http://dashboard.docker.localhost/ and we can also confirm that the API is still secured using an API Key.

Now, let's promote the Weather API Ingress to be a managed API with Traefik Hub API Management:

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/api.yaml

What did we just do here?

The applied manifest has created a managed API CRD describing the properties of the Weather API, like the location of its OpenAPI Specification.
It also created an API Access CRD describing who can access the Weather API.
It removed the API Key authentication middleware, as we'll use Traefik Hub's built-in identity provider for user and credential management. The API is still secured, as we'll see it shortly.
It added a simple one-line annotation on the existing IngressRoute CRD with a link to the API CRD created in the 1st point, effectively promoting the ingress to be now a Traefik Hub-managed API.

hub.traefik.io/api: walkthrough-weather-api # <= Link to the API using its name

We can confirm this API is still not publicly exposed without authentication as it returns the expected 401 Unauthorized HTTP code. To be able to consume the API, let’s create a user for this API in the Traefik Hub API Management online dashboard.

The user created previously will connect to an API Portal to generate an API key, so let's deploy our API Portal!

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/api-portal.yaml

After a couple of seconds, the API Portal should be reachable on http://api.walkthrough.docker.localhost. Log in with the admin user and create an API Key. The Weather API is now consumable with the new token, and it’s also visible on the portal:

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  http://api.walkthrough.docker.localhost/weather

Traefik Hub API Portal

The Weather API has documentation built from the OpenAPI specification, and the API can also be interactively tested right on the Portal with the try-out functionality.

And that's it! You’ve now seen the very short and quick upgrade steps of each tool and also got a glimpse of using them.

Wrapping It Up

The seamless migration journey across Traefik Labs' products ensures that everything built remains intact while adding new capabilities as needed. Investing time and resources into Traefik Proxy, Traefik Hub API Gateway, or API Management is a smart choice for businesses looking to future-proof their infrastructure. There is no rip-and-replace—just an in-place upgrade!

Consider Traefik’s cloud-native, fully declarative, and GitOps-driven approach to ensure efficient and effective API operations in your organization. The functionally additive upgrade path, robust feature set, and ease of management make it the best choice for handling the complexities of modern cloud-native environments. Scale your operations confidently with Traefik as your business grows and evolves.

]]>

How the Latest API Gateway Innovations Are Reshaping Operational Efficiency and Error Metrics

Immánuel Fodor — Thu, 15 Aug 2024 23:04:24 GMT

Downtime is not just an inconvenience—it can be a critical threat to business operations. The cost of downtime extends far beyond immediate financial losses; it can damage your company's reputation, erode customer trust, and, in some industries, such as healthcare and chemical processing, can lead to dangerous situations.

Modern, declarative API gateways are pivotal in managing the complexities of microservices architectures. They provide GitOps capabilities that reduce human error, allow for immediate rollback to a stable state, and make things repeatable and scalable even for a small staff. They can also provide advanced observability and monitoring capabilities, automating key tasks and offering real-time insights that let DevOps teams quickly identify and resolve issues.

And as you already know, issues are going to happen. The question is how much they’ll cost you.

The Cost of Downtime

Downtime can affect everything from financial performance to customer trust, so understanding and minimizing downtime is crucial for maintaining operational efficiency and ensuring continuous service availability. To get a tangible feel for downtime costs, we can look at the Mean Time to Recovery (MTTR) and the associated cost per minute of a system that’s not operating properly. In some situations, even brief downtimes are unacceptable, showing the business benefits of implementing robust observability and monitoring tools combined with GitOps to mitigate these risks.

Mean Time to Recovery (MTTR)

MTTR is a critical metric in DevOps, representing the average time it takes to restore a system to operational status after a failure. A lower MTTR indicates a more resilient and efficient system, which is essential for minimizing downtime. Quick failure recovery can make a significant difference in environments where uptime is paramount, such as healthcare or financial services. Rapid issue resolution reduces the impact on users and helps maintain trust in the system's reliability.

Cost per Minute of Downtime

The financial impact of downtime can be staggering. According to a 2016 study, the average cost of IT downtime was as much as $9,000 per minute, which translates to $540,000 per hour—and that was almost 10 years ago. (You can get a visceral sense of how quickly this adds up here.) Even for smaller businesses, where the cost per minute can be in the hundreds of dollars rather than thousands, it’s still significant.

The costs can be even higher in industries with critical dependencies on IT systems, such as healthcare, finance, and telecommunications. For example, when a bug turned the automatic update of CrowdStrike’s Falcon antivirus software into a catastrophic event, many hospitals found themselves unable to access diagnostic systems or even patient registration. All in all, it cost healthcare providers an estimated $1.94 billion, with an average of 64.6 million for each provider.

Even more important, however, is that when a hospital’s systems go down, critical care might be delayed, putting lives at risk. Therefore, minimizing downtime may not be just a technical or financial challenge, and instead may be far more high-stakes.

While a modern API gateway couldn't have helped companies avoid the CrowdStrike outage, they can be pinnacle in preventing other outages.

The API Gateway Innovations Reshaping Incident Prevention and Mitigation

The combination of GitOps and comprehensive observability is revolutionizing the way we manage and monitor API gateways.

GitOps leverages version control and automation to bolster the reliability, security, and scalability of these critical components, ensuring seamless, error-free operations. By defining infrastructure and API configurations declaratively and managing them in version-controlled repositories, GitOps enables rapid, consistent deployments while enhancing audit trails and compliance tracking.

Meanwhile, comprehensive observability extends monitoring beyond traditional methods, providing real-time insights through logs, metrics, and traces. This holistic approach enables teams to proactively identify and address potential issues, minimizing downtime and maintaining high system performance. Together, GitOps and observability form a powerful duo, empowering organizations to optimize their API gateway operations in today's demanding digital landscape.

The GitOps Revolution

GitOps is crucial for modern API gateways because it leverages the principles of version control and automation to enhance reliability, security, and scalability. Because API gateways act as the gatekeepers for all incoming and outgoing traffic in microservices architectures, the need for robust, error-free, and agile operations is paramount. GitOps addresses these needs by providing a framework where infrastructure and API configurations are defined declaratively and managed in version-controlled repositories.

This approach not only facilitates rapid, consistent deployments but also significantly improves audit trails and compliance tracking. By automating deployment processes, GitOps reduces human error and increases operational efficiency, making it an essential strategy for organizations looking to optimize team performance in a cloud-native environment.

But as we all know, errors will happen. With a GitOps strategy, code can be instantly rolled back to a stable state, creating the time necessary to diagnose and remedy issues without impacting stakeholders. For this reason, it’s important to choose an API gateway that supports GitOps end-to-end.

The Advent of OpenTelemetry and Comprehensive Observability

When it comes to mitigating the effects of downtime, in addition to GitOps, your best friend is “comprehensive observability,” or the ability to fully monitor and understand the state and behavior of a system across its entire stack by collecting, analyzing, and visualizing data from its various components. This concept extends beyond traditional monitoring to include three key types of telemetry data:

Logs: timestamped records of events
Metrics: quantitative data that measure the performance and health of the system
Traces: which track the journey of requests as they flow through the various components of the system

By implementing comprehensive observability solutions, your organization can gain real-time insights into your systems' health and performance. This proactive approach enables teams to identify and address potential issues before they escalate into significant problems, minimizing downtime and associated costs and, using GitOps, to roll back if necessary.

Observability tools that can monitor your API gateway offer detailed metrics, logs, and tracing capabilities that help DevOps teams maintain high system performance. These tools allow for monitoring key performance indicators (KPIs) such as request latency, error rates, and system throughput. By closely watching these metrics, teams can quickly detect anomalies and take corrective actions, ensuring systems remain operational and performant.

One thing that makes this kind of comprehensive observability possible is the advent of OpenTelemetry, an open-source standard for observability. Any tool that reads and/or writes it can share information with any other tool that understands it. By choosing an API Gateway that supports OpenTelemetry, you’re providing yourself with much greater possibilities.

As you might imagine, the observability strategies you have available to you are going to depend on the richness of logs, metrics, and traces your API gateway supports.

How to Reduce the Impact of Downtime

While businesses can implement several measures to minimize downtime, advanced observability and monitoring strategies are essential for proactively managing system health. This includes a combination of general best practices, GitOps, and specific observability-focused strategies to help you reduce downtime. DevOps teams can ensure higher system availability and performance by leveraging tools, such as their API gateway, for real-time monitoring, detailed metrics, and distributed tracing.

Generic Measures for Avoiding Downtime

While specific strategies vary, there are several generic measures are fundamental to the cause:

Regular System Updates and Maintenance: Regular maintenance schedules help ensure that all components function optimally and that known vulnerabilities are addressed promptly.
Robust Disaster Recovery Plans: This plan should include regular backups, clear recovery procedures, and periodic testing to ensure that systems can be quickly restored in case of a failure.
Redundant Systems and Failovers: Implementing redundancy at various levels—such as servers, storage, and network connections—ensures that a failure in one component does not bring down the entire system. Failover mechanisms should be in place to automatically switch to backup systems if a primary system fails.

Now let’s explore the more innovative measures.

GitOps-Focused Strategies

GitOps, by integrating core practices of DevOps with Git-based workflows, offers several strategic approaches to enhancing reliability, preventing downtime, and accelerating recovery in system operations. They include:

Automated Rollbacks and Progressive Delivery: It takes a bit of extra setup, but by using automated rollbacks, teams can rapidly address issues without manual intervention, drastically reducing potential downtime. Furthermore, progressive delivery techniques such as canary releases or blue-green deployments can be managed through GitOps workflows, allowing for a small subset of users to be exposed to new changes, limiting the impact of potential failures and ensuring that any critical issues can be identified and rectified before they affect the entire system.
Declarative Configuration Management: With GitOps, all system configurations are defined declaratively and stored in version control systems. This approach ensures that the configurations are reproducible and consistent across all environments, which eliminates common causes of downtime such as configuration drift or manual errors in setup. If a new configuration update causes a problem, teams can quickly revert to a previous configuration, ensuring continuous system availability.
Continuous Validation and Testing: Continuous integration tools integrated within GitOps workflows can automatically trigger tests and validation checks every time a change is made to the repository. This means that code and configurations are continuously tested, and only changes that pass predefined checks are deployed to production environments. This reduces the risk of introducing errors that can lead to downtime, ensuring that only stable, thoroughly tested updates are released.
Collaboration and Quick Response: GitOps fosters a culture of collaboration by ensuring that all changes are visible and traceable through the Git platform. Transparency helps teams to quickly identify and respond to issues, minimizing response times and reducing downtime. Additionally, having a centralized platform for all operations simplifies communication among team members, which is critical during incident response.

Observability-Focused Strategies

Observability is key to minimizing downtime. It involves monitoring systems comprehensively to proactively detect and address issues:

Real-time Monitoring: Using an API gateway with a monitoring dashboard, such as Grafana, for continuous health checks and performance monitoring ensures that any deviations from normal operation are immediately detected.
Detailed Metrics and Logging: For example, monitoring CPU usage, memory consumption, and network latency helps identify performance bottlenecks. Logs offer a detailed record of system activities, which is useful for diagnosing and troubleshooting issues.
Distributed Tracing: Some API gateways (e.g., Traefik’s) offer distributed tracing capabilities for tracking request flowing across various services. Tracing provides a holistic view of how different components interact, making pinpointing and addressing issues easier.
Proactive Issue Resolution: For example, if an observability tool indicates an increase in error rates, the team can investigate and address the underlying cause before it affects users.

Capabilities of an Effective Observability Solution

An effective observability solution should provide the following capabilities:

API Gateway Status Monitoring: Most API gateways provide status monitoring and alerts, enabling teams to detect and address issues with the gateway itself.
Tracing and OpenTelemetry: Tracing helps in understanding the flow of requests through the system. For example, Traefik supports OpenTelemetry, an open-source format for observability, enabling integration with any monitoring tool that supports it. This standardization allows for real-time troubleshooting and comprehensive system analysis.
Distributed Tracing: Distributed tracing is essential for understanding interactions across different services. Traefik, for example, helps tracing tools make correlations between various sources, such as the gateway, database, and microservices, providing a detailed view of the request lifecycle, and helping teams understand performance issues and optimize the system. The end result is greater transparency rather than the proverbial “black box.”

It’s important to realize that the level of observability in your API Gateway is more than just a formality. Not all gateways are created equal, and your ability to avoid and quickly bounce back from downtime will be directly related to your observability capabilities.

Conclusion

In today’s fast-paced and highly competitive digital landscape, minimizing downtime and enhancing system performance are critical to business success. Through the use of GitOps and integration with monitoring and observability capabilities, modern API Gateways provide robust solutions to these challenges. By reducing MTTR and its associated costs and leveraging real-time insights, organizations can swiftly address issues, ensuring continuous service availability and optimal performance.

Additional Resources

Traefik’s Blog on Observability and Tracing: Visit Traefik’s blog to learn more about how its observability features can enhance your API gateway management.
DORA Reports and Metrics: Understand the key metrics for evaluating DevOps performance and how Traefik can help improve them. Access the 2023 DORA Report.
Google Cloud’s ROI Whitepaper: Discover the financial benefits of investing in robust observability and monitoring tools in the Google Cloud ROI Whitepaper.

]]>

Strategic API Gateway Migration: A Comprehensive Blueprint

Immánuel Fodor — Fri, 02 Aug 2024 21:19:31 GMT

You’ve had it.

As the head of your company’s DevOps process, you’ve been frustrated with the web of APIs your company has created for some time. There’s no centralized governance; there’s no service discovery so developers are constantly searching for the right APIs to use; load balancing is a nightmare … but last night’s security breach tears it. Those credentials should have been deactivated months ago.

It’s time for a real, honest-to-goodness API gateway.

Or maybe you already have an API gateway, but it’s inconsistent across different environments and it hurts team efficiency by requiring constant situational manual interventions. Either way, you’re going to need to move your workloads to a new system that solves these problems.

Of course, that makes you cringe, too. The migration will be a nightmare.

But it doesn’t have to be.

By doing a strategic migration of your systems and moving a few APIs at a time, you can minimize risk and increase your agility during the process. Yes, it will take a bit longer, and there will be more moving parts, but a solid GitOps strategy will smooth the transition, and it will definitely be worth avoiding the stress that’s churning in your stomach right now as you have another cup of coffee to make up for not sleeping last night

Let’s look at specifically what it takes to perform a strategic migration to an API gateway model.

Phase 1: Planning and Preparation

Just as with everything else in software, the foundation of a successful API gateway migration lies in meticulous planning and preparation. This initial phase sets the stage for a smooth transition by addressing all critical aspects, from gathering detailed requirements to engaging stakeholders and evaluating potential solutions.

By taking the time to thoroughly prepare, you can identify and mitigate risks, align the migration with business objectives so the execs are on your side, and make sure that all the resources and tools you’ll need are in place. Effective planning not only reduces the likelihood of unforeseen issues but also enhances the overall efficiency and success of the migration process.

Let's look at the key steps involved in this phase.

Step 1: Requirement Gathering and Analysis

The first step in a successful API gateway migration is thorough requirement gathering and analysis. Identify and document both business and technical requirements, considering current pain points and future scalability needs. Conduct a detailed analysis of the existing API infrastructure, evaluating functionalities, dependencies, and potential integration challenges. By establishing a comprehensive understanding of requirements, you lay a solid foundation for a strategic and well-executed migration process.

Step 2: Stakeholder Engagement

In this phase you’ll want to closely collaborate with stakeholders to make sure you understand their needs and expectations so the new API gateway aligns with organizational goals.This kind of effective stakeholder engagement is crucial for a successful API gateway migration.

Start by identifying key stakeholders across departments, including development, operations, security, and business units. Engage with these stakeholders early to gather their input and address their concerns. You want to clearly communicate the benefits of the migration, such as improved security, scalability, and centralized management, but don’t forget to highlight potential risks and the mitigation strategies in place.

Regular updates and transparent communication throughout the process will help promote trust and collaboration, ensuring all parties are aligned and supportive of your migration efforts. What’s more, engaged stakeholders are more likely to contribute valuable insights and resources, helping move toward the migration's success.

Step 3: Create a Migration Plan

Now that you’ve spoken to stakeholders it’s time to create a detailed migration plan. Make sure that you:

Outline each phase of the migration, specifying tasks, timelines, and responsible parties. Include key milestones, risk assessments, and contingency plans to address potential challenges.
Define clear objectives for each phase, from initial setup to full deployment, making sure that you align with business and technical goals.
Incorporate best practices and lessons learned from previous projects to optimize the migration strategy.

A well-structured migration plan provides a roadmap for the project, enabling smooth execution, effective resource management, and timely delivery of the new API gateway solution.

Step 4: Assessment and Selection

Now that you know what you need, you can choose a new API gateway. Don’t skimp on the selection process, because you’ll likely live with your choices for a long time. The selection process consists of several steps:

Start by evaluating various API gateway options against your documented requirements, focusing on scalability, security, ease of integration, cost, and support.
Make sure the gateway you choose supports GitOps so you can easily make and rollback changes to your configuration.
Consider the community and vendor support available for each option.
Conduct proof-of-concept tests to validate the gateway’s performance and compatibility with your existing systems.
Involve stakeholders in the evaluation process to gather diverse perspectives and ensure all needs are addressed.

Selecting the right API gateway lays the groundwork for a successful migration and long-term operational efficiency (not to mention fewer sleepless nights) so take all the time you need (within reason).

You can learn how to evaluate modern API gateways in our Buyer’s Guide for Modern API Gateways.

Step 5: Define Success Criteria

Now that you know what you’re trying to do, you need to decide how you’ll know when you’ve done it. Defining clear success criteria is essential to measure the effectiveness of the API gateway migration. Good success criteria will:

Establish specific, measurable goals beyond just completing the migration from point A to point B.
Include key performance indicators (KPIs) such as improved latency, fewer security vulnerabilities, and increased scalability.
Set benchmarks for user satisfaction, system reliability, and overall performance improvements.
Ensure these criteria align with both business and technical objectives, providing a comprehensive framework for evaluating the migration’s success.

Clearly articulate what success looks like, so you can focus your efforts, track your progress, and demonstrate the value of the migration to stakeholders.

Step 6: Environment Setup

Now you’ll need the environment in which to perform the migration. Obviously you’ll need a testing environment, because you wouldn’t run this directly in production, right? Right???

You’ll want a testing environment that closely mirrors the production setup to identify and address potential issues early. Make sure all the necessary tools and resources, such as monitoring systems, logging mechanisms, and CI/CD pipelines, are in place and properly configured.

Basically, make sure you can do thorough testing and validation of the new API gateway before it goes live. You need to be able to mitigate risks, improve system stability, and ensure a smoother transition to the new gateway.

Phase 2: Initial Rollout

The initial rollout phase is where the groundwork you laid during planning and preparation begins to take shape. This phase focuses on carefully selecting a pilot group of APIs for migration, configuring and deploying the new API gateway, and conducting thorough testing to ensure everything operates smoothly. It also emphasizes the importance of monitoring and gathering feedback to refine the process before moving on to broader implementation. By starting with a controlled and manageable subset of APIs, you can identify and address any issues early on.

Step 1: Pilot Group Selection

Selecting the right pilot group is critical for a successful initial rollout of the API gateway migration. You want to identify a small, manageable set of APIs and functionalities to migrate first, prioritizing non-critical or low-risk APIs to minimize potential impact of any issues.

Choose APIs that represent a variety of use cases to comprehensively test the new gateway's capabilities, and involve a cross-functional team of developers, testers, and users to ensure diverse feedback. Having the proper team helps you validate the migration process, identify and resolve issues early, and build confidence for subsequent phases of the migration.

Step 2: Configuration and Deployment

The configuration and deployment phase is where everything comes to fruition. You’ll want to:

Begin by configuring the new API gateway according to the documented requirements, ensuring all settings align with your security, performance, and integration needs.
Deploy the selected APIs to the new gateway in the testing environment, meticulously following your migration plan.
Utilize automated deployment tools to streamline the process and reduce the risk of human error.

If this step fails, your migration fails, so be extra diligent.

Step 3: Testing

Now you need to thoroughly test to ensure the new API gateway functions correctly and meets all specified requirements. To make sure that everything is working properly:

Perform a comprehensive suite of tests, including functional, performance, security, and integration testing.
Use automated testing tools to enhance efficiency and coverage, ensuring all aspects of the gateway are evaluated.
Verify that the new gateway handles API requests correctly, maintains performance standards, and adheres to security protocols.

Testing should simulate real-world scenarios to uncover any potential issues so you can identify and resolve problems early, before they impact your users.

Step 4: Monitoring and Feedback

Once you’ve done the initial deployment of the API gateway, ongoing monitoring and feedback collection are crucial. Make sure that you:

Continuously track the performance and usage of the migrated APIs, comparing them against predefined success criteria.
Use monitoring tools to detect anomalies, measure response times, and assess overall system health.
Collect feedback from users and developers to identify any issues or areas for improvement.
Document lessons learned during this phase to refine the migration process.

At this stage, negative comments are actually a good thing, because they will help you make the system better. By actively monitoring and gathering feedback, you can promptly address any problems and optimize not only the gateway's performance, but also the overall process internally.

Phase 3: Gradual Expansion

Now that you’ve done the initial rollout and you know that everything works, it’s time to start giving users access to the new gateway.

This phase involves iteratively migrating additional APIs to the new gateway in small, manageable batches, doing regular testing, monitoring for issues, and keeping stakeholders in the loop. To ensure continuity and reliability, in this phase you’ll operate the old and new gateways in parallel.

Step 1: Parallel Operation

In this phase you’re making the new API gateway available, but you’ll still need the old one, both as a fallback for the migrated APIs and because it’s got the non-migrated API’s on it. In other words, operating the old and new API gateways in parallel ensures a safety net during the migration process.

This dual operation enables you to seamlessly switch between gateways if issues arise, minimizing downtime and service disruptions. Implement version control and routing strategies to efficiently manage API traffic between the two gateways and ensure smooth transitions and consistent performance.

This parallel setup also provides an opportunity to compare the performance and functionality of both gateways in real-time. By maintaining parallel operation, you can safeguard against potential problems, ensure reliability, and build confidence before fully decommissioning the old gateway.

Step 2: Iterative Migration

Iterative migration involves gradually moving additional APIs to the new gateway in manageable batches. When you started, you started with the least critical so that any issues that came up caused the least amount of disruption. Now it’s time to reverse that. Prioritize the business's most important APIs (e.g., the biggest volume, or the most complex) to move the most workloads to the new API gateway.

Another approach is to start with the APIs that hurt the most, then the rest is easy. If you leave the most painful APIs for last, there is a very good chance that they will not be migrated as the team's focus shifts over time.

After each batch, perform the same thorough testing, monitoring, and feedback collection as in the initial rollout phase. This incremental approach allows for continuous assessment and adjustment, reducing risks and ensuring stability at each step.

Step 3: Stakeholder Communication

Effective stakeholder communication is essential throughout the migration process. To keep in touch with stakeholders:

Regularly update users, developers, and business stakeholders on the progress of the migration, highlighting milestones and addressing any concerns.
Share insights from monitoring and feedback phases to demonstrate transparency and build trust.
Evaluate the migration's impact against success criteria and communicate these results to stakeholders, showing tangible benefits and improvements.

By maintaining open lines of communication and involving stakeholders at every stage, you can ensure alignment with organizational goals and get continued support for the migration initiative, which will be important at budget time.

Step 4: Documentation and Training

Comprehensive documentation and training help you make sure your transition to the new API gateway is smooth. In this step, do the following:

Update all API documentation to reflect the changes and new capabilities introduced by the new gateway.
Make sure that this documentation is clear, detailed, and easily accessible to developers and users.
Provide training sessions to familiarize the development team and end-users with the new system.
Offer ongoing support and resources, such as tutorials and FAQs, to assist with the transition.

By investing in thorough documentation and effective training, you can facilitate user adoption, reduce onboarding time, and enhance the overall success of the migration.

Phase 4: Full Migration and Decommissioning

You’re almost there! The final phase of the API gateway migration involves the complete transition of all remaining APIs and the careful decommissioning of the old gateway. This phase focuses on ensuring that all functionalities and dependencies are successfully replicated or enhanced in the new gateway.

For example, post-migration monitoring is crucial to verify stability and performance, while continuous improvement efforts ensure the gateway remains optimized and aligned with business needs. Additionally, the decommissioning process involves securely phasing out the old infrastructure, ensuring no residual traffic or dependencies remain. By executing a thorough and methodical migration and decommissioning, you can achieve a successful transition and position your organization for ongoing API gateway success.

Let's delve into the specific steps involved in this final migration and decommissioning phase.

Step 1: Final Migration

The final migration phase involves moving the remaining APIs to the new gateway, ensuring that all functionalities of the old gateway are replicated or enhanced. You’ll want to:

Conduct a thorough review to confirm that all dependencies are addressed and that no critical elements are overlooked.
Execute the migration carefully, following established procedures to minimize disruptions.
Once the final APIs are successfully migrated, perform comprehensive testing to verify that the new gateway operates smoothly and meets all performance, security, and functionality criteria.

Be meticulous in this step; you’re almost finished, it would be a shame if everything fell apart now, just when you were starting to believe you were going to be rid of all this pain.

Step 2: Decommissioning

Decommissioning the old API gateway is the final step in the migration process, coming after all workloads are being executed on the new gateway. To get there:

Verify that all dependencies are removed and that no residual traffic is being routed through the old gateway.
Conduct a thorough audit to confirm that all functionalities have been successfully transferred and are operating as expected on the new gateway.
Once you’re comfortable that everything has been moved to the new system and is running properly, you can proceed with securely shutting down and dismantling the old gateway infrastructure.

Proper decommissioning ensures a clean transition, reduces maintenance overhead, and eliminates potential security vulnerabilities associated with the outdated system. So, tempting as it may be to leave the old system in place “just in case,” don’t.

Step 3: Post-Migration Monitoring

Now that you’ve shut down the old system you’ll be tempted to consider the project finished, but it’s not. You need to continuously monitor the new gateway to track its operation, identify any issues, and assess its performance against predefined success criteria, using monitoring tools to gather real-time data on API usage, latency, and error rates.

Should any post-migration issues pop up (as they almost definitely will) make sure to address them immediately to maintain service reliability and user satisfaction. Regularly review and analyze the monitoring data to identify trends and areas for further optimization.

Effective post-migration monitoring helps maintain the integrity of the new gateway and supports ongoing improvements.

Step 4: Continuous Improvement

Speaking of ongoing improvements, continuous improvement is key to maximizing the benefits of your new API gateway. After the migration, regularly evaluate the gateway's performance, security, and scalability to identify areas for enhancement and implement iterative updates and optimizations based on user feedback and monitoring data.

Stay informed about emerging technologies and best practices to keep your API gateway up-to-date. The idea is to create a culture of continuous learning and adaptation within your team to proactively address challenges and leverage new opportunities. By committing to continuous improvement, you ensure that your API gateway remains robust, efficient, and aligned with evolving business needs.

Additional Considerations

Now, while all of that may cover the actual migration, there are a few more things you need to consider, both positive and negative.

Security

Ensuring robust security measures is essential when migrating to a new API gateway, so make sure that you:

Implement strong authentication and authorization protocols to protect your APIs from unauthorized access.
Use encryption to safeguard data in transit and at rest.
Regularly conduct security audits and vulnerability assessments to identify and address potential threats.
Integrate security best practices into your development and deployment processes, ensuring that security is a continuous focus.
Stay updated on the latest security trends and threats to proactively defend against emerging risks.

Remember, security shouldn’t be a “bolt-on.”

Scalability and Performance

Designing for scalability and performance is crucial for the long-term success of your new API gateway, and is probably one of the reasons you decided to upgrade in the first place. To do that:

Ensure the gateway can handle increased traffic and grow alongside your business needs by implementing scalable architecture and infrastructure.
Optimize performance by fine-tuning configurations, employing efficient load balancing, and utilizing caching strategies to reduce latency.
Regularly conduct performance testing to identify bottlenecks and areas for improvement.
Monitor the gateway's performance metrics to ensure it meets or exceeds current capabilities and adapts to varying loads.

By focusing on scalability and performance, you can deliver a reliable and responsive API experience to your users.

Easy Rollbacks

When you’re selecting a new API gateway, you want to make sure to implement easy rollback mechanisms to ensure that configuration changes and deployments can be quickly reversed in case of issues.

The best way to do this is to use a system that is based on version control, tracking changes, and maintaining previous versions of configurations and code. This enables you to automate rollback procedures within your CI/CD pipeline to enable rapid and reliable reversion to stable states.

This architecture is called GitOps, because you are essentially controlling your system by making changes to a git repository and merging them. The changes are then propagated to the API gateway platform’s configuration. (Remember that not all API Gateways support this capability. Traefik API Gateway is one that does.)

Regularly test rollback processes in your testing environment to ensure they function as expected. By planning for easy rollbacks, you can quickly address unforeseen problems, maintain service continuity, and reduce the impact of potential disruptions during the migration process.

Backup and Recovery

As with anything else in software, implementing a robust backup and recovery plan is critical for safeguarding your data during the API gateway migration. Make sure to:

Regularly back up all configurations, data, and critical system components to secure storage locations.
Ensure that backups are comprehensive and include all necessary elements to restore operations in case of data loss or corruption.
Test recovery procedures frequently to verify that backups can be restored quickly and effectively.
Implement automated backup solutions to maintain up-to-date copies of your data.

All of this is in order to ensure business continuity, protect against data loss, and quickly recover from any unexpected issues during the migration.

Team Collaboration

Strong team collaboration is vital for a successful API gateway migration, and fortunately, the way GitOps works fosters that spirit of collaboration, as everyone can work together without stepping on each other. You can further this goal by making sure to:

Promote open communication and collaboration among all team members, including developers, operations, security, and business stakeholders.
Use collaborative tools and platforms to share information, track progress, and coordinate tasks effectively.
Conduct regular meetings and update sessions to keep everyone aligned and informed about the migration’s status and any emerging issues.
Encourage a culture of teamwork and collective problem-solving to leverage diverse expertise and perspectives.

By enhancing team collaboration, you can ensure a smoother migration process.

Compliance and Governance

Ensuring compliance and robust governance is essential when choosing and migrating to a new API gateway. Some things to consider:

Align the new gateway with relevant regulations and industry standards, such as FIPS-140-3, GDPR, HIPAA, or PCI-DSS, to maintain compliance and avoid legal issues.
Implement governance policies to manage the API lifecycle, including versioning, deprecation, and documentation standards.
Regularly review and audit the gateway to ensure ongoing adherence to compliance requirements and internal policies.
Establish clear roles and responsibilities for governance to maintain accountability and transparency.

By focusing on compliance and governance, you can protect your organization from regulatory risks and ensure the API gateway operates within established guidelines.

Conclusion

Migrating to a new API gateway is complex but highly rewarding. It requires strategic planning, meticulous execution, and continuous improvement. However, by following a phased approach, you can minimize risks, ensure a smooth transition, and achieve significant improvements in API management, security, and performance.

Each phase, from initial planning and preparation through to full migration and decommissioning, plays a crucial role in the overall success of the project. Additionally, addressing key considerations such as security, scalability, compliance, and team collaboration further strengthens the migration process.

As an IT Director, leading this strategic migration not only enhances your organization’s technological capabilities but also positions you for long-term success in managing and optimizing your operations.

Embrace the journey, leverage the insights gained along the way, and transform your API infrastructure for the future.

... And finally get some sleep.

]]>

Getting started with Kubernetes Gateway API and Traefik

Nicolas Mengin — Tue, 16 Jul 2024 17:43:58 GMT

We're continuing our in-depth series on Traefik 3. If you missed it, be sure to read the previous articles on migrating from Traefik v2, WASM support with Coraza WAF, Open Telemetry, and SPIFFE, Tailscale, and HTTP/3. This article dives into how to get started with GatewayAPI and Traefik.

In 2015, when Traefik was just born, Kubernetes released the first version of its Ingress specifications. The goal of Ingress was to provide a straightforward, vendor-neutral method for exposing Kubernetes services. However, the premise has quickly been broken.

While Ingress could handle basic service exposure, adding functionalities like rate-limiting or IP checking required numerous vendor-specific annotations.

To address these limitations, a new standard began to take shape in 2019: GatewayAPI was born. Officially released in its v1.0 form at the end of 2023, the GatewayAPI aims to be the new standard for exposing services in Kubernetes. It offers a set of core rules designed to meet most needs while allowing the Gateway Controller to extend these rules with their features.

Traefik has been a Kubernetes first-class citizen for a long time, initially supporting Ingress and later expanding with our IngressRoutes to surpass Ingress limitations, adding features like TCP, UDP, and structured options declaration.

We quickly embraced the big changes in Kubernetes with the Gateway API, offering early experimental support since 2022.

With the release of Traefik v3.1, we’ve taken a major step forward: our GatewayAPI provider is now production-ready. Traefik v3.1 today meets and exceeds 100% of the core requirements, as shown in our SIG network conformance tests report.

Let’s see together how you can start with GatewayAPI and Traefik v3!

Install Traefik as GatewayController

To begin, ensure you have installed the most recent version of Traefik on your Kubernetes cluster, utilizing the Helm Chart for streamlined deployment. This setup enables the GatewayAPI provider to effectively discover and expose your applications.

Define your installation configuration

First, customize your values to enable Traefik to discover GatewayAPI objects on your Kubernetes cluster. Create a values.yaml file with the following minimal configuration:

## File values.yaml ##
providers:
  # Disable the Ingress provider (optional)
  # We do not want to use Ingress objects anymore!
  kubernetesIngress:
    enabled: false
  # Enable the GatewayAPI provider
  kubernetesGateway:
    enabled: true
# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  namespacePolicy: All

Next, execute the following commands to deploy Traefik in the traefik namespace using the previously described configuration:

# helm commands to execute:

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ kubectl create namespace traefik
$ helm upgrade --install --version 0.29.1 --namespace traefik traefik traefik/traefik -f values.yaml

And voilà! You have deployed Traefik on your Kubernetes cluster:

$ kubectl describe deployments.apps traefik --namespace traefik
Name:               	traefik
Namespace:          	traefik
…
  Containers:
   traefik:
	Image:   	docker.io/traefik:v3.1
	Ports:   	9100/TCP, 9000/TCP, 8000/TCP, 8443/TCP
	Host Ports:  0/TCP, 0/TCP, 0/TCP, 0/TCP
	Args:
  	--entryPoints.web.address=:8000/tcp
  	--entryPoints.websecure.address=:8443/tcp
  	--api.dashboard=true
  	--ping=true
  	--providers.kubernetescrd
  	--providers.kubernetesgateway
  	--entryPoints.websecure.http.tls=true
	…

As you can see, both kubernetescrd and kubernetesgateway providers are enabled. The kubernetescrd provider allows you to define Traefik specific resources like Middlewares.

The entrypoints web and websecure will be used to expose your applications.

Additionally, when Traefik is installed with the GatewayAPI provider enabled, it automatically creates a default GatewayClass named traefik:

$ kubectl describe GatewayClass traefik
Name:     	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	GatewayClass
…
  Controller Name:  traefik.io/gateway-controller

As you can see above, the option Controller Name is set to traefik.io/gateway-controller. Thus, it allows you to expose every Gateway attached to the traefik GatewayClass using your Traefik Gateway Controller.

Additionally, the Gateway traefik-gateway is also deployed:

$ kubectl describe Gateway traefik --namespace traefik
Name:     	traefik-gateway
Namespace:	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	Gateway
…
Spec:
  Gateway Class Name:  traefik
  Listeners:
	Allowed Routes:
  	  Namespaces:
    	    From:  All
    Name:  	web
    Port:  	8000
    Protocol:  HTTP

This Gateway allows you to expose HTTPRoute in HTTP on the port 8000 of your Traefik Gateway Controller.

Let’s now expose your first application using a GatewayAPI HTTPRoute.

Deploy your first HTTPRoute

A GatewayAPI HTTPRoute enables you to expose an application through Gateways using HTTP(S).

To do this, reference the Services to reach (using the backendRefs option) and the Gateways (using the parentRef option) as shown in the snippet below:

# Application to expose
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
    app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
---
# Service to reach the application on the cluster
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: whoami
  selector:
    app: whoami
---
# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80

This example demonstrates how to expose a web server that displays some headers when Traefik is accessed using the hostname whoami.myexample.io.
You can use the following curl command to verify that the application is correctly exposed:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-7nqmf
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.9
IP: fe80::e8c0:86ff:feba:5e06
RemoteAddr: 10.42.0.14:59316
GET / HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Real-Ip: 10.42.0.1

Congratulations, you've successfully deployed your first HTTPRoute with Traefik! Now, let's take it a step further by adding a few operations to the requests.

Go beyond with Filters

GatewayAPI Filters enable Traefik to perform various operations on requests and responses.

There are three types of filters:

Core Filters: Mandatory filters for every GatewayController, such as requestHeaderModifier and requestRedirect.
Extended Filters: Optional filters for GatewayControllers, such as responseHeaderModifier and requestMirror.
ExtensionRef Filters: Additional filters provided by the GatewayController. In Traefik, these are the HTTP middlewares.

Let's modify the HTTPRoute to:
Add the header x-post-topic using the core filter.
Add the prefix gatewayapi to the request using the middleware AddPrefix.

# HTTPRoute with the Filters
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80
    filters:
    # Core filter which adds a header
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: x-post-topic
            value: GatewayAPI
    # ExtensionRef filter to use the Traefik Middleware AddPrefix
    - type: ExtensionRef
      extensionRef:
        group: traefik.io
        kind: Middleware
        name: addprefix
---
# Traefik Middleware AddPrefix
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: addprefix
  namespace: whoami
spec:
  addPrefix:
    prefix: /gatewayapi

You can check the modification using a curl command:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-2xkpr
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.10
IP: fe80::b8b5:6fff:fe48:19e9
RemoteAddr: 10.42.0.14:40044
GET /gatewayapi/ HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Post-Topic: GatewayAPI
X-Real-Ip: 10.42.0.1

As we have seen, the GatewayAPI integration in Traefik simplifies the initial setup significantly. We successfully exposed our first HTTPRoute and added a couple of operations to the requests. Traefik has many more GatewayAPI capabilities, such as TCPRoute, TLSRoute, and ReferenceGrant that are not covered in this blog post, but we highly encourage you to explore them further.

Conclusion

GatewayAPI is undoubtedly the new standard for exposing your resources in a Kubernetes cluster. Its specifications already surpass those of Ingress, and with a strong community backing it, GatewayAPI will continue to grow.

As a Kubernetes first-class citizen, Traefik is of course strongly supporting this effort and will continue to contribute to this community project.

We've improved status management, updated route priority rules, and introduced the ReferenceGrant feature to make Traefik even more robust and flexible for your Kubernetes setup.

But it’s just the beginning! We plan to further advance this integration by adding support for features like GRPCRoute, UDPRoute, and BackendTLSPolicy.

Our goal is to provide comprehensive support, enabling you to fully leverage GatewayAPI in your Kubernetes journey. Stay tuned for more updates!

Useful Links

]]>

Announcing Traefik Proxy v3.1

Emile Vauge — Tue, 16 Jul 2024 17:40:52 GMT

Traefik v3.0 was released less than 3 months ago with key new features introduced like WASM, Open Telemetry, and Kubernetes Gateway API support. The feedback from the community members about the v3 has been overwhelmingly positive (thank you!) and extremely motivating to continue improving Traefik Proxy. Today we are thrilled to release the v3.1 which further enhances WASM and Gateway API integrations.

With Gateway API now poised to become the new standard for exposing resources within a Kubernetes cluster, we are proud to announce that Traefik v3.1’s Kubernetes Gateway API is ready for production use 🎉

Let’s jump in!

Gateway API now production ready

Traefik has been a GatewayController since the early days of the Gateway API specification, but needed some adjustments to pass the conformance tests and fully meet the specification. Traefik v3.1 now meets and exceeds 100% of the core requirements, as shown in our SIG network conformance tests report. Our GatewayAPI provider is now ready for production use!

Oh, by the way, we just released a deep dive into Gateway API & Traefik, we strongly invite you to check it out.

Let’s see what’s new in v3.1 with Gateway API. First things first, If you were already using Gateway API in Traefik, you can now remove the experimental option from the helm chart:

## File values.yaml ##
experimental:
  kubernetesGateway:
    enabled: true

As usual, you can enable Gateway API by simply enabling kubernetesGateway:

## File values.yaml ##
providers:
  # Disable the Ingress provider (optional)
  # We do not want to use Ingress objects anymore!
  kubernetesIngress:
	enabled: false
  # Enable the GatewayAPI provider
  kubernetesGateway:
	enabled: true
# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  namespacePolicy: All

Now that you are up and running, let’s dig deeper into the changes made in Traefik.

Improved Status Management

Status management is an important piece of the Gateway API, enabling real-time monitoring of your infrastructure to ensure your GatewayAPI objects are ready to manage traffic. In Traefik v3.1, we've improved support for status management across all GatewayAPI objects, especially HTTPRoutes. Our status calculation now aligns perfectly with the GatewayAPI specification, providing you a ready-for-production status monitoring of your objects.

Route Priority Updates

Traefik has always used a route priority mechanism based on the rule length to prevent overlaps.

However, the GatewayAPI specification has its own priority rules, which differ from Traefik. In Traefik v3.1, we've updated our priority calculation for HTTPRoutes. This ensures that you can switch to Traefik from any other GatewayController without changing your route-matching system.

Introducing ReferenceGrant

The Ingress specification has always struggled with cross-namespace references because of security issues. However, in some cases, like multi-tenant environments, cross-namespace references are essential. Traefik has previously allowed such references through our custom IngressRoute, by adding a specific option to allow it (though security concerns remain).

With GatewayAPI's ReferenceGrant object, these security issues are addressed.

Using ReferenceGrant, you can now declare a Gateway in Traefik that serves a TLS certificate stored in a secret from another namespace or an HTTPRoute targeting a Service in a different namespace. This new feature makes cross-namespace referencing secure and straightforward.

# HTTRoute in the default namespace.
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-http
  namespace: default
spec:
  parentRefs:
    - name: traefik
      kind: Gateway
  rules:
     - backendRefs:
        - name: whoami
          namespace: whoami
          port: 80

# ReferenceGrant and Service in the whoami namespace.
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: whoami
  namespace: whoami
spec:
  from:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      namespace: default
  to:
    - group: ""
      kind: Service
      name: whoami

---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
spec:
  selector:
    app: whoami
  ports:
    - port: 80

Beyond the core features

In addition to the core features required to meet the specification, GatewayAPI offers optional extended features that a GatewayController can implement. Traefik v3.1 brings several of these extended features, including HTTPURLRewriteFilter, HTTPRouteRedirect, and support for method and query parameter matching.

More than the specification

When we started our journey with GatewayAPI, our goal was not just to meet the specification but to bring the same robust feature set as our own Kubernetes provider. That's why Traefik v3.1 also includes support for TCPRoute and TLSRoute, as well as the ability to add Traefik middlewares to your HTTPRoutes using the ExtensionRef mechanism. These features provide even more flexibility and control over your traffic management.

# HTTRoute in the default namespace.
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-http
  namespace: default
spec:
  parentRefs:
    - name: traefik
      kind: Gateway
  rules:
    - backendRefs:
       - name: whoami
         namespace: default
         port: 80

      filters:
       - type: ExtensionRef
         extensionRef:
           group: traefik.io
           kind: Middleware
           name: add-prefix


# Traefik Middleware.
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: add-prefix
  namespace: default
spec:
  addPrefix:
    prefix: /prefix

WASM better than ever

Traefik v3.0 introduced support for WASM plugins along with the original Yaegi plugins. However, the WASM support had some limitations. One of the limitations was the inability to make HTTP calls using the Go standard library from plugins. The technical reason for this is that a function export mechanism is needed for this, and it’s not yet implemented in Go (but is an accepted proposal).

The team found a workaround to simulate WASM exports in a WASM compiler in Go. It means, as of today, it is possible to create a full featured WASM plugin in Traefik which does HTTP calls (through the host), import any Go library, etc.

On top of that, we added the possibility to mount shared directories in plugins and also to configure environment variables.

experimental:
  plugins:
    example:
      moduleName: github.com/traefik/plugindemowasm-http-call
      version: v0.0.2
      settings:
        mounts:
          - /path:/path:ro # Read only mount
          - /tmp/test:/tmp/test # Read Write mount
        envs:
          - TEST_ENV_1
          - TEST_ENV_2

Here is a simple example of a plugin in wasm that makes HTTP calls through the host.

# Static configuration
experimental:
  plugins:
    example:
      moduleName: github.com/traefik/plugindemowasm-http-call
      version: v0.0.2
---
# Dynamic configuration
http:
  routers:
    my-router:
      rule: host(`demo.localhost`)
      service: service-foo
      entryPoints:
        - web
      middlewares:
        - my-plugin

  services:
   service-foo:
      loadBalancer:
        servers:
          - url: http://127.0.0.1:5000
  
  middlewares:
    my-plugin:
      plugin:
        example:
          headerName: X-World-Time

Thanks to this innovative and unique approach, WASM is now a powerful plugin platform in Traefik that we will continue to improve upon moving forward.

Other Improvements

Several contributions were made on Kubernetes:

By Marc Mognol who brought health checks to ExternalName services
By Jesper Noordsij who migrated the Kubernetes provider to the EnpointSlices API
By Joris Vergeer who added the possibility to use Node IPs for NodePort services

Another great contribution came from Antoine Aflalo who added Zstandard to the compress middleware (in addition to Gzip & Brotli). This algorithm is much faster, especially at decompressing.

Support for Content-Security-Policy-Report-Only was added by Roman Donchenko to the headers middleware.

Finally, we added support to systemd socket activation, which allows systemd to listen on socket and dynamically start the associated service. Simply use the same name for your entrypoint and file descriptor, and Traefik will start on systemd demand.

The full release note is available here.

Conclusion

Traefik release notes are usually pretty packed and this one is no exception. Traefik v3.1 brings critical features to the project with state-of-the-art WASM support that makes Traefik’s plugin platform one of the best in the industry. Being a Kubernetes native product, it goes without saying that Traefik closely follows the latest evolutions of the platform. Traefik is now getting full support of Gateway API v1.1.0 and is ready for production use.

Traefik is almost 10 years old, but like a good wine, is still getting better and better 🙂.

Lastly, a huge thank you to all contributors. Your assistance is invaluable.

See you on GitHub!

Useful Links

Traefik 3.1 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

The API Gateway Model: Centralizing Control in Complex Microservices Architectures

Immánuel Fodor — Wed, 26 Jun 2024 22:24:48 GMT

Imagine an organization with hundreds or even thousands of microservices, each responsible for a specific function or feature. Routing requests across these services, managing authentication and authorization, load balancing traffic, and ensuring system resilience can quickly become a daunting task.

Mortgage provider Amerisave, with a presence in 49 states, 5,000 employees, and on-prem legacy infrastructure with more than 150 servers handling up to 40,000 requests per second, had to consider that in its migration to microservices using Docker Swarm. Without centralized control of routing, security, and other distributed functions, Amerisave and organizations like it risked introducing silos, inconsistencies, and potential security vulnerabilities, ultimately making it more difficult to scale and adapt to changing business needs.

One such solution that exemplifies the principles of centralized control of distributed functions is the use of an API gateway.

The API Gateway Model

The API gateway model provides companies with a centralized way to manage routing, authentication, and authorization. This model enables developers to offload the complexities of routing and security to DevOps, platform, or security teams, allowing developers to focus on building the applications. Let’s discuss the architecture of this model.

But first, what is an API gateway?

An API gateway is a single entry point into your microservices that routes your clients’ API requests to the backend microservices in your network. An API gateway functions very similarly to a reverse proxy for standard user-facing applications in that it routes incoming requests to the correct API in the backend.

Considering we have covered this already in our glossary, I won’t be going in more depth here. If you need a refresher on the basics, please see this API Gateway glossary article.

API gateway in modern infrastructure

Because API gateways live in front of all services and DevOps teams are able to centralize control, the API Gateway model is far more efficient than models that require, for example, authentication and authorization to be configured in each service.

There are numerous benefits to the API gateway model, a few of which we’ll explore here.

Improved Routing, Load Balancing, and System Resilience

At the core of centralized control lies the ability to consolidate routing logic and load balancing mechanisms. Because they’re the single entry point for all external traffic, API gateways simplify routing in complex architectures and eliminate the need for individual microservices to handle routing logic independently, ensuring consistency, reducing redundancy, and facilitating easier maintenance and updates.

Additionally, modern, cloud native API gateways offer dynamic service discovery with automated default configurations, advanced load balancing (blue/green, canary, etc.), and intelligent routing based on factors such as resource utilization, response times, and health checks. Add rate limiting and circuit breaking to this list, and you can see why API gateways are effective tools for streamlining the management of complex architectures, fostering scalability, and enhancing system resilience.

Better Authentication and Authorization Management

A core reason companies move towards an API gateway model is security. Not only do API gateways tend to be compatible with modern authentication and authorization protocols (OIDC, JWT, etc.), but they also streamline security management by centralizing access control and even helping to automate it.

Managing access control at the service level, in contrast, requires tedious manual configuration, increases the risk of human error, and opens you up to attacks. Add to this that most companies have hundreds, if not thousands of microservices and APIs to expose, and you can see how embedding access control into each service is not scalable and overly burdensome on developers.

Managing access control on service level

By enforcing authentication and authorization policies at the API gateway level, you’re able greatly simplify the configuration and maintenance of new and existing access control policies. And with everything in one place, managed by one team, policies can be enforced consistently (and even automatically with dynamic service discovery and configuration). This leaves less room for human error, reduces development overhead, and minimizes vulnerabilities from inconsistent implementations across different services.

Managing access control via API gateway model

If you choose a modern API gateway for your stack, you’ll also be able to take advantage any authentication and authorization protocol—including LDAP, JWT, OAuth, OpenID Connect, HMAC, and API keys—as well as direct integrations with identity providers (IdPs). The best gateways also support advanced authorization mechanisms, such as Open Policy Agent (OPA), which allow organizations to implement fine-grained access control policies based on complex rules and contextual data.

Streamlined Change Management

As new instances of microservices are introduced or existing ones are scaled up or down, they have to be configured. Whether it’s registering, routing, load balancing, or securing the service, change management is no small feat.

And the more decentralized and manual this process, the more likely you are to have issues. With the API gateway model, however, this can all happen at a single point, ensuring optimum consistency and control.

The best API gateways take this even further and automate service discovery and default configuration. As your team spins up new services or makes changes to existing ones, the API gateway gets to work. New services are registered and the default routing, load balancing, rate limiting, access control, and other defaults are applied dynamically. The best solutions are able to deploy these changes dynamically without restarts or service disruptions.

This dynamism ensures teams remain agile and responsive to changing business needs, which is crucial in today's fast-paced business environment.

Comprehensive Monitoring and Observability

Monitoring and observability are essential. Whether it's identifying performance bottlenecks, detecting issues before they happen, or ensuring compliance with service-level agreements, the ability to monitory your system is a must.

As architectures become more complex, having a centralized tool that gathers and monitors system data becomes all the more important. This is where an API gateway model can help.

Monitoring traffic flow, load balancing, and overall system health is much more challenging without a centralized point through which all this data flows. And considering how API gateways help you centralize routing, security, etc., observability is no different.

API gateways offer some built-in metrics that cover a wide range of dimensions. These can include request rates, response times, error rates, and more. They may also provide built-in dashboards or the ability to visualize data using third-party observability tools, such as Prometheus, Grafana, and Elasticsearch.

This is nice, but modern API gateways take observability to the next level. They are increasingly aligned to OpenTelemetry (OTel), a standard for dynamic systems telemetry data that replaces multiple vendor-specific solutions. OTel unifies disparate data, ensures consistent instrumentation across various programming languages, and offers enhanced insights into application use, health, and performance.

For example, API gateways with best-in-class OTel support can illuminate request patterns, authentication and authorization events, system-level events, and issue correlations. You may even have the ability to remove or mask GDPR-related Personally Identifiable Information (PII) in logs to comply with security requirements. This level of visibility and control enables organizations to troubleshoot issues more effectively, identify potential security threats, and optimize their applications performance.

Integration with Orchestration Tools

The last benefit of the API gateway model we’ll discuss here is the stitching together of complex, often disparate, environments. As organizations move from monolithic to microservices architectures, they often need to manage routing, security, et al. across hybrid cloud, multi-cloud, and mulit-orchestrator deployment models.

This is no easy feat. And the larger the company, the more you’re likely have have different teams working in different environments. So how can you tame such a complex beast?

API gateways can be used as a unifying layer, ensuring consistent control and governance over distributed systems, regardless of the underlying infrastructure or orchestration platform. One consideration here is that not all API gateways are natively compatible with all orchestrators.

This is another situation where modern, cloud native API gateways shine. Regardless of your orchestrator or deployment model, the best gateway solutions will behave the same in all instances. They don’t require you to operate them differently depending on the environment, and they might even be able to communicate with each other to further centralize your operations.

With this level of flexibility, your organization can grow and change, being responsive to it’s needs today and in the future, without the pressures of vendor lock-in or imminent rip and replace headaches.

It’s Time to Centralize Control

The API gateway model has clear advantages, particularly as infrastructures scale and become more complex. Centralizing critical functions, such as routing, security, load balancing, and observability can have profound effects on your organization. Efficiency, consistency, agility, and scalability can all increase dramatically when an API gateway model is implemented well (and when the API gateway you choose is right for you).

Modern, cloud native API gateways magnify all the benefits of the API gateway model. From dynamic configurations to native support for all orchestrators and deployment models, the best of the best API gateways tame complexity and make it possible to scale how and where you need.

Are you ready for that?

]]>

5 Essential Qualities All API Gateways Need in 2024

Immánuel Fodor — Thu, 06 Jun 2024 22:49:07 GMT

In the constantly evolving digital landscape, APIs are the connective tissue that powers modern applications. A well-designed API gateway acts as the central nervous system for your API ecosystem, streamlining access, management, and security. But with the rapid pace of innovation, simply having an API gateway isn't enough. In 2024, you need a modern API gateway to stay ahead of the curve and not be left behind in the digital stone age.

Let’s explore the five essential features your API gateway must possess to empower your developers, elevate security, and maximize business agility. We'll explore simplicity and ease of use, container orchestration integration, security features, extensibility, and embracing GitOps principles for declarative configuration. With these at the top of your evaluation criteria, you'll be better equipped to select the perfect API gateway for your organization's needs.

#1 Simplicity and Ease of Use are The Cornerstone of Agility

Developers are your key partners. A modern API gateway should prioritize developer experience (DX) by offering a clean, intuitive interface and a robust set of tools, ensuring they can navigate, configure, and deploy APIs and microservices without unnecessary friction. This translates to:

Reduced learning curve: A convenient user interface and clear documentation enable developers to grasp the API gateway's functionalities quickly, minimizing onboarding time and maximizing productivity. Note that a user interface doesn’t necessarily mean a clickable UI; it can also be any interface that allows configuration.
Streamlined development: Powerful tools for defining routes, security policies, and request or response modification should be readily available along with automation, allowing developers to focus on business logic rather than getting bogged down in configuration.
Rapid prototyping and iteration: Simplicity fosters agility. The ease of publishing and modifying APIs within the gateway empowers developers to prototype and iterate, accelerating innovation and time-to-market rapidly.

Here are two real-world examples: Axione, a French telecommunications company, leveraged a user-friendly API gateway to save countless hours every month by reducing configuration updates in multiple clusters to just a few minutes with zero downtime. ABAX, a multi-national IoT (internet of things) company, spends very little time training people. They give them the basic understanding they need and then move on. Saving so much time, like these companies, can also allow you to capitalize on a critical market window that might never come again.

But simplicity extends beyond developer efficiency. A modern API gateway should also champion:

Faster time to change: Configuration changes should be straightforward and easily rolled out, minimizing downtime and disruption during updates. Imagine a scenario where a critical bug fix needs immediate deployment. A modern, streamlined API gateway enables this with minimal hassle.
Rapid disaster recovery: In the event of an outage, a modern API gateway with predictable configuration rollback features allows for swift restoration, ensuring business continuity and minimizing revenue loss. According to the trusted DORA metrics, companies with high, on-demand deployment frequency can minimize the change-failure rate to only 5% and recover from a failure in less than 1 hour (which can still cost businesses an average of $9,000 per minute, so you should cut it down as much as possible). Since the industry average for recovery is between a day and one week, a speed like this can put you at the top performing 20%, minimizing customer impact and protecting your brand reputation.

Finally, a modern API gateway shouldn't burden developers with tedious configuration tasks. Here's where features like the following help:

Sane defaults: Out-of-the-box configurations catering to common use cases minimize manual setup and avoid divergence from industry best practices.
Auto-discovery of services: Modern API gateways can automatically detect and integrate with backend services, saving developers valuable configuration time.
Scalability and high availability: The gateway should seamlessly scale to accommodate traffic spikes and ensure continuous API access, freeing developers from infrastructure concerns and allowing them to focus on building exceptional applications.

While many API gateways exist, some fall short on simplicity and ease of use. Avoid UI-heavy solutions because a complex, cluttered user interface can hinder developer productivity. Some API gateways on the market even require navigating a labyrinth of menus and options just to perform basic configurations.

Don’t frustrate your developers, nor waste their time! Also, beware of manual drudgery and gateways without auto service discovery. Manually configuring every backend service integration can be a tedious and error-prone process, especially for large deployments.

#2 Embrace the Container Revolution with Native Integrations

The landscape is shifting. Bare-metal deployments and virtual machines (VMs) are giving way to cloud-native architectures built on containerization. But considering cloud migration doesn’t happen overnight, modern API gateways need to be able to handle both legacy and cloud-based workloads.

Deep container orchestration integration, first and foremost, means Kubernetes. Kubernetes reigns supreme in the container orchestration realm. A modern API gateway should seamlessly integrate with Kubernetes, allowing you to leverage its powerful features for service discovery, load balancing, and health checks. This simplifies API deployment and management within your containerized environment.

But what about beyond Kubernetes?

A truly modern API gateway shouldn't limit you. It should offer first-class support for other cloud-native orchestrators like Docker Swarm and HashiCorp Nomad, ensuring flexibility for your specific infrastructure needs.
Modern applications often leverage a diverse set of backend services; furthermore, many organizations operate in hybrid environments, with a mix of on-premises infrastructure and cloud deployments. A modern API gateway should bridge this gap and be able to serve APIs from many sources simultaneously. It should seamlessly expose APIs residing on-premises and across various cloud providers and Kubernetes distributions. This flexibility empowers you to manage your entire API ecosystem from a single pane of glass.

Effective traffic routing is crucial for application performance and user experience to support all this complexity. A modern API gateway acting as the central hub for all your API traffic should offer intelligent routing capabilities, allowing you to route requests based on various factors like hostname, path, HTTP method, request headers, JWT claims, or even a mix of all these. This granularity empowers you to deliver the right service to the right user at the right time, optimizing performance and security.

To give you a real-life example, AmeriSave, a mortgage company in the financial services industry, faced similar challenges as they were migrating from an on-premises monolith to a cloud-based microservices architecture. They needed to keep APIs online from both systems during the transition. A flexible, multi-orchestrator API gateway helped them to make this happen and have a smooth and consistent process.

However, not all integrations are created equal, you should avoid pitfalls while working in the field. While container orchestration offers numerous advantages, some API gateways fall short in this area. Watch out for:

Limited Kubernetes capabilities: While some API gateways might offer a basic Kubernetes integration or can be run inside Kubernetes, true interoperability is key. Avoid API gateways that lack robust Kubernetes features like service discovery, health checks, and Kubernetes-native CRDs (Custom Resource Definition) to declare configuration, as these will only increase the manual configuration burden and potential for human error. Many solutions were born before Kubernetes or even the cloud-native era, so they lack this critical capability.
Cloud vendor lock-in: Beware of API gateways restricting you to a specific cloud provider's (Kubernetes) offering. A modern API gateway should be cloud-agnostic, supporting various Kubernetes distributions across multiple cloud providers. This flexibility ensures you're not locked into a single vendor and can adapt to your evolving needs.
Orchestration singularity: While Kubernetes is the leader, some situations can leverage other orchestrators, and vice versa, one single non-Kubernetes orchestrator support is also not enough. Make sure your API gateways don’t limit you to just one or two environments. Instead, seek one that offers broader support for various container orchestration platforms to make your containerization strategy future-proof.

These are just a few cautionary tales. Remember, a modern API gateway should empower you to build and deploy APIs that align with your unique infrastructure and business situation.

#3 Protecting Your APIs in a Hostile World

Security is non-negotiable in a world marked by relentless cyber threats and strict regulatory requirements. A modern API gateway must serve as the vanguard of your digital fortress, offering a comprehensive suite of security features to safeguard your data, services, and reputation.

Secure a wide range of protocols easily: Modern APIs leverage diverse protocols. A robust API gateway should secure these protocols uniformly at a centralized point. This includes common protocols like HTTP, TCP, UDP, WebSockets, and even gRPC. Centralizing security policies can streamline management and ensure consistent protection across your entire API ecosystem.
Authentication and authorization: The bedrock of any secure API ecosystem lies in robust authentication and authorization mechanisms to control who can access what. A modern API gateway should offer a robust arsenal of options, including industry standards like API keys, LDAP, JWT (JSON Web Tokens), OAuth2, OIDC (OpenID Connect), and HMAC (Hash-based Message Authentication Code). Implement the most suitable authentication mechanisms for each API, ensuring untrusted actors can’t misuse them.
Effective traffic management: This is essential for optimizing performance, enhancing resilience, and ensuring a superior user experience. A feature-rich API gateway facilitates advanced traffic management capabilities, including:
- Traffic mirroring—troubleshooting and security analysis
- Blue/green and canary deployments—safely roll out new API versions with minimal risk
- Session stickiness—maintain user sessions with specific upstream backends
- Active health checks—continuously monitor the health of backend services and ensure API uptime SLAs
- Circuit breakers—protect against cascading failures
- Retries and buffering—enhance API resilience towards temporary network issues
- Distributed in-flight request and rate limiting—evade denial-of-service attacks.
Built-in Let's Encrypt support: Securing communication channels with Transport Layer Security (TLS) is a non-negotiable security best practice. Let’s Encrypt is the most well-known ACME (Automated Certificate Management Environment) compatible TLS certificate authority. A modern API gateway should offer built-in ACME/Let's Encrypt support, simplifying the process of obtaining and managing TLS certificates. This aligns with the esteemed Electronic Frontier Foundation's (EFF) recommendation to leverage tools like Traefik that offer integrated support for certificate automation, moving away from external tools like Certbot.

Without such robust features, Leroy Merlin, a home improvement and gardening retailer, couldn’t have made it "effortless to manage" 50 clusters in parallel securely. With other API gateways, the team would expend time building and maintaining certificates with CertManager or other external tools requiring maintenance, expertise, and attention.

Beware of API gateways that offer a limited security feature set. While they might provide basic functionalities, advanced capabilities like fine-grained security policy enforcement might be missing. This can leave your APIs vulnerable to sophisticated attacks. You can also encounter API gateways that offer robust security features but are limited to a specific orchestration platform. You must not compromise on security and must be able to leverage best-of-breed security solutions across your entire infrastructure.

Remember, security is an ongoing process that requires constant adaptation. Stakes are high, and threats are widespread—avoid gateways that restrict customization and extensibility options to tailor security measures to your specific needs.

#4 Embrace Extensibility for a Future-Proof API Gateway

Meeting your current needs is nice, but a modern API gateway must also evolve alongside your organization's growth and changing requirements.

Imagine you're stuck in a walled garden. You can still move around, but it's hard to get out. Vendor lock-in is similar. The vendor creates a system where switching to another provider is inconvenient and costly, and you must move at the pace the vendor lets you by shipping new features according to their roadmap, not yours. For this reason, it’s best to avoid API gateways that lock you into a specific vendor's feature set. Extensibility empowers you to integrate best-of-breed solutions and customize your API gateway to meet your unique needs when needed.

A rich plugin ecosystem is a hallmark of a modern API gateway, amplifying its versatility. Built-in plugins can extend core functionalities, while the ability to create custom plugins allow you to address specific use cases. Security is a prime example. Custom plugins can be leveraged to fortify the gateway with advanced security features like a Web Application Firewall (WAF).
WebAssembly (Wasm) has emerged as a game-changer in extensibility. By supporting WASM-based plugins, an API gateway opens the door to a world of possibilities, enabling portable, lightweight, high-performance customization without sacrificing security or efficiency. This opens the door to a vast array of innovative plugins that can extend the gateway's capabilities in unprecedented ways.
Open-source development fosters innovation and rapid improvement. An API gateway and a plugin ecosystem built on open-source foundations driven by a passionate community ensures continuous feature enhancements and a future-proof platform. You can also contribute and receive feedback and improvements to your added functionality from the community.

Did you know that Traefik has recently joined OWASP, a well-known non-profit organization focused on improving software security, particularly web application security? OWASP is endorsing the use of the Coraza WAF, which is the successor to ModSecurity-based WAF solutions. Traefik v3 adds support for Coraza as a Wasm-based WAF plugin, created by a community member who is, in fact, the author of the Coraza WAF. What a great example of the opening three points working together!

A modern API gateway with WAF integration empowers you to leverage industry-leading protection mechanisms to safeguard your APIs against sophisticated attacks. This is made possible by the power of extensibility, allowing you to integrate best-in-class security solutions as they become available on the market, not by relying on the vendor's mercy to add them when they feel like it.

Vendor lock-in is also a barrier to observability. A modern API gateway should embrace OpenTelemetry, an industry standard for collecting and analyzing API metrics, traces, and logs. This empowers you to get comprehensive insights into your API ecosystem's health and performance not just from one vendor-specific tool but any compatible tool, even your existing monitoring tools if they support the standard.

What are the signs that your API gateway is failing in these areas?

Some gateways rely on Lua or C-based extensions. While these programming languages have their place, working with them can be cumbersome. Compiling the gateway from source code for even basic extensions adds unnecessary complexity and hinders development agility.
Also, beware of gateways that lock essential features behind a paywall, such as analytics, which should be core offerings in a modern API Gateway.
While WASM promises extensibility and allows you to leverage the language that best suits your development team's expertise, resource consumption remains a concern. A modern API gateway should balance extensibility with resource efficiency, ensuring smooth operation even with a robust plugin ecosystem.
Deep integrations can be convenient, but they can also lock you into a single vendor and hinder your ability to leverage best-of-breed solutions from other providers. Some gateways are so deeply intertwined with a specific cloud vendor's ecosystem or offered by the cloud provider itself that extensibility becomes an afterthought.

And last but not least, your needs might evolve as your API ecosystem matures. A modern API gateway should make migrating to a more comprehensive API Management solution smooth. Extensibility is key here, ensuring a seamless transition and minimal disruption to your existing API infrastructure.

#5 Taming Complexity with Declarative Magic

We've explored four out of the five essential features of a modern API gateway: simplicity, container orchestration integration, robust security, and extensibility. But how do you harness these capabilities and streamline your API development workflow? Enter GitOps, the unifying force that brings them all together.

GitOps embraces a declarative approach to configuration management. Instead of writing custom, complex scripts procedurally, developers simply define the desired state of the API gateway in a Git repository. They can then focus on configuring security settings, traffic management policies, plugins, and more through a declarative, code-based approach, leaving the underlying infrastructure management to GitOps automation.

This beautifully fits with our emphasis on developer experience. By leveraging Kubernetes CRDs stored in Git, GitOps eliminates the need to learn complex configuration languages specific to the API gateway. It also makes auditing changes a breeze, enhancing security and deploying changes easily without manual tinkering. All these translate to faster onboarding, fewer errors, and a more productive development environment.
Remember our focus on simplicity? GitOps delivers on this promise by enabling you to deploy your API gateway configuration to any environment using the same GitOps workflows. Imagine seamlessly deploying your API gateway configuration from development to staging and then to production just by using simple Git commits, automatically checking for mistakes in the process. This consistency minimizes errors and streamlines the deployment pipeline across your entire infrastructure.
GitOps also empowers you to leverage Git's power for security and incident management. Version control ensures a clear audit trail of all configuration changes, allowing you to easily roll back to a previous version in case of an issue. Additionally, Git branching strategies can be used to implement secure development workflows, such as pull requests for reviewing changes before deployment. This fosters a culture of security by design within your API development lifecycle.
Managing API versions has never been easier; GitOps excels at it. Each commit in your Git repository represents a specific API gateway configuration. Developers can experiment with new API versions in isolation before deploying them to production, minimizing risk and ensuring a smooth evolution of your API ecosystem.

For a deeper look at how GitOps empowers platform teams, check out this other article on the Traefik Labs blog: How Modern API Gateways Make DevOps Engineers More Efficient.

While GitOps offers a compelling approach to managing API gateways, some fail to deliver in this crucial area:

Avoid gateways that require additional tools, pretending they comply with GitOps-based workflows. A modern API gateway should natively embrace GitOps for declarative configuration management. However, some gateways require convoluted GitOps pipelines that become cumbersome to manage, especially when scaling to many APIs. This defeats the purpose of GitOps—it should simplify, not complicate.
Relying on an imperative (and not declarative) administrative API approach can also be inflexible and error-prone. Declarative GitOps empowers you to define the desired state, leaving the "how" to the automation tools. Rely on it for its clarity, version control benefits, and ease of rollback.
Modern gateways embrace configuration hot reloading, minimizing downtime during configuration updates, which can be frequent with GitOps. Avoid gateways that require full proxy reloads after every configuration change, losing active connections and hurting uptime.
And finally, if GitOps compatibility isn't a major focus for the API gateway vendor, it might not be the future-proof solution you seek.

Wrapping It Up

These days, applications often rely on many microservices, each with its own API, effectively increasing complexity to unprecedented levels. The API gateway has emerged as the cornerstone of this, now, often virtualized infrastructure. It is the gatekeeper between internal systems, external partners, and end-users, orchestrating the data flow and interactions with great finesse and efficiency.

But not just any API gateway will do. Only a modern solution equipped with the discussed 5 essential capabilities empowers you to build exceptional APIs, accelerate development lifecycles, and confidently navigate the ever-evolving landscape of application development.

Want to know more about choosing an API gateway in 2024? Download our free API gateway Buyer’s Guide by clicking the promo below for more details.

]]>

Fortify Your Frontlines: Distributed Security with HashiCorp Vault, Let's Encrypt, and Traefik

Immánuel Fodor — Tue, 28 May 2024 18:47:00 GMT

Distributed systems have become the backbone of the digital economy, enabling the seamless operation of everything from cloud services to mobile applications. These systems have specific requirements when it comes to encryption, which is provided by Transport Layer Security (TLS). With Traefik's API Gateway, TLS support can be handled by a combination of a Certificate Authority (CA) aligned to the ACME protocol (e.g., Let's Encrypt) and HashiCorp Vault, which collectively secure your distributed systems and provide key management while maintaining high availability.

In this article, we'll explore how these tools combine at a high level to provide encryption for distributed architectures and why that's important.

Note: For a deep dive into how to configure this in your environment using Traefik and HashiCorp Vault's PKI Engine, Nomad, and Consul, see this post by Open Source Consultant and Trainer Chris van Meer.

Why a Distributed System Requires TLS

A distributed system, at its core, is a collection of independent computers that appear to its users as a single coherent system. This architecture is designed to manage tasks that are too complex for a single computer by dividing them across multiple machines, thus leveraging the combined processing power, storage capacity, and specialized functionalities of all of them. The distributed nature of these systems offers several advantages, including scalability, fault tolerance, and resource sharing. However, these benefits also introduce unique challenges, particularly in the realm of security, that require the use of an encryption protocol such as TLS:

Decentralized Infrastructure and Data Transmission Across Networks: In a distributed system, components communicate over a network, potentially between different geographical locations. They exchange data that can range from sensitive user information to critical operational commands. This transmission occurs over channels that could be accessible to attackers, making the data susceptible to eavesdropping and interception. TLS secures these communications by encrypting the data, ensuring that even if data packets are captured, they cannot be deciphered by unauthorized parties.
Complex Interactions Among Components: Distributed systems often involve complex interactions between various applications, services, and databases, each possibly having its own security mechanisms and vulnerabilities. Consistently implementing TLS across all these interactions ensures a baseline level of security, providing a unified approach to protect against man-in-the-middle attacks and ensuring that data remains secure in transit between components.
Dynamic Scalability: One of the hallmarks of distributed systems is their ability to scale dynamically, adjusting resources in response to fluctuating demand. This scalability often involves the automatic deployment of new instances or services, which must immediately communicate securely with existing components. TLS certificates can be automatically managed and deployed to new instances, ensuring that they are immediately secured and can be trusted by other parts of the system.
Authentication: Beyond encrypting data, TLS also facilitates authentication through the use of certificates, enabling both servers and clients to verify each other's identity. This is particularly important in distributed systems where services may be interacting for the first time or where components are provided by different vendors.
Regulatory Compliance and Trust: Distributed systems, especially those handling personal data or sensitive information, must comply with a myriad of regulations regarding data protection and privacy, such as GDPR or HIPAA. TLS not only helps in complying with these regulations by securing data in transit but also bolsters trust among users and stakeholders by demonstrating a commitment to security.

In other words, the intrinsic characteristics of distributed systems—their networked, decentralized, and dynamic nature—significantly increase their vulnerability to security threats. TLS addresses these vulnerabilities by providing a robust mechanism for encryption, authentication, and data integrity across all communications. In other words, implementing TLS is not just about protecting data; it's about ensuring the resilience and reliability of the distributed system in the face of evolving cyber threats.

But in order to make this happen in an automated environment, we need a way to programmatically request and manage the TLS certificates that form the foundation of the protocol. Fortunately, we have Let’s Encrypt. Let's Encrypt is a non-profit CA that plays a pivotal role in securing distributed systems by offering free, easily managed TLS/SSL certificates. Its use of the ACME protocol automates certificate management, reducing the administrative burden and minimizing the risk of security lapses. By leveraging Let's Encrypt (or another ACME aligned certificate authority), organizations can ensure their components communicate securely, contributing to the safety and reliability of their services and the internet at large.

What is Vault and How is It Involved in This Process?

HashiCorp Vault stands out as a pivotal component in securing distributed systems, primarily due to its comprehensive approach to managing secrets, such as API keys, passwords, and TLS/SSL certificates. Its role in these systems extends beyond mere storage, encompassing secret management, data encryption, and access control, all of which are crucial for maintaining the integrity and confidentiality of communications within distributed architectures. Vault is involved in this process in several ways:

Centralized Secrets Management: In distributed systems, the need to securely manage a multitude of secrets across various services and environments is paramount. Vault centralizes the storage of these secrets, offering a single point of control and auditing. This centralization simplifies the secrets management process, reducing the risk of leaks or unauthorized access that could compromise the system's security.

Vault's dynamic secrets system is particularly beneficial for distributed systems. Unlike static secrets, which remain the same until manually changed, dynamic secrets are generated on-demand and are valid for a strictly limited duration. This means that even if a secret were to be exposed, its short lifespan significantly limits the potential for misuse. This feature is invaluable for TLS, where temporary credentials can be used for encrypted sessions, minimizing the risk associated with long-lived certificates.

Encryption as a Service: Vault provides encryption services, enabling applications to encrypt and decrypt data without managing encryption keys directly. This functionality supports the secure transmission of data across distributed systems by ensuring that sensitive information remains encrypted both at rest and in transit. When combined with TLS, which encrypts data during transmission, Vault's encryption services offer an additional layer of security for data stored within the system.
Automating Certificate Management: Vault integrates seamlessly with the TLS process by automating the management of TLS/SSL certificates. It can issue certificates directly using its internal CA, or it can act as an intermediary with external CAs, including those compatible with the ACME protocol, such as Let's Encrypt. This automation extends to the renewal and revocation of certificates, ensuring that distributed systems consistently use valid certificates without manual intervention.

Managing the lifecycle of TLS/SSL certificates is a critical aspect of maintaining secure communications. Vault automates the renewal process, issuing new certificates before the old ones expire, and can revoke certificates that are no longer needed or have been compromised. This capability is crucial for distributed systems, where outdated or revoked certificates can lead to security vulnerabilities or system outages.

In essence, Vault's comprehensive suite of features for managing secrets and certificates plays a vital role in the security and operational efficiency of distributed systems. By automating the creation and revocation of TLS/SSL certificates, Vault acts as a middleman that enables Traefik's API Gateway to renew certificates, and to ensure that secure, encrypted communication is a standard practice, not an afterthought. Its involvement in the TLS process not only bolsters security but also contributes to the reliability and resilience of the distributed system as a whole.

What is ACME and How Does It Help Secure Distributed Systems?

The Automated Certificate Management Environment (ACME) is a protocol designed to automate interactions between CAs and web servers, streamlining the process of obtaining, renewing, and revoking digital certificates. ACME has several goals:

Simplifying Certificate Issuance: ACME standardizes the process for verifying domain ownership and automating certificate issuance. Systems can programmatically request and receive certificates, ensuring that secure, encrypted communication channels can be established without manual intervention.
Automating Renewals and Revocation: Certificates have a limited validity period and need to be renewed regularly to maintain secure connections. ACME automates this renewal process, allowing systems to automatically request new certificates as expiration dates approach. Similarly, if a security breach occurs or a certificate is otherwise compromised, ACME can facilitate the rapid revocation of these certificates, helping to minimize potential security risks.
Domain Validation: ACME includes mechanisms for automated domain validation, a prerequisite for issuing a certificate. This process verifies that the requester has control over the domain for which the certificate is being requested. ACME automates this verification through several methods, such as placing a specific file in a predefined directory on the web server or making certain DNS changes. For distributed systems, this automation removes a significant barrier to securing communications, ensuring that validation and certificate issuance can occur seamlessly as new services are deployed or as systems scale.

Vault as an ACME Client

At the heart of the Vault and ACME integration is Vault's ability to function as an ACME client. This capability enables Vault to directly interact with ACME-compliant CAs, such as Let's Encrypt. Vault can automatically request, renew, and revoke TLS/SSL certificates based on predefined policies and the needs of the system it secures. This process is managed through Vault's interface, leveraging ACME to handle the operational details with the CA.

This integration provides centralized management of certificates: integrating Vault with ACME provides a centralized platform for managing all aspects of TLS/SSL certificates within a distributed system. Administrators can define policies within Vault that dictate how certificates are issued, renewed, and revoked. This centralized approach not only simplifies management but also provides a clear audit trail of certificate-related activities, enhancing security and compliance.

The integration of Vault and ACME transforms the management of TLS/SSL certificates from a complex, manual task into a streamlined, automated process. This integration not only enhances security and compliance but also simplifies the operational aspects of managing certificates in distributed systems, enabling organizations to focus on their core objectives while maintaining a strong security posture.

How Traefik’s API Gateway Optimizes Distributed Systems

In distributed systems, the API gateway serves as a critical intersection, managing incoming traffic to various application components securely and efficiently. Traefik's API Gateway’s design optimizes this process by minimizing latency and ensuring that connections are always directed to the optimal endpoints, thereby enhancing overall system performance and reliability.

In other words, Traefik's API Gateway simplifies the routing of client requests to the appropriate backend services, ensuring efficient load balancing, traffic management, and network resilience. Key features include:

Dynamic Configuration: Unlike traditional API gateways, Traefik automatically detects changes in service configurations within a cluster, adapting routes without requiring manual updates or restarts. This dynamic response is essential for environments where services are frequently scaled or updated.
Built-in Security with TLS Management: Traefik integrates seamlessly with Let’s Encrypt to automate TLS certificate generation and renewal, ensuring encrypted and secure communications without manual intervention. This automation is critical for maintaining continuous security compliance and protecting data integrity across services.
Middleware Customization: Traefik allows the use of various middlewares that can modify requests and responses, implement additional security checks, or manage access control, providing enhanced flexibility to meet diverse operational requirements.
Observability and Monitoring: With native support for various monitoring tools, Traefik can enable these tools to provide detailed insights into API traffic patterns and health metrics, facilitating proactive management and optimization of network resources.

Traefik's API Gateway stands at the forefront of network management solutions by providing an adaptive, secure, and efficient routing mechanism that supports the complex demands of modern distributed systems. Its ability to integrate advanced security protocols, coupled with dynamic configuration capabilities, makes it an indispensable tool for developers and enterprises aiming to optimize application delivery and performance.

Traefik vs Traditional API Gateway Considerations

While Traefik offers several enhancements over traditional API gateways like NGINX, there are challenges to consider:

Complexity in Initial Setup and Learning Curve: The dynamic and flexible nature of Traefik's configuration might present a steeper learning curve compared to others with a history predating the cloud-native era. Organizations might find the initial setup of Traefik to be a bit more complex due to its abstracted and automated mechanisms but the initial investment pays off in the long run. Traefik also offers easy to understand video courses to cover from the basics to advanced load balancing in the free Traefik Academy.
Dependency on External Services: Traefik’s effectiveness, especially in certificate management, often hinges on seamless integration with external services such as Let’s Encrypt. This can introduce a dependency that might affect gateway functionality in the unlikely event that these services go down, but this is the case with any other gateways that integrate with external services. However, more manual control over certificate management usually isn't worth the effort. Even the Electronic Frontier Foundation (EFF) argues that Traefik should replace certbot, an auxiliary certificate automation tool, as built-in certificate automation is the future.
Resource Usage: Traefik’s advanced features and continuous monitoring for changes can lead to higher resource usage than others with minimal static setups. This can be a challenge for environments where strict resource optimization is critical. However, there are always trade offs, and Traefik’s added capabilities and automation can bring many times more value to the table than the saved resource costs.
Mature Ecosystem and Community Support: Users of traditional API gateways might benefit from a longstanding user community and a wealth of knowledge accumulated online. Although there are many plugins that can extend traditional gateway functionality, these often require recompilation or are written in embedded simple languages. Traefik still has some way to go in growing its ecosystem and knowledge base, but it’s growing fast, and has recently added WebAssembly (Wasm) support for its plugin system, besides the existing Go-based solution, to make plugin development even simpler.

However, where Traefik's API Gateway shines, it really shines.

Automated certificate discovery: Navigating the complexities of TLS/SSL certificate management in distributed systems, particularly the issue of applying new or updated certificates without causing service downtime, is a formidable challenge. For example, traditional proxy servers often require restarting or reloading the proxy instances to apply these certificates, leading to potential availability issues and the disruption of existing sessions. By contrast, Traefik's API Gateway provides automatic discovery of new certificates, which offers a sophisticated solution, eliminating the need for manual intervention and significantly enhancing system resilience and uptime.
Plug-in Ecosystem: While still growing, Traefik’s plug-in ecosystem is tailored towards modern, dynamic environments and includes plug-ins for authentication, security enhancements, and traffic management tailored to the needs of microservices architectures.

Traefik's API Gateway’s ability to apply TLS/SSL certificate updates without requiring proxy restarts represents a significant leap forward in achieving high availability for distributed systems. This capability not only ensures continuous operation and enhanced security but also aligns with the operational demands and expectations of modern digital services. By adopting technologies and practices that support this capability, organizations can deliver more reliable, secure, and user-centric services.

Conclusion

In conclusion, the seamless integration of Vault and ACME with Traefik's API Gateway, coupled with innovative solutions like automatic certificate discovery, represents a significant advancement in managing TLS certificates for distributed systems. By addressing the challenges of certificate management and avoiding the need for proxy restarts, these technologies ensure that distributed systems can maintain high availability without compromising on security. This blend of security and availability is essential for the modern digital landscape, where the reliability and integrity of distributed systems are paramount.

]]>

Traefik 3.0 With SPIFFE, Tailscale, and HTTP/3

Emile Vauge — Thu, 23 May 2024 17:17:25 GMT

We're continuing our in-depth series on Traefik 3.0. If you missed it, be sure to read the previous articles on migrating from Traefik v2, WASM support with Coraza WAF, and Open Telemetry. Today, we'll be exploring state of the art technologies added to Traefik: SPIFFE, Tailscale, HTTP/3.

SPIFFE

Deploying dozens or hundreds of applications in production, in a distributed environment, comes with many challenges. One of them is network security, i.e. how to ensure application A is authorized to request application B? One of the historical solutions is to protect private networks with security controls like firewalls and VPNs. But many companies have adapted their security architectures to revolve around a concept known as Zero Trust networking, based on the paradigm that even private networks are untrusted. This means that applications need to authenticate themselves with other services.

In an effort to address this problem, the community created SPIFFE, the Secure Production Identity Framework For Everyone Project, which defines a framework and set of standards for identifying and securing communications between applications. The runtime counterpart, SPIRE, is a toolchain of APIs for establishing trust between applications.

Thanks to Julien Levesy, Traefik now supports SPIFFE mTLS with its backend servers! Here is a quick overview of the setup. First of all, you need to enable it in the static configuration:

## Static configuration
spiffe:
  workloadAPIAddr: localhost

The workloadAPIAddr configuration defines the address of the SPIFFE Workload API.
Now that SPIFFE is enabled globally, you need to configure your routes, at the ServersTransport level, for example, using SPIFFE trust domain (which makes Traefik allow any SpiffeID that comes from the given trust domain):

## Dynamic configuration
http:
  serversTransports:
    mytransport:
      spiffe:
        trustDomain: spiffe://trust-domain

Traefik is now able to connect to the Workload API to obtain an x509-SVID used to secure the connection with SPIFFE enabled backends 🎉. Yes, it’s as simple as that!

Tailscale

Tailscale is a zero-config VPN for building secure networks. If you are not familiar with this technology yet, I strongly recommend you to have a look, it is probably the simplest and most convenient way to set up servers in a VPN network. If you need to protect websites within a Tailscale network, Traefik can now request TLS certificates from the Tailscale API.

This new feature allows you to access HTTPS-enabled services on your tailnet behind Traefik Proxy without the hassle of managing certificates or exposing an endpoint for TLS challenges from Let’s Encrypt. Instead, Tailscale handles your certificate lifecycle, automatically renewing your Let’s Encrypt certificate, and then hands over to Traefik to handle the TLS secured requests.

Let’s see how to enable this feature. To obtain a TLS certificate from Tailscale, a Tailscale certificate resolver needs to be configured in the dynamic configuration:

certificatesResolvers:
  myresolver:
    tailscale: {}

Now all you have to do is reference myresolver from any router or entrypoint that is part of your tailnet. Here is an example using Docker labels:

## Dynamic configuration
labels:
  - traefik.http.routers.blog.rule=Host(`monitoring.yak-bebop.ts.net`) && Path(`/metrics`)
  - traefik.http.routers.blog.tls.certresolver=myresolver

Here is another example in a Kubernetes ingress resource:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: blogtls
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`monitoring.yak-bebop.ts.net`) && Path(`/metrics`)
      kind: Rule
      services:
        - name: blog
          port: 8080
  tls:
    certResolver: myresolver

Your apps are now able to access other apps securely within your tailnet, using Tailscale TLS certificates. Pretty straightforward right?

HTTP/3

HTTP is the backbone of the Internet. Since HTTP/2 was approved in 2015, significant progress has been made toward the next milestone: HTTP/3. This new version is a major shift, as it no longer relies on the traditional TCP, but instead uses UDP, leveraging QUIC—a transport protocol developed by Google in 2012. HTTP/3 offers numerous benefits, such as improved performance, faster connection establishment, and simpler encryption. Now, HTTP/3 is out of the experimental stage and fully production-ready in Traefik!

Let’s enable HTTP/3 on an entrypoint:

entryPoints:
  foo:
    http3: {}

Keep in mind that, as HTTP/3 actually uses UDP, when traefik is configured with a TCP entryPoint on port N with HTTP/3 enabled, the underlying HTTP/3 server that is started automatically listens on UDP port N too. As a consequence, it means port N cannot be used by another UDP entryPoint. Since HTTP/3 requires the use of TLS, only routers with TLS enabled will be usable with HTTP/3.

Wrapping Up

Simplicity and outstanding user experience has always been at the core of Traefik. Those new features packed in v3 follow strictly this principle. How straightforward it is to add Tailscale TLS certificates support, how elementary Traefik can be set up to retrieve SPIFFE x509-SVIDs, and how effortless you can migrate to the latest major revision of the Hypertext Transfer Protocol HTTP/3, all those added capabilities are great examples of the main core value of Traefik: keep it simple, yet state of the art.

The Traefik community has been instrumental in implementing those key features. We can't praise the outstanding work of Traefik contributors enough ❤️

Stay tuned for more deep dives on Traefik 3.0 key features!

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

Useful Links

]]>

Monitor Your Production at a Glance With Traefik 3.0 and OpenTelemetry

Nicolas Mengin — Thu, 16 May 2024 17:49:45 GMT

Today, we’re continuing our blog series on Traefik v3 (checkout our previous posts on migration path from Traefik v3 and the Wasm support with Coraza WAF plugin). This article will focus on how to leverage OpenTelemetry in Traefik v3.

Monitoring and observability have always been a priority for Traefik, consistently following industry standards such as OpenTracing integration starting with Traefik v2. That's why incorporating OpenTelemetry, the latest emerging standard, into our toolkit was an easy decision.

How Observability Ensures Infrastructure Robustness

In the microservices era, monitoring and observability—with rich metrics, tracing, and logs—are fundamental necessities in order to maintain the reliability and efficiency of our systems.

Tracing allows you to track the flow of operations within your system. Using traces, and spans, you can identify performance bottlenecks and pinpoint applications causing slowdowns, so you can optimize response times effectively.

Logs provide real-time insight into the health of your system. It enables swift error detection and intervention through alerts. By centralizing logs, you streamline the debugging process during incident resolution (say goodbye to grep).

Metrics offer a comprehensive view of your infrastructure's health. They allow you to monitor critical indicators like incoming traffic volume. For instance, Traefik enables you to dynamically scale your infrastructure based on observed traffic patterns. Additionally, metrics graphs and visualizations are helpful during incident triage to understand the causes and implement proactive measures.

OpenTelemetry: One Standard to Rule Them All

There are many tools in the market from a wide variety of vendors. Sifting through all the choices can be difficult, and picking the wrong one can be a huge pain as switching later is extremely cumbersome.

Part of what makes it difficult in deciding is because each vendor does things differently. Some scrape data, some pull it. Some are cloud-based, whereas others are on-premises. Switching between them is complex and time-consuming, if even feasible at all.

That's where OpenTelemetry comes in. It's a fusion of two older standards, OpenTracing and OpenCensus. OpenTracing focused on tracing, while OpenCensus focused on metrics and tracing. OpenTelemetry brings them together and adds logging specifications.

OpenTelemetry is an open-source project that belongs to the CNCF since 2019. Unlike with OpenTracing and OpenCensus, OpenTelemetry provides many implementations in different languages, like a Go SDK for example.

Open Telemetry makes life easier for everyone. For vendors and data providers like Traefik, having one standard obviously makes it simpler to support. For users, it ensures that the observability APIs will stay consistent regardless of the tools chosen.

OpenTelemetry in Traefik

While Traefik v2 supports many metrics and tracing vendors, it doesn’t support every solution in the market.

Traefik v2 observability solutions support

Now, with Traefik v3, we've made it possible to export metrics and traces in the OpenTelemetry format (logs support will be available as soon as the Go SDK is ready).

This allows us to ensure we support every OpenTelemetry consumer with no further implementation changes, making Traefik easy to integrate into your infrastructure.

To make sure we're fully compliant with the OpenTelemetry standard, Traefik adheres to the specified semantic conventions. These conventions tell us what data to provide based on what our software does in your setup.

Since Traefik Proxy is a reverse proxy, we've included metrics and spans information as expected in the OpenTelemetry framework.

Now, let's take a look at how easy it is to set up OpenTelemetry in Traefik.

How to install OpenTelemetry with Traefik v3?

Let's walk through the straightforward steps to install OpenTelemetry with Traefik.

OpenTelemetry Architecture

First off, it's worth noting there are two main ways to integrate Traefik (or any OpenTelemetry data provider) with an OpenTelemetry data consumer:

Vendor Direct Access: Traefik can directly connect with a consumer-like Prometheus or Jaeger to send OpenTelemetry info. It's typically used for development purposes.
OpenTelemetry Collector: This software acts as a middleman, importing data from Traefik, potentially transforming it, and then exporting it to the consumer. This approach offers centralized management of OpenTelemetry data, and scalability, and is the recommended method for handling OpenTelemetry information in large-scale applications.

Traefik v3 Configuration

Regardless of which method you choose, the cool thing is that the configuration stays the same.

First, you deploy the OpenTelemetry collector (or the solution you need) as well as Traefik on your cluster.

Then, you just set up the collector endpoints, and you're set. Whether you're adding a collector or switching up the consumer, all you have to do is tweak the address in the configuration as described below (check the documentation for more options):

## Traefik v3 ##
################

# Send metrics data to a Prometheus instance 
metrics:
  otlp:
    http:
      endpoint: http://prometheus:9090/api/v1/otlp/v1/metrics

# Send tracing data to a Jaeger instance 
tracing:
  otlp:
    http:
      endpoint: http://jaeger:4318/v1/traces

And that's it! Simple, right?

Traefik v2 Migration

While this setup works smoothly for users starting fresh with Traefik v3, what about those looking to migrate from Traefik v2?

Concerning the metrics, the good news is you can migrate seamlessly—i.e., the vendor-specific integration still works. But do not hesitate to migrate to OpenTelemetry when you’re ready for it!

However, if you've been using Traefik v2 for tracing, it's important to note that you may need to migrate to OpenTelemetry in Traefik v3. Maintaining both vendor-specific implementations and OpenTelemetry wasn't feasible, hence the decision to break backward compatibility.

Conclusion

As you have seen, integrating Traefik with OpenTelemetry couldn't be easier.

Sure, OpenTelemetry is relatively new and still evolving (with ongoing work on logs integration, expanding semantic conventions, and more), but it's promising and undoubtedly the best standard for monitoring and observability.

So, why wait? Dive in, give it a try, and don't forget to share your feedback to aid in refining our integration.

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

See you on GitHub.

]]>

Traefik 3.0: Deep Dive Into Wasm Support With Coraza WAF Plugin

Emile Vauge — Thu, 09 May 2024 14:34:27 GMT

We’re continuing our deep dive series on Traefik 3.0 which was released a week ago. Make sure to check out last week’s article on the migration path from Traefik v2. Today, we will deep dive into WebAssembly support.

Custom plugins for Traefik are one of the most requested features going back to the early days of the project, starting with this issue, from back in 2017:

The author suggested using the brand new plugin feature introduced in go 1.8. The idea was to compile go code into dynamic libraries which could be loaded and executed at runtime. It looked very promising and we immediately started working on this. Sadly, some severe limitations were found and this pursuit was put on hold. A second attempt was made a few months later, but was eventually abandoned due to the incomplete plugin implementation provided by the go team (only on Linux with CGO_ENABLED).

After many discussions, we ended up with a different solution. If the go language ecosystem wouldn’t provide any solid solutions for building plugins, let’s implement our own (crazy, I know) fully compliant Go interpreter. In 2019, Yaegi—Yet Another Elegant Go Interpreter—was released. The project quickly got a lot of traction being the best Go interpreter, and middleware plugins became a reality within Traefik soon after. Yaegi enabled many to develop middlewares or providers for their context. To this day, we can count more than a hundred middleware plugins made available through the catalog, plus many more that are kept private.

However, Yaegi only provides the option to build plugins using go, and this can be a bit restrictive for some people. This is the reason why some community members started to work on the brand new plugin engine for Traefik in 2023: Web Assembly, abbreviated WASM.

WASM Overview

WebAssembly is a low-level assembly-like language with a compact binary format that runs with near-native performance. It provides languages such as C/C++, Go or Rust (and many more) with a compilation target so that they can run on the web, and in other runtimes thanks to WASI, the WebAssembly System Interface.

Announced in 2015, WASM evolved into an open standard and got support in all major browsers in 2017.

In a nutshell, WASM allows you to compile almost any language in a portable binary format that can run on any platform, with native performance.

It looks like we found the perfect plugin engine for Traefik 🙂

Traefik + WASM

The idea to bring WASM support to Traefik was appealing. However, many challenges had to be tackled. The first one was to write an interface between Traefik, the host, and WASM plugins, the guests. This is called an Application Binary Interface (ABI) and in the case of Traefik middleware plugins, we had to provide an interface to the net/http HTTP server library. Writing and maintaining a fully compliant ABI for the net/http library is not an easy task but luckily, talented developers had already released different options, two of which stood out: proxy-wasm and http-wasm. After many discussions with community members, the clear winner was http-wasm, thanks to its much simpler and straightforward integration within a go codebase.

Then everything accelerated. A community contributor—Jesse Haka—opened a pull request, and a month later, after careful reviews from maintainers and external contributors like José Carlos Chávez, Traefik had support for WASM middleware plugins!

Let’s see how to write a WASM plugin for Traefik that customizes the routing pipeline that handles requests and responses. The Traefik routing pipeline basically splits into 3 main components: entrypoints, routers and services. Entrypoints define the network entry points into Traefik. Routers are in charge of connecting incoming requests to the services that can handle them. In the process, routers may use a chain of middleware to do some pre- and/or post-processing of the request. Our goal here is to write a custom middleware that will be added to the chain in a Traefik router. To simplify, we will use go as the plugin langage, but we could also use C or Rust, as soon as the code implements the http-wasm interface.

Ultimately, the only thing you have to do is implement 1 function: handleRequest which handles the pre-processing or handleResponse, the post-processing. You can get a simple example of a WASM plugin in the repository. Here is boilerplate:

package main

import (
	"encoding/json"
	"fmt"
	"os"

	"github.com/http-wasm/http-wasm-guest-tinygo/handler"
	"github.com/http-wasm/http-wasm-guest-tinygo/handler/api"
)

// Config the plugin configuration.
type Config struct {
	Headers map[string]string `json:"headers,omitempty"`
}

func main() {
	var config Config
	err := json.Unmarshal(handler.Host.GetConfig(), &config)
	if err != nil {
		handler.Host.Log(api.LogLevelError, fmt.Sprintf("Could not load config %v", err))
		os.Exit(1)
	}

	mw, err := New(config)
	if err != nil {
		handler.Host.Log(api.LogLevelError, fmt.Sprintf("Could not load config %v", err))
		os.Exit(1)
	}
	handler.HandleRequestFn = mw.handleRequest
}

// Demo a Demo plugin.
type Demo struct{}

// New created a new Demo plugin.
func New(config Config) (*Demo, error) {
	return &Demo{}, nil
}

func (d Demo) handleRequest(req api.Request, resp api.Response) (next bool, reqCtx uint32) {
	return true, 0
}

Then load your compiled plugin into the static configuration:

# Static configuration
experimental:
  localPlugins:
    example:
      moduleName: github.com/traefik/plugindemowasm

```

And finally, customize a router with your new plugin:

# Dynamic configuration
http:
  routers:
    my-router:
      rule: host(`demo.localhost`)
      service: service-foo
      entryPoints:
        - web
      middlewares:
        - my-plugin

  services:
   service-foo:
      loadBalancer:
        servers:
          - url: http://127.0.0.1:5000
  
  middlewares:
    my-plugin:
      plugin:
        example:
          headers:
            Foo: Bar
```

As you can see, this is pretty straightforward to customize the routing pipeline in Traefik thanks to WASM plugins. Let’s tackle a production use case. How about embedding a Web Application Firewall into the middleware chain?

Traefik + Coraza

Traefik is a critical piece of many companies’ infrastructure and is playing a great role in securing connectivity with applications and APIs. It's impossible for any single security measure to cover every potential attack angle. Security operates in layers, each with its own focus, sometimes with areas of overlap. A global approach, acknowledging the diverse threat landscape, is the way to go. One of the main links in the security chain are Web Application Firewalls. They look for malicious and unwanted content within incoming requests and mitigate those potential threats. Having a WAF integrated to Traefik as a plugin is clearly a game changer and adds a first line protection layer to your infrastructure.

Coraza is an open source, go based, high performance, Web Application Firewall. It’s an OWASP project, understands Modsecurity’s seclang language, and it’s able to enforce OWASP core rule sets.

Of course, when Coraza’s leaders proposed to provide an integration of Coraza in Traefik within a WASM plugin, we were all pretty excited. José Carlos Chávez quickly came up with a working solution now available on the Traefik Plugins Catalog!

First of all, let’s load the Coraza plugin into the static configuration:

# This config was generated by "mage updateVersion". DO NOT EDIT.

entryPoints:
  web:
    address: :80

providers:
  file:
    filename: /etc/traefik/config-dynamic.yaml

experimental:
  plugins:
    coraza:
      moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
      version: v0.2.1

Let’s update the middleware section for a router in the dynamic configuration:

http:
# ...
  middlewares:
    waf:
      plugin:
        coraza:
          directives:
            - SecRuleEngine On
            - SecDebugLog /dev/stdout
            - SecDebugLogLevel 9
            - SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"
```

Then curl -I 'http://localhost:8080/admin' will return a 403 error as specified by the configuration rule SecRule REQUEST_URI "@streq /admin.
curl -I 'http://localhost:8080/anything' will return a 200 as there is no matching rule.

A more advanced example can be found here, where we attempt to use the log4shell attack on a vulnerable application. Log4shell is a zero-day vulnerability in Log4j involving arbitrary code execution. This attack can be mitigated by simply enabling CRS rule 932130 to look into REQUEST_HEADERS:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: config
  namespace: traefik
data:
  config.yaml: |
    http:
      middlewares:
        waf:
          plugin:
            coraza:
              directives:
                - Include @recommended-conf
                - Include @crs-setup-conf
                - Include @owasp_crs/*.conf
                - SecRuleUpdateTargetById 932130 "REQUEST_HEADERS"
                - SecRuleEngine On
EOF

As you can see, the WASM plugin feature in Traefik provides a state-of-the-art extension technology and covers the most advanced use cases.

Takeaways

Traefik already had an extension engine based on the Yaegi Go interpreter. It is extremely easy to set up and provides a powerful foundation for customizing Traefik. However, it requires writing plugins in Go, and some people could find this too restrictive.

WebAssembly has gained huge popularity lately and provides exactly what is needed to build a perfect plugin engine: portable binaries, open standard, fast, multiple languages support, and multiple host environments. Traefik 3.0 adds full support of WASM plugins for middlewares and allows to run binary code compiled from many different languages like C, Rust, or JavaScript.

Finally, to show the full potential of this new plugin engine, we have integrated the powerful Web Application Firewall Coraza, as a WASM plugin. This will bring new critical capabilities to Traefik, adding an additional layer of security to your environment.

The Traefik community has played a central role in brainstorming and implementing those major features. We can’t say enough good things about the exceptional work made by Traefik contributors ❤️

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

Stay tuned for more deep dives on Traefik 3.0 key features!

]]>

Traefik 3.0 GA Has Landed: Here's How to Migrate

Emile Vauge — Tue, 30 Apr 2024 12:43:09 GMT

This year will mark the 9th anniversary of Traefik. Back in 2015, while I was writing the first lines of code for a cloud native reverse proxy, I couldn’t imagine how big this would go. From a side project, Traefik became one of the most deployed modern gateways on earth, with more than 3 billion downloads and 750+ contributors. It is ranked in the Top 15 on DockerHub and has 47,000 GitHub stars. Insane.

Traefik 1.0 was released in 2016. Three years later, Traefik 2.0 was born. Today, after 5 years of development, the wait is finally over! We are proud to announce that Traefik 3.0 is generally available 🎉

This major release is a huge step forward in the cloud native world, adding support for the latest technologies like WASM, OpenTelemetry, Kubernetes Gateway API, and SPIFFE.

After many years of effort, the changelog is pretty wild, with more than 200 pull requests merged, each coming with new features. The list of new possibilities is so big that we want to provide a series of blog posts, each deep diving into one feature.

Today’s article will focus on the beginning, the elephant in the room 🥁🥁🥁: the migration.

Migration Under Control

A new major version is always something eagerly awaited: new design, new features, better user experience … But the counterpart is usually the migration aspect. A major version often means breaking changes, but that shouldn’t imply a painful migration experience, and with Traefik v3 it doesn’t.

With Traefik v3, we are introducing a streamlined transition process from v2. As a reminder, Traefik has 2 kinds of configurations:

the static configuration which is loaded when Traefik starts up and manages global options
the dynamic configuration can be updated while Traefik is running and contains all the routing information

Minimal breaking changes have been made on specific options on the static configuration, and we are ensuring backward compatibility with v2 syntax on the dynamic configuration. This will offer a gradual path for adopting the v3 syntax, allowing users to progressively migrate their Kubernetes ingress resources, Docker labels, etc. to the new format.

Let’s deep dive into the migration path with a typical use case: you have to migrate 100 Traefik instances from v2.11 to v3.0 in Kubernetes.

Step 1: Prepare and Test

The first thing to do is to identify how your static configuration is impacted by the changes made in v3. The breaking changes are minimal and target very specific options, in 90% of use cases, this process should take a couple minutes only. As an example, Docker and Swarm are now 2 providers, HTTP/3 is no longer an experimental option, Rancher v1 has been dropped as the project is no longer being actively maintained, etc. Please refer to the static configuration section of the migration documentation, as well as the operations section, to get the full list and prepare your new v3 static configuration.

Add the following snippet to your static configuration, this will use by default the v2 syntax for your routing rules:

# static configuration
core:
  defaultRuleSyntax: v2

When you are ready to test it, start Traefik v3 with this new configuration. If you don’t get any error log, you are good to go! Otherwise, no problem, the remaining migration options are highlighted in the logs, you can simply apply them.

Once your Traefik test instances are starting and routing to your applications, you can move to the next step.

Step 2: Rolling Update

Now that you have tested your updated static configuration, it is time to progressively migrate your production instances to Traefik v3. Use the Kubernetes rolling update mechanism to incrementally replace the current Pods with new ones.

⚠️ Before triggering any change on your production, ensure you to have a working real time monitoring solution on your ingress traffic that will allow you to instantly detect any issue. Traefik provides support for many observability solutions.

While the rolling update is in progress, constantly monitor your ingress traffic looking for unexpected (and unusual) errors, and be prepared to rollback to a working state. Then, leverage debug and access logs provided by Traefik to understand and fix the issue before upgrading again. In case you're not sure, jump onto the community forum.

Once every pod is updated, congratulations, you are now on Traefik v3!

You can move to the last step.

Step 3: Progressive Ingresses Update

Now that you run Traefik v3, start migrating your ingress resources to the new format.

📍 Keep in mind that this step can be done later, as Traefik v3 is compatible with the v2 format for dynamic configuration. But of course, you can start using new features right away in new ingresses, and migrate older ingresses later.

The dynamic configuration in v3 has undergone a few changes. For example, the Router Rule Matchers have an updated syntax, the Kubernetes Ingress API Group has been changed, and the TCP LoadBalancer terminationDelay option has been removed. The full list can be found in the dynamic configuration section of the migration documentation.

Progressively, switch each router to the v3 syntax, test & update each ingress resource and check that the ingress traffic is not impacted. Once a v3 ingress resource migration is validated, you can delete the v2 ingress resource and deploy the v3 version. Repeat those steps for every ingress resource.

At the end of the process, you can safely remove the snippet added at step 1:

# static configuration
core:
  defaultRuleSyntax: v2

Et voilà!

You are now fully migrated to Traefik v3 🎉! And you did this progressively, keeping control during the whole process, with the option to rollback any change at any time.

This example with Kubernetes can be transposed to any orchestrator or environment, the process stays the same.

In the coming weeks, we will continue this series, and release deep dive articles on WASM support (and Coraza Web Application Firewall plugin), OpenTelemetry, SPIFFE/Tailscale/HTTP/3, and Kubernetes Gateway API.

Ultimately, I'd like to express immense gratitude to all contributors. Your support is incredibly valuable.

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

See you on GitHub!

Useful Links

Traefik 3.0 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Level Up API Operations: Granular API Access Control Meets GitOps

Immánuel Fodor — Thu, 18 Apr 2024 19:47:17 GMT

We're thrilled to announce the release of a powerful new security feature for managing APIs with Traefik Hub: granular API access control with operation filters. This feature empowers you to control who can access your APIs and what they can do with them with unprecedented precision.

Modern applications rely heavily on APIs, as evidenced by the Cloudflare 2024 API Security report, which found that APIs comprise over half (57%) of internet traffic. As the building blocks of digital infrastructure, APIs play a pivotal role in organizational success. However, securing these critical resources can be challenging. Traditionally, access control options have been limited to all-or-nothing approaches, leaving organizations vulnerable to security risks and compliance gaps.

With Traefik Hub's new granular API access control, you can finally move beyond this restrictive approach. You now have the fine-grained control you need to secure your APIs at the HTTP method and URI path level, ensuring only authorized users can access specific functionalities within your APIs.

The Growing Challenge of Securing APIs in Modern Applications

The expanding landscape of APIs presents a significant challenge for organizations seeking to maintain robust security measures. According to Cloudflare, nearly 60% of organizations grant "write" access to at least half of their APIs, and the prevalence of shadow and zombie APIs further complicates the picture:

Shadow APIs, essentially unmanaged and unsecured application programming interfaces, pose a severe threat due to their inherent lack of oversight and potential access to sensitive data. Cloudflare’s machine learning algorithms found ~33% more API endpoints vs. what customers self-reported. These rogue APIs can lead to data breaches and expose critical information when compromised.
Zombie APIs were once active but have been deprecated or abandoned. They may already be managed but not used actively. Likely, these APIs are not well secured or left unsecured, so they impose an ever-growing risk of becoming compromised or being used to carry out malicious activities.

Additionally, the rise of generative AI introduces new security concerns. While AI holds great promise, its integration also brings potential vulnerabilities, as both AI models and AI-generated code can be susceptible to attack. Forrester predicts that by 2024, a lack of proper security measures could lead to several high-profile data breaches originating from AI-generated code. However, they also believe that up to 90% of data breaches will include a human element.

Furthermore, APIs used for business to business (B2B) communication, while often perceived as more trustworthy due to their authenticated nature, can make organizations fall into a false sense of security. Because these APIs typically serve a smaller, known set of partners, security teams might underestimate the potential for abuse. However, malicious actors can exploit APIs in unforeseen ways, accessing confidential data and restricted functionalities or compromising previously trusted partners, infiltrating their systems to search for more vulnerable APIs.

Finally, the OWASP Top 10 API Security Risks reveal a worrying trend, with over half of the listed points focusing on API consumption and authentication flaws. These vulnerabilities—such as broken authentication, unrestricted resource consumption, and improper access controls—can be exploited by attackers to gain unauthorized access to sensitive data and functionalities.

Granular API Access Control to the Rescue

Traefik Hub now empowers you to move beyond the limitations of traditional, all-or-nothing API access control. Our new granular API access control feature allows you to precisely define who can access your APIs and what actions they are authorized to perform on a per-endpoint basis. This fine-grained control, achieved through HTTP method and URI path filtering, ensures that only authorized users can access specific functionalities within your APIs, significantly enhancing your overall security posture.

We've taken a modern and efficient approach, leveraging the power of GitOps and Kubernetes Custom Resource Definitions (CRDs). GitOps, a popular methodology for infrastructure automation, allows you to manage your access control rules in Git repositories, ensuring version control, collaboration, and easy rollbacks. CRDs, on the other hand, provide a Kubernetes-native way to define custom resources specific to your needs, such as API access control rules. This combination offers several benefits:

Easy to read and understand: CRDs are written in a human-readable format, making them easier to understand and manage than complex configuration files.
Validation and consistency: CRD validity is enforced through specific schema validation, ensuring the consistency and integrity of your access control rules.
Version control and auditability: Git provides built-in version control capabilities, allowing you to track changes to your access control rules and easily roll back to previous versions if needed. Additionally, Git enables comprehensive auditability of access control policy changes.
Automated policy application: By leveraging Kubernetes-native labels and selectors, Traefik Hub can automatically apply your API access control policies to newly published APIs based on defined criteria, streamlining the security configuration process.

Key Features of Traefik Hub's Uniquely Flexible API Security

The flexibility of Traefik Hub's granular API access control sets its apart from similar solutions. Here are the key features and functionalities that define our unique approach:

Support for all standard HTTP methods: Our solution empowers you to define access control rules for all commonly used HTTP methods, including GET, POST, PUT, DELETE, PATCH, and more. This fine-grained control allows you to grant or deny access based on the specific actions users attempt to perform on your APIs.
Versatile URI path definition: You can define URI paths with various options, including:
- Exact paths: Grant or deny access to specific API endpoints
- Path prefixes: Control access to a group of related endpoints sharing a common prefix
- Regular expressions: Define complex patterns for matching multiple URI paths under a single rule, simplifying access control management.
Powerful rule combinations: Go beyond simple allow/deny rules and leverage AND/OR logic to create sophisticated access control scenarios. This allows you to combine path and method restrictions to define precisely who can access what and under what conditions.
Simplified rule management with operation sets: Introduce reusable sets of path and method combinations, called "operation sets," to streamline rule creation and management. This approach promotes code organization and reduces redundancy, especially when defining access control rules for multiple APIs with similar functionalities.
Granular control at both API and API version levels: Enforce access control not only at the API level but also at the individual API version level. This enables you to adapt access policies based on specific functionalities or security requirements introduced in different API versions.

Business Benefits: Efficiently Elevate Your API Security

By utilizing Traefik Hub’s novel operation filtering, you can unlock a plethora of benefits for your organization:

Craft highly customized rules tailored to your specific API security needs with streamlined access control policy management leveraging GitOps and CRDs for efficient access control rule creation, version control, and auditability.
Mitigate security risks and limit data exposure by granting access only to authorized users and actions on your APIs. Implement zero trust principles by restricting API access based on specific needs, minimizing the potential for data breaches and unauthorized exposure.
Improve developer experience by enabling them to easily understand and manage access control policies, thanks to clear and concise rules.
Facilitate adherence to industry regulations and compliance requirements. Many industries have regulations governing data access and security. Granular access control simplifies compliance by enabling you to demonstrate control over who can access your APIs and the specific data they can access.
Improve governance by enforcing consistent access control policies across your API landscape. Adhering to the same security standards can reduce confusion and potential vulnerabilities.
Keep your agility! Quickly define and modify access rules, and reuse existing access policies across APIs and API versions.

Traefik Hub supports the growing complexity of your API and access control needs. As your API landscape evolves, you can manage the increasing complexity of access requirements without sacrificing security or efficiency.

In the next section, we'll showcase example scenarios demonstrating how granular API access control can be applied to address specific security challenges.

Example: Protecting Sensitive Data in a Healthcare API

Imagine a healthcare provider utilizing an API to manage patient data. This API likely contains sensitive information, such as medical records, prescriptions, and insurance details. While specific doctors and nurses require access to this data to provide care, it's crucial to restrict access to unauthorized individuals.

Traditionally, without granular access control, the provider might have only two options:

Grant all-or-nothing access: This approach exposes all data to all authorized users, posing significant security risks if unauthorized access occurs
Implementing complex custom business logic: This can be time-consuming to develop and maintain, especially as the API and its users evolve.

With Traefik Hub's granular API access control, the provider can achieve precise control by defining fine-grained API access rules:

Limit access to specific patient data API endpoints with URI path rules based on user roles. For example, doctors can access detailed medical records, while nurses can only view basic information.
Restrict access to specific HTTP methods. For example, allow only GET requests for read-only access and disallow modification methods like PUT or DELETE for nurses.
Mix and match these to create more advanced rules.

Let’s see the following configuration example we want to express for an Admin API that is versioned with two versions and an unversioned Patient API.

Rule Placement	Group	URI Path	HTTP Method
API Version	admin	/v1/admin (prefix)	*
API Version	admin-beta	/v1/admin (prefix) /v2/admin (prefix)	*
API	nurse	/patients (strict) /patients/[0-9]+$ (regex)	GET GET
API	doctor	/patients (prefix) /.* (regex)	GET, POST, PUT, DELETE GET

The Admin API has two versions defined using URL path-based versioning:
- The regular admin group can access everything on the current 1st version.
- The beta tester admin group can access everything on both the current and the upcoming v2 version.
The patient API exposes a /patients endpoint with a sub-path /patients/:id to access basic info of specific patients and a /patients/:id/details endpoint to get all the details. All of these support modifying and deleting the available data and creating new records.
- Nurses can only view basic information. They can list patients with /patients using strict URI path matching to limit access only to the list endpoint and can access basic info with /patients/:id.
- However, doctors can access full medical records, meaning everything under /patients like /patients/:id and /patients/:id/details using URI path prefix matching.
On the other hand, besides the Patient API, doctors can access all Admin API versions with read-only permissions to retrieve basic information about the hospital administration.

Let’s see the examples expressed as Traefik Hub CRDs. The relevant additions of granular API access control are highlighted.


---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: admin-api
  namespace: apps
  labels:
    area: health
spec:
  pathPrefix: /admin
  currentVersion: admin-api-v1
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: admin-api-v1
  namespace: apps
  labels:
    area: health
spec:
  apiName: admin-api
  release: v1.0.0
  title: "Title of Admin API v1"
  routes:
    - pathPrefix: /v1
  stripPathPrefix: true
  service:
    name: admin-service-v1
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: admin-ops-v1
          matchers:
            - path: /admin
        - name: read-all
          matchers:
            - methods:
                - GET
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: admin-api-v2
  namespace: apps
  labels:
    area: health
spec:
  apiName: admin-api
  release: v2.0.0
  title: "Title of the beta Admin API v2"
  routes:
    - pathPrefix: /v2
  stripPathPrefix: true
  service:
    name: admin-service-v2
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: admin-ops-v2
          matchers:
            - path: /admin
        - name: read-all
          matchers:
            - methods:
                - GET
---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: patient-api
  namespace: apps
  labels:
    area: health
spec:
  pathPrefix: /patients
  service:
    name: patient-service
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: nurse-ops
          matchers:
            - path: /patients
              methods:
                - GET
            - pathRegex: "/patients/[0-9]+$"
              methods:
                - GET
        - name: doctor-ops
          matchers:
            - pathPrefix: /patients
              methods:
                - GET
                - POST
                - PUT
                - DELETE
        - name: read-all
          matchers:
            - methods:
                - GET

First, we define the Admin API and its two versions with operation sets specific to that version. Note that you don’t need to add the version matcher to the operation set, it will automatically apply—i.e., we only define /admin, and depending on the version, it will apply to /v1/admin and /v2/admin. There is also an operation set allowing read access on both versions, using the same operation set name to be reusable across even multiple APIs when we give access to users. On the Patient API, we define the nurse and doctor operation sets with explicit rules as to who can access what. The generic read-all set is also present here, extending to multiple APIs.

Then we define our API Access policies to match the user groups with the APIs, API Versions, and filter for the operation sets.


---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: admin-access
  labels:
    area: health
spec:
  groups:
 - admin                             # Note the admin group
  apis:
    - name: admin-api                # Has Access to the Admin API
      namespace: apps
  operationFilter:
    include:
      - admin-ops-v1                 # With the v1 operation set
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: admin-beta-access
  labels:
    area: health
spec:
  groups:
    - admin-beta                     # Note the beta tester admin group
  apis:
    - name: admin-api                # Has access to the Admin API
      namespace: apps
  operationFilter:
    include:
      - admin-ops-v1                 # With the v1 operation set as regular admins
      - admin-ops-v2                 # And the new beta features of v2
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: nurse-access
  labels:
    area: health
spec:
  groups:
    - nurse                          # Note the nurse group
  apis:
    - name: patient-api              # Has access to the Patient API
      namespace: apps
  operationFilter:
    include:
      - nurse-ops                    # With the restricted operations allowed to
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: doctor-access
  labels:
    area: health
spec:
  groups:
    - doctor                         # Note the doctor group
  apiSelector:
    matchLabels:
      area: health                   # Can access APIs (current+future) with this label
  operationFilter:
    include:
      - doctor-ops                   # But restricted to operations defined for doctors
      - read-all                     # and read-all operations over multiple APIs

What stands out here is that the beta admin group can access two operation sets across the two API versions. In contrast, the regular admin group will only be granted access to the first version of the API. Nurses can only access the nurse operations, that’s simple. However, the policy for doctors is defined using a Kubernetes-native label selector to select all APIs that match the health label. This way, the filter for the doctor-ops operation will apply to the Patient API, and the read-all operation filter will be used throughout the two API versions of the Admin API. Since the read-all operation is also present on the Patient API, but the doctor-ops provide extended access to doctors, the broader rule overrides it.

When the Admin API v2 becomes stable, we could deprecate the v1 version to have only read-only access by modifying the related operation set and the regular admins’ access policy. Showing only the changes below:


# Admin API v1
operationSets:
  - name: admin-ops-v1
    matchers:
      - path: /admin
        Methods:                    # New line
          - GET                     # New line

# Regular Admin Access
operationFilter:
  include:
    - admin-ops-v1
    - admin-ops-v2                  # New line

In case a new API is added later with the same health label, that API will not grant automatic access to doctors to prevent data leaks, even though the label selector applies. Doctors can only gain access if the doctor-ops is also defined on that new API or if it defines some other operations that will be explicitly added to the doctors’ API access policy.

We hope these examples shed light on the flexibility of Traefik Hub’s new granular API access control solution with operation sets and filters.

Conclusion: Take Control of Your API Security

In today's API-driven landscape, robust security measures are essential. Traditional, all-or-nothing access control approaches leave organizations vulnerable to data breaches, compliance gaps, and operational inefficiencies. Traefik Hub's granular API access control empowers you to address these challenges head-on. Gain the control you need to secure your APIs, optimize developer workflows, and build a foundation for secure and thriving digital interactions.

Ready to learn more? Dive deeper into our documentation of operation set definitions for API & API Version and use them in operation filters in the API Access CRD.

]]>

How Modern API Gateways Make DevOps Engineers More Efficient

Immánuel Fodor — Wed, 03 Apr 2024 19:06:40 GMT

Imagine, for a moment, that you are in charge of a large chain of home improvement stores. You have hundreds of stores and tens of thousands of employees spread across 12 countries. You receive tens of thousands of orders every single day spread over 100,000 requests every single second. You’ve got four different IT platforms, all on Kubernetes. Latency is unstable. Debugging is time-consuming. Developers are frustrated.

And then the pandemic happens, sending the whole world online.

That’s the situation home improvement and gardening retailer Leroy Merlin found themselves in in 2020. Like so many companies, they knew they needed to face that “digital transformation” buzzword before they lost their developers to The Great Resignation. And in this case, that meant integrating an API gateway into their processes and architecture.

It’s a problem you may also be facing.

Why Consider an API Gateway Model?

An API gateway serves as an intermediary between clients and backend services, streamlining request routing, enforcing security measures, managing APIs and microservices, and optimizing the communication flow within complex software architectures. That’s a mouthful, but the important thing is that it provides a centralized and unified way for developers to interact with the backend services that power your applications and infrastructure.

An API gateway also provides several other features that an enterprise with significant infrastructure needs.

Traffic Management

Whether you have just a few lightly trafficked services or you’re dealing with 100K requests per second, it’s obviously crucial to make sure those requests are routed properly, but that involves much more than simply sending them to the right service.

The API gateway has to provide efficient routing, of course, but it also has to provide load balancing; it would be simple for a single service to not just bog down and stop responding, but also bring the entire system to a standstill, particularly during traffic spikes. The API gateway has to handle these spikes gracefully.

To do all this, the API gateway has to optimize service communication, making sure that data and requests take the most efficient paths through the system.

Authentication and Authorization

Just because an API gateway makes it possible to more easily access services doesn't mean that anyone and everyone should be able to access every service within the system. An API gateway needs to ensure that non-public resources can only be accessed by those to whom they should be available, whether that’s a user or another service.

To make this happen, you need to be able to verify digital identity through the most appropriate means. In some cases that means issuing, managing, and verifying tokens. In others, it may mean implementing multi-factor authentication (MFA). The API gateway also has to ensure that that particular identity actually has the privileges it’s trying to exercise so that sensitive areas are reserved for those with the right privileges, keeping customer data and critical services safe and sound. In other words, you need to make sure you’re wrapping API endpoints in layers of security that deter would-be digital trespassers.

Caching and Rate Limiting

Speed is the lifeblood of the digital experience. For example, Leroy Merlin’s site required multiple service calls, so to ensure it felt responsive, each call could take no more than 20ms. The API gateway, with its caching capabilities, stores copies of frequently requested data so that future requests can be fulfilled faster. It can also use smart caching strategies, anticipating the customer’s next move and ensuring the most relevant and up-to-date information is always at the ready, minimizing wait times.

Sometimes, however, the API gateway has to limit requests from a specific user or service in order to prevent overuse and preserve resources for the rest of the system.

Log Management

In modern applications and infrastructure, every action, every request, and every error leaves a trace in the form of logs. The API gateway acts as the central library for these vast volumes of data, collecting and organizing them so that they can be easily accessed, studied, and acted upon.

But a library is only as good as its librarians. By integrating with external monitoring—for example, Leroy Merlin needed a connector to send their logs to Datadog—the API gateway ensures that the logs are not just collected but analyzed, turning raw data into actionable insights that can guide the evolution of digital services.

Of course, this is probably most useful when things go awry, as they inevitably do. In this case, the collected logs become the map that leads developers and operators back to the source of the problem.

Finally, in the realm of digital commerce, compliance with laws and regulations is not just a necessity but a cornerstone of trust. Through meticulous log management, the API Gateway helps safeguard customer data and maintain the integrity of digital operations.

Certificate Management

Of course, it’s impossible to talk about digital security without talking about SSL/TLS for end-to-end encryption. The API gateway not only handles encryption, but also tackles certificate renewal, not just taking care of the actual renewal, but ensuring that all servers and services in the system received the new certificates—preferably without having to perform a full reload, resulting in the loss of current sessions.

For years functions like this have been handled by the Electronic Frontier Foundation’s Certbot, but more modern applications have been using open source projects like Traefik.

All of these capabilities can be helpful for any business, Leroy Merlin included. But API gateways that are aligned to modern operations, namely GitOps, are more than just helpful. They're transformative.

Why Choose a GitOps-Driven API Gateway

While the capabilities of a traditional API gateways bring a number of benefits, many were created before the cloud native and operations revolutions. To maximize the benefits of switching to an API gateway model, however, pairing a GitOps strategy with a GitOps-driven API gateway is a must.

GitOps is a set of practices that uses Git as the single source of truth for declarative infrastructure and applications. Essentially, it applies the principles of version control, collaboration, compliance, and CI/CD (Continuous Integration/Continuous Deployment) automation to infrastructure and application management.

Everything in the system—infrastructure, networking, applications—is described in a declarative manner. This means you specify the desired state of your system, and automated processes work to maintain that state. When you want to make a change, you change that stored configuration and commit the change to the repository, and the system makes the change when it’s merged. The process enables code review, automated testing, and collaboration before changes are merged and applied, which improves the stability and security of the system.

This way, the actual state of the system matches the declared state in the repository. If discrepancies arise, the system can automatically correct itself, or alert an operator to intervene.

This process provides a number of benefits for developers.

Automation

Because changes are applied automatically, developers can roll out a new configuration by simply making a single change, and as soon as it’s approved and applied, it can be deployed everywhere it needs to go. This way developers can make changes quickly without risking the inevitable human error that comes with manual changes.

That enables developers to work more quickly and confidently, knowing their changes can be tested and reviewed before they roll out, and that every change is tracked, versioned, and most of all, reversible.

Easy Roll Backs

If a problem does develop, recovery from failures becomes a streamlined process with GitOps. Instead of frantic fixes, teams can methodically revert to a stable state, analyze what went wrong, and apply lessons learned. All of this lets developers keep downtime to a bare minimum.

Knowing they can easily undo changes encourages experimentation and innovation, so developers can try new features and approaches with the safety net of easy rollbacks, fostering a culture of innovation.

Increased Reliability & Repeatability

Consistency is key to reliability. GitOps ensures that every environment, from development to production, is a mirror image, eliminating the "it works on my machine" syndrome and bolstering developer confidence in every release.

It also gives developers the ability to reproduce environments and deployments at the push of a button, which means the power to quickly spin up new instances, test environments, or recover from (the inevitable) disasters. Cloning existing environments is as simple as cloning a repository and applying the configuration elsewhere.

What’s more, because all of these environments are based on the same images, developers get the standardization that is the foundation of efficiency. GitOps enforces a uniform setup across all infrastructure, simplifying management, reducing complexity, and ensuring that best practices are universally followed.

Better Collaboration

GitOps provides a common framework for operations, establishing a unified language and approach that streamlines workflows and harmonizes efforts across the developer landscape.

It creates a collaborative environment where developers, operations, and even security teams work from the same playbook, enhancing communication and reducing friction. Every change is tracked, every decision logged, making sure all team members are on the same page, fostering trust and helping create smoother operations.

With infrastructure and configuration codified, contributing changes becomes as simple as contributing code, lowering barriers for team members to suggest improvements or fixes, and the GitOps workflow encourages dialogue across departments, making sure that everyone's voice is heard and that decisions are made with a holistic view of the organization's objectives.

Putting it all together

By opting for a modern, GitOps-based API gateway (in this case, Traefik’s API Gateway), Leroy Merlin solved a number of issues. This approach creates significant benefits that improve workflows and productivity, streamline deployment processes, and make them more automated, consistent, and error-free. All this drastically reduces the time and effort Leroy Merlin and companies like them need for manual configurations and deployments.

The clear, version-controlled history of infrastructure changes improves collaboration between team members, allowing for easier code reviews, quicker onboarding of new developers, and a better development environment. Additionally, the ability to quickly rollback to previous states in case of errors or unforeseen issues significantly lowers the risk associated with new releases, fostering an environment where innovation and experimentation are encouraged without the fear of catastrophic failures.

All of this can not only boost the morale and efficiency of your development team, but also accelerate the pace of digital innovation, just as it did for Leroy Merlin, ensuring they could rapidly respond to market demands and maintain a competitive edge in the retail space.

]]>

We Built The Most Modern API Management from Scratch. Here’s How.

Emile Vauge — Thu, 21 Mar 2024 14:40:29 GMT

Complaining about things that don't work well is easy. What's hard is coming up with a better solution.

That statement is true of many things in life, including one we know well at Traefik: API management. After all, there is plenty to criticize about traditional API management — such as poor integration with cloud-native technologies, inefficient incident management, challenging release management and poor governance strategies.

But what does a better approach to API management look like? How, specifically, do you implement a tool that enables cloud native-friendly, scalable, collaborative API management? Those are not simple questions to answer.

But these are the questions that we had to solve in order to build Traefik Hub. We had to make careful design choices to determine how API management workflows based on our tool should work. We also had to analyze a variety of technical considerations, such as how best to implement tunneling and a high-availability data plane.

We're proud of the solutions we arrived at. We're so proud that we'd like to share them — and the rationale behind them — in this article, with the goal of offering our community a peek into the design process and thinking that led to Traefik Hub.

Why We Decided to Reboot API Management

Before delving into the choices we made when designing Traefik Hub, let's talk about why we decided to build the solution in the first place.

The reason was simple: we believe traditional API management is badly broken. Most conventional API management tools were designed and built before cloud-native architectures became widespread, and before the widespread adoption of collaboration-centric, DevOps-based cultures. As a result, they lack the scalability, modularity and efficiency to drive optimal results in today's scale-out, distributed world.

We conceived of Traefik Hub as a way to reboot API management, so to speak. By building a new solution from the ground up — instead of trying to extend or overhaul an existing API management tool — we gave ourselves total freedom to reinvision what modern API management should look like.

Our Design Principles

Starting from scratch meant we had virtually unlimited leeway to design Traefik Hub in whichever ways we thought best. But it also meant we had to make a ton of design choices, since there was no pre-existing foundation to guide us or get us started.

We settled on an approach oriented around the following design principles.

The Unix Philosophy

For starters, we wanted to adhere to the so-called Unix philosophy, which emphasizes designing a tool that does one thing and does it well. That's why Traefik Hub doesn't try to be a Swiss army knife of API-related functionality. Instead, it focuses on API "day 2" operations — API deployment, runtime management, observability, and security.

Other tools are available to assist with processes like API design and documentation, and we don't think it's healthy to try to pack too many features into one tool.

Declarative Management

In the cloud-native era, engineers have become accustomed to declarative management. That means writing code that describes how something should operate, then applying the desired configuration automatically.

Traefik Hub does this by enabling a GitOps-based approach to API management: by using files that you can manage through Git repositories, you describe what should happen to APIs, then Traefik Hub makes it happen automatically. In addition to helping to keep configuration data centralized and accessible, this approach enables scalability and automation in the realm of API management.

Kubernetes-Native

We decided not just to be cloud-native in general, but to focus on creating a truly Kubernetes-native solution, because Kubernetes has become the de facto solution behind cloud-native environments.

By "truly Kubernetes-native," we mean that Traefik Hub is CRD-driven and can be managed directly through standard kubectl commands, rather than requiring a proprietary tool.

Intuitive and Rapid Time-to-Value

We don't think anyone should have to take a course or spend months reading documentation and following tutorials to manage APIs with Traefik Hub. Instead, we prioritized a simple, intuitive design that minimizes the learning curve for anyone who is already familiar with cloud-native concepts and tooling.

Lightweight, Composable Architecture

Most cloud-native architectures are composable, meaning they involve a variety of loosely coupled components. Traefik Hub adopts the same type of design by giving users freedom to pick and choose from a variety of modern solutions when building their stacks. In other words, Traefik Hub doesn't force you into a particular stack or platform; you can run it alongside other tools of your choice.

Data Plane and Control Plane Separation

Finally, we decided that it was important to keep the data plane and the control plane separate — partly because we believe in modular and composable design, as we just noted, but also to enhance the security and reliability of our API management solution. We'll say more later about exactly how we implemented the data plane with this goal in mind.

Building Traefik Hub: Challenges We Faced and Lessons We Learned

Deciding on high-level design principles is one thing. Actually implementing them is another. And while we don't have room in this article to discuss every technical decision we made as we built Traefik Hub according to the design concepts described above, we'd like to highlight a couple key choices we made as we worked through implementation challenges.

Tunneling

One was tunneling. Since we had to expose some resources publicly, creating tunneled connections to them was an elegant solution. We wanted the tunnels to be:

Highly available
Encrypted to protect data
Compatible with full-duplex mode (so that data could flow from either plane to the other plane simultaneously)
Transparent to firewalls, so users would not require public IP addresses to expose the data plane
Capable of supporting multiplexed data transfer

There are a variety of tunneling technologies that have the potential to support at least most of these goals. At first, we considered using TCP and SSH, but that would have required a public IP address, and the connection would not have been reliable because there is no true fallback solution on SSH disconnect. We also thought about using WebSocket and SSH, which doesn't require a public IP, but is still subject to the disconnect issue.

Since SSH-based tunneling didn't seem ideal, we began exploring approaches based on Yamux, an open source multiplexing library. Initially, we considered using Yamux alongside GRPC, but that felt too hacky and the data envelope was too large. TCP and Yamux also proved suboptimal because it would have required a public IP.

Finally, we settled on WebSocket and Yamux — which doesn't need a public IP, allows firewall passthrough, provides built-in encryption via TLS, and is more reliable following disconnects than an SSH tunnel.

Data Plane High Availability

When implementing the data plane for Traefik Hub, we wanted to make sure our solution would be resilient against regional network failures. In other words, we didn't want a problem in one part of the network to make the entire solution unavailable.

The first step in making this possible was to deploy dozens of brokers in different regions. But on their own, distributed brokers don't guarantee high data plane availability because a data plane would still fail if it's connected to just one broker and that broker goes down. So, instead, we designed our data plane to connect to at least three different brokers simultaneously — meaning that it can tolerate the failure of a minimum of two brokers before the data plane becomes unavailable.

In addition, we dynamically optimized load balancing for data plane connections. This ensures that if failures occur in some parts of the network, traffic is automatically rebalanced to keep data flowing as efficiently as possible.

Conclusion: A New Approach to API Management

Again, there's plenty more to say about how Traefik Hub works under the hood, and why we decided to make it work that way. But we hope that the details above have provided at least a basic sense of what our thought process was as we reconceptualized API management, as well as the types of technical iterations we worked through to get things just right — which means bringing scalability, simple incident and release handling, and intuitive collaboration to the realm of API management.

]]>

Traefik Hub - API Gateway Perfection

Emile Vauge — Tue, 19 Mar 2024 02:34:03 GMT

Traefik Hub has profoundly transformed API Management. It provides a fully declarative approach to define, manage and run APIs while offering modularity and freedom of choice for its users through deep integration with the Cloud-Native ecosystem.

API Management brings critical capabilities to help companies handle their API fleet. Runtime API Governance is the cornerstone of Traefik Hub API Management: dynamic API discovery, logical grouping through collections, change and incident management through versioning and linters, and ease-of-consumption through developer portals. Traefik Hub API Management also adds an advanced security layer on top of it with industry-leading Granular Access Control and provides rich awareness and control of the whole API pool with precise monitoring and state-of-the-art observability.

Delivering on this simple yet powerful vision, Traefik Hub now delivers essential capabilities to help companies seamlessly start their API journey. See this as the wildly popular and familiar Traefik Proxy plus core API-focused features like Access Control (OAuth, OIDC, etc) and native scalability capabilities like Distributed Rate Limiting.

To put it more simply: Traefik Hub seamlessly transforms Traefik Proxy into the perfect API Gateway!

Let’s dig deeper into it.

From Traefik Proxy To Traefik Hub API Gateway

One key element of Traefik Hub API Gateway is how seamless it is to upgrade from Traefik Proxy. Take an existing Traefik Proxy ingress controller installed on Kubernetes.

Step 1: Install the Traefik Hub agent with Helm, providing the same Traefik Proxy configuration options.
Step 2: Remove Traefik Proxy, et voilà, you have a fully functioning API Gateway, that will act as the ingress controller exposing the existing ingress ressources.

API capabilities are configured using the Traefik Proxy middleware well known mechanism.

To illustrate this, let’s take a rate limiter attached to an IngressRoute used with Traefik Proxy:

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: ratelimit
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ratelimit
spec:
  rateLimit:
    period: 1m
    average: 600

To add OAuth authentication to this existing IngressRoute, just add an OAuth middleware and Traefik Hub will enforce OAuth authentication on it!

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: ratelimit
      - name: oauth-client-creds
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ratelimit
spec:
  rateLimit:
    period: 1m
    average: 600
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-client-creds
spec:
  plugin:
    oAuthClientCredentials:
      url: https://tenant.auth0.com/oauth/token

How simple it is! As you can see, getting access to API Gateway capabilities when you already use Traefik Proxy is transparent.

Now, let’s explore all these new middlewares!

The many flavors of OAuth

OAuth 2.0 is an open standard dealing with resource access control and is the latest version of the authorization protocol OAuth. An OAuth client provides web, desktop, and mobile application authorization flows. We brought to Traefik Hub one of the most complete OAuth implementations which comes with two OAuth middlewares:

Token Introspection

Token introspection allows Traefik Hub to retrieve information about the user from an authentication server. Then, once the token is retrieved, it uses fine-grained configuration, including nested claims, to grant access or not to the route.

Let’s imagine that the token it retrieves is the following

{
  "active": true,
  "grp": "admin",
  "scope": "reader writer deploy",
  "referrer": "http://example.com/foo/bar",
  "areas": [
    "office",
    "home"
  ]
}

You can validate this token using many functions, for example:

Equals(‘grp’, ‘admin’) checks if the value of grp in the token is equal to “admin”
Prefix(‘referrer’, ‘http://example.com’) checks if the value of referrer in the token starts with http://example.com
Contains(‘referrer’, ‘/foo/’) checks if the value of referrer has the sub string foo. When used on an array of values, it will check whether the value is in the list or not.
And others…

Most importantly, you can combine any of these using operators (and, or, not) creating powerful ways to express who should have access to what.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oauth-oauth-intro
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-oauth-intro
spec:
  plugin:
    oAuthIntrospection:
      tokenSource:
        header: Authorization
        headerAuthScheme: Bearer

Client Credential

Client Credential allows Traefik Hub to secure routes using the OAuth 2.0 Client Credentials flow as described in the RFC 6749. Access tokens are cached using an external KV store.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oauth-client-creds
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-client-creds
spec:
  plugin:
    oAuthClientCredentials:
      url: https://tenant.auth0.com/oauth/token

OIDC

OIDC is an authentication layer built on top of the OAuth 2.0 protocol. OpenID Connect allows an application to obtain user login information by exchanging cryptographic tokens with an identity provider and is often used to implement federated SSO between multiple applications. With OIDC, Traefik Hub allows you to delegate the authentication process to a third party (Google Accounts, LinkedIn, GitHub, etc.) to obtain the end-user information for authorization purposes, where you get the same set of functions / operators to define the rules.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oidc
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oidc
spec:
  plugin:
    oidc:
      issuer: "https://tenant.auth0.com/realms/myrealm"
      redirectUrl: "/callback"
      clientID: my-oidc-client-name
      clientSecret: mysecret

JWT

JWT is a popular tool for authenticating API calls and single sign-on (SSO) applications. It’s a method of digitally signing information as a JSON object. The JWT includes a set of claims, which typically describe what an authenticated user is allowed to do. With JWT validation, Traefik Hub is capable of verifying the validity of the token, extracting its information, and then as usual checking against the rules you want to define.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: jwt
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: jwt
spec:
  plugin:
    jwt:
      signingSecret: urn:k8s:secret:my-secret:signingSecret

API Keys

When you need a quick authorization mechanism, API Key is one of the best options. You configure the values that should allow consumption of the API, and where to retrieve it from (header, query param, cookie), and it’s as simple as that.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: apikey
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: apikey
spec:
  plugin:
    apiKey:
      keySource:
        header: Authorization
        headerAuthScheme: Bearer

Distributed Rate Limit

When scaling Traefik Hub with standard rate limiting, each replica computes its own rate limit, meaning that if you configure 10 requests per second, you’re actually allowing (instance number) * request limit over time. When reaching the limit, depending on how well traffic is load-balanced between the instances, there might be some (small) inconsistencies in the response.

The distributed rate limit solves this problem and eases the configuration, allowing each replica to be aware of the remaining allowed requests. When you configure a given request per time allowance, you’re guaranteed that whatever the number of replicas you run, this is what will go to your service.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: distributedratelimit
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: distributedratelimit
spec:
  plugin:
    distributedRateLimit:
      limit: 100

Wrapping Up

An API journey is long and complex. The last thing you want is to choose the wrong product to support you. Start with an over-engineered product and your ability to quickly iterate under control will vanish. Start with a product you can’t upgrade and you will be locked in at an early stage, without the ability to evolve to more advanced practices and technologies.

Traefik Labs provides companies a very progressive stack to deploy and manage APIs. From the ingress controller use case with Traefik Proxy to route and load balance services, to the API Gateway with critical capabilities around security and centralized control, up to the full API Management solution. All this within Traefik Hub, the most modern and progressive API runtime platform for Cloud-Native environments.

]]>

Why does WAF matter in API security?

José Carlos Chávez — Mon, 18 Mar 2024 19:56:59 GMT

Nowadays, APIs power the internet. We see APIs (Application Programming Interfaces) everywhere: webshops, social networks, mobile apps, government sites and most of the media entertainment is delivered to us over APIs. This is why APIs are so important and also why protecting APIs is fundamental for every organization being on the internet.

According to OWASP Top 10 API Security Risks 2023[1], 4 of the top 5 security risks are related to authentication and authorization, however by October 2023, the U.S. SEC launched an investigation around the MOVEit mass-hack that has exposed the personal data of at least 64 million people[2]. Considered the biggest hack of 2023, it began by exploiting a previously unknown SQL injection vulnerability in MOVEit Transfer. On the same line, cvedetails.com reports that the majority of CVE security vulnerabilities in 2023 came as Cross-Site Scripting (XSS) attacks, followed by memory corruption and SQL injection [3].

While identity and permissions are the foundation of API security, footprint attacks (XSS, SQL injection, Remote Code Execution etc) are still leading the game when it comes to security breaches hence the importance of protecting our APIs looking at the footprints.

Web Application Firewall (WAF) as part of a wider solution

No security layer can address all possible attack vectors, there are different layers of security with different concerns which sometimes narrowly overlap. A multi-layered strategy that acknowledges the complexity of the threat landscape is preferred, hence the need for a diverse array of defenses where each layer is a barrier to attacks, and together, they create a resilient shield capable of adapting to the shifting tactics of adversaries.

WAFs are proxy-based tools that mainly inspect incoming HTTP(S) requests. Its capabilities vary, however the basic function is to provide an application layer filter for web and API traffic. This stage looks for malicious and unwanted content within incoming requests (headers and payloads) and acts accordingly as well as making sure that only allowed actions can be performed.

Some of the WAF capabilities that are crucial for API security are:

Known attack detection: Recognizing common attack strategies and blocking unauthorized access.
Malformed/anomalous request detection: Distinguishing legitimate API requests from those with malicious intent.
Virtual patching: Intelligently delivering security patches to early protect API endpoints until the root cause can be fixed.
Anti-bot automation: Identifying and blocking malicious bot traffic.
Audit logs: Producing audit information about matched rules and anomaly scores that can be aggregated to come up with actions.

Any system that receives traffic (internal or external) is a target for attacks and any system that is target for attacks can be protected by a WAF, from websites, blogs, government portals, e-commerces, forums. Some of the attacks categories a WAF can protect you against of is:

SQL Injection (SQLi): malicious traffic could manage to inject strings directly into SQL queries to perform undesired actions e.g. retrieve users data.
Cross Site Scripting (XSS): attacker injects malicious executable scripts into the code of a trusted application or website
Local/Remote File Inclusion: attacker includes local/remote files on the server to later execute them e.g. to expose server data.
Code Injection: attacker injects code into the application execution through a vulnerability e.g. to expose environment credentials in the output.
Session Fixation: permits an attacker to hijack a valid user session
Bot Detection: the process of identifying and distinguishing between human users and automated bots.
Metadata/Error Leakages: leaks server or implementation metadata or internal errors to the attacker e.g. to exploit known vulnerabilities.

In a multilayer security approach, a WAF sits in the first line, protecting our systems from traffic attacks, even before authentication or authorization as those could also be compromised by a footprint attack.

PCI DSS 4.0 has WAFs as a hard requirement

The Payment Card Industry Data Security Standard (PCI DSS) is an information security standard used to handle credit cards from major card brands. The requirements are specifically designed to protect against security breaches.

While merchants, e-commerce businesses, retailers, and financial institutions are common entities that must comply with PCI DSS, other industries and service providers also fall under its scope. For instance, healthcare organizations, hospitality businesses and in general any internet site that handle card payments for bookings and services must comply with PCI DSS.

Even an organization that does not process cardholder data could follow the PCI Standard to implement a robust cybersecurity program for any of its important data and gain the confidence of its customers.

In version 4.0 (deadline: March 2025) WAF takes a fundamental role. Requirement 6.4.2 [4] explicitly requires affected businesses to "Deploy an automated technical solution for public-facing web applications that continually detects and prevents web-based attacks" with compliance to at least the following criteria:

Is installed in front of public-facing web applications and is configured to detect and prevent web-based attacks.
Actively running and up to date as applicable.
Generating audit logs.
Configured to either block web-based attacks or generate an alert that is immediately investigated.

WAFs targeting deeper protection

While originally WAFs were created and designed to protect API gateways as a first line of defense from outer traffic (so called north-south), strategies like zero-trust tell us that we should be protecting the system components as if the attacker is already inside of the network.

Hence in the same way we leverage a WAF in the API gateway we should be protecting the individual components from the internal traffic (so called east-west) with a WAF using a refined set of configurations and more symbiotic deployments where WAF is in the ingestion path of the component e.g. in the sidecar as part of a service mesh.

This is specifically important when it comes to lift and shift. Legacy systems being onboard in the cloud, also onboards all its vulnerabilities. Sometimes patching these systems is not an option because the code is too old, it lacks tests or requires risky upgrades while being critical. A WAF can help to protect them in the cloud without the need of doing changes into its code but instead rolling out a reverse proxy in front e.g. as a sidecar intercepting all the incoming traffic looking for exploits.

WAFs as part of the development workflow

With the proliferation of microservices, macroservices, lift and shift and lambdas, teams became more independent in a fast changing architecture which means that security teams can roll out general purpose organization-wide policies but it is the individual development team’s responsibility to compliment the security measures with curated WAF rules more related to their components and its specific risks.

This is a shift in the security mindset which moved from an afterthought and only delegated to security teams into a shared responsibility between development and security teams. A cloud native WAF promotes the so-called “shift-left” which is the practice of moving security, testing, quality, and performance evaluation early in the development process and making development teams accountable for that.

WAF unblocking your security pipeline

In December 9th 2021 a new vulnerability was disclosed in the super popular Apache log4j library, affecting an estimated 10 percent of all digital assets at time of discovery [5], allowing remote code execution (RCE) on underlying servers that run vulnerable applications. The next day, Log4j 2.15.0 was released and security teams were urged to install the latest version. In the next seven days two more vulnerabilities were disclosed and by January 4th, US Federal Trade Commission told companies to patch Log4j vulnerability threatening legal actions.

Two years later, more than 1 in 3 applications using Log4j currently run vulnerable versions of this library [6]. Research found that, in general, once developers are alerted to a vulnerable library through a scan, 50 percent of vulnerabilities are fixed in 89 days overall, in 65 days for high severity vulnerabilities and in 107 days for medium severity vulnerabilities.

These numbers tell us that patching vulnerabilities does not occur as frequently and quickly as one might expect, however it is a negligence to leave systems vulnerable and hence WAF features like Virtual Patching [7] can chime in to protect those unpatched systems.

Virtual patching is a method of protecting new vulnerabilities in the short-term to keep hackers away from breaching systems until they can be patched in code. A newly discovered CVE in OSS triggers several processes both in the OSS code as well as proprietary code and a patch landing into users’ systems requires several other processes leaving the system vulnerable during that time unless a virtual patch has been applied.

Conclusion

WAFs play a critical role in API security, serving as an essential component of a multi-layered defense strategy. By detecting and mitigating a wide range of threats in real-time, WAFs help safeguard APIs, protect business-critical sensitive data, and ensure the integrity of online services.

WAF not only takes part of the operational stage of the systems bringing protection and compliance, it brings more benefits across the entire security strategy across the engineering teams that will translate into customer trust and good reputation.

References

]]>

Revolutionizing API Operations: A Dive into GitOps-Based API Management

Immánuel Fodor — Thu, 14 Mar 2024 22:26:21 GMT

APIs (application programming interfaces) are a hot topic these days with the rise of machine-to-machine communication and AI, microservice-based cloud native architectures. Add to that the never-ending need for efficiency to reuse existing software components and services to build up solutions faster and cheaper without reinventing the wheel.

Traditional API management solutions are often slow and suboptimal for efficiency, built with click-operations in mind, leading to stateful and procedural proprietary scripting practices, which result in a steep learning curve, preventing the fast onboarding of new team members, and making collaboration difficult at scale. Many existing tools are 10+ years old, built before the Docker and Kubernetes era, which makes them far from ideal for modern environments. Add all these up, and you get bottlenecks, long release times, faulty deployments, and outdated documentation.

Here, we arrive at the need for a more efficient and up-to-date central view of managed APIs, their configurations, usage, and health analytics, with the capability to audit changes and maintain an up-to-date single source of truth.

We invite you on a journey to unravel the power of GitOps in streamlining API management, offering transparency and reliability, and paving the way for a more agile and collaborative future.

GitOps: A Proven and Efficient Answer

GitOps, a methodology widely adopted by infrastructure teams operating applications and services, can bring an interesting, fresh approach to API management with a focus on operational efficiency.

Let’s see the principles behind GitOps:

Declarative configuration: Humans declare the desired state, and systems autonomously determine the steps to achieve it—similar to a GPS defining the route given a desired final destination. Automated early checks can ensure that the declared state is consistent, error-free, and compliant. The declarative approach promotes a shared understanding within teams, fostering collaboration by discussing freely and within the code itself.
Versioning and immutability: Committing every change to Git in an immutable fashion enables a clear audit trail and the ability to reproduce current and previous configurations. As a result, it ensures accountability and traceability throughout the configuration lifecycle. In case of issues, Git provides seamless rollbacks to any previous working state, minimizing downtime.
Automatic pulling: When Git is established as a single source of truth for configurations, automatic pulling mechanisms eliminate surprises, creating a predictable modern environment. Substituting human effort with automated pipelines guarantees that changes are deployed consistently and efficiently, reducing the likelihood of manual errors and simplifying compliance.
Continuous reconciliation: Running the reconciliation process in a continuous loop can automatically correct any unexpected deviations from the declared state, contributing to a reliable and resilient system architecture. It also ensures stability by preventing unintended changes from piling up.

GitOps aligns seamlessly with the Unix and Kubernetes philosophy of doing one thing well and being composable with other tools. By adhering to these principles, GitOps ensures simplicity, efficiency, and compatibility, making it a versatile and powerful methodology for managing infrastructure—and API operations, as we’ll see soon.

The Philosophy of GitOps for API Operations

The core of GitOps for API operations lies in treating APIs as infrastructure, represented by Custom Resource Definitions (CRDs) within Kubernetes. This allows API configurations to be managed declaratively, just like other Kubernetes resources, simplifying management and reducing configuration errors. See an API CRD below as an example:

apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: employee-api
  namespace: apps
  labels:
    area: employee
    module: crm
spec:
  pathPrefix: "/employees"
  service:
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000
    name: employee-app
    port:
      number: 3000

GitOps promotes human-readable and editable configurations stored in Git repositories. APIs, rate limits, security policies, and other constructs are clearly defined, enabling collaboration between developers, operations, and security teams.

GitOps-based API Operations provide full auditability with version control. Every change, comment, and approval is linked to its specific version, creating a clear historical record. Deployment status is continuously visible, ensuring transparency and accountability.

Tools that understand declarative API configuration can perform automated checks and analysis of configuration changes before deployment. These checks identify potential errors, security vulnerabilities, and performance impacts, allowing proactive action to address them before causing disruptions.

By leveraging GitOps, API-related deployments become fully automated. Changes are committed to Git and automatically applied to the API Gateway or API Management platform. This eliminates manual installations, reduces downtime, and minimizes human error by eliminating repetitive tasks.

API operations become agile and reversible with GitOps. With Git as the single source of truth, rolling back to previous versions becomes simple, offering a safety net for experimentation and updates. Continuous updates become the norm, ensuring APIs remain secure and aligned with evolving business needs.

Operational Benefits Adopted from GitOps to APIOps

Building on the core philosophy of GitOps for API Operations, let's dive deeper into the specific operational benefits this approach unlocks:

Separation of concerns: streamlined development and deployment. API code and configuration releases can be decoupled, allowing faster innovation and bug fixes on either side without impacting the other.
Transparency and auditability: trace every change with confidence. You can easily see who made what change, when it happened, and why. This empowers compliance and security teams to assess risks and identify potential issues. Debugging configuration issues also becomes a breeze accelerating problem resolution.
Authentication and authorization: control who can change what. This is a two-fold concept. You can implement role-based access control (RBAC) within your GitOps workflow, defining who can push changes to different parts of the API configuration. Also, you can define RBAC for APIs in Git, ensuring only authorized personnel have access to sensitive API paths and methods. Each commit is associated with a user, promoting accountability and preventing unauthorized modifications of API access policies.
Risk reduction: rollback and logging assurance. Experiment and make changes fearlessly with confidence, knowing you can easily revert if needed. Git’s comprehensive record-keeping helps identify the root cause of issues and facilitates post-mortem analysis for continuous improvement.
Security policy: code-enforced vigilance. By integrating automated security scans and checks early in the process, GitOps helps shift left security, catching and addressing vulnerabilities before they reach production. This proactive approach enhances the overall security posture of your APIs.
Scalability and consistency: a self-healing API infrastructure. As API deployments grow, GitOps scales efficiently, managing configurations consistently across environments. API configurations are automatically re-applied when deviations occur, ensuring the desired state is maintained.

By adopting GitOps principles for API management, you gain a streamlined workflow with declarative configuration, clear audit trails, granular access control, and the safety of effortless rollbacks. The transparency improves security through early vulnerability detection and promotes consistency in multiple environments for a highly scalable and manageable API infrastructure.

Traefik Hub: GitOps-Driven Kubernetes-Native API Management

Imagine an API Management solution seamlessly integrated into your Kubernetes environment, built with GitOps for rock-solid configuration and automation. We’ve built Traefik Hub with all these principles and benefits in mind on the well-known and highly performant open source Traefik Proxy to make this dream a reality.

No more complex configurations! Traefik Hub leverages familiar Kubernetes Custom Resource Definitions (CRDs) for publishing and managing APIs. Simply declare your APIs, their versions, access policies, and group them using standard Kubernetes labels and selectors. This intuitive approach grants you powerful control without sacrificing ease of use.

Traefik Hub is built up from the following architectural components:

The Traefik Hub Agent: The guardian standing at your cluster's door, securing and monitoring incoming traffic on the data plane, hosting your API gateways, and ensuring smooth operation.
The API Dev Portal: Empower your API consumers with a dedicated portal. They can explore API documentation, request API keys, and get the most out of your offerings. The portal UI is available as an open-source example on GitHub, which you can tweak and extend to your liking.
The Traefik Hub UI: Gain a bird's-eye view of your API kingdom. View connected agents, API configurations, and other resources across multiple clusters and users.
The Traefik Hub Static Analyzer: Shield your APIs from errors before they deploy! This proactive tool meticulously analyzes configurations, pinpointing potential issues before deployment. Its human-readable impact analysis, embedded directly in Git pull requests, offers clear insights into the consequences of every change, empowering informed decisions and ensuring smooth deployments.

In the next sections, we’ll see two use cases of utilizing GitOps-based API Management with Traefik Hub. We’ll focus on operational excellence in day 2 operations, when every architectural component is in place, and you need to make APIs run smoothly and securely.

API Change Management with GitOps

In the first use case, we look into configuration safety. Traefik Airlines, our fictitious company, is a modern airline company that offers various digital services to its customers and partners. To meet evolving needs while maintaining business stability, Traefik Airlines must ensure its API Change Management is robust to minimize service downtime caused by configuration errors. Let's explore how this can be achieved by using Traefik Hub’s GitOps-driven API management.

Traefik Airlines regularly publishes changes to its Git repository, where the configuration of all managed internal and external APIs is stored. However, they configured the Traefik Hub Static Analyzer to run automatically for every pull request to Git.

The analyzer has a linter functionality that acts as your first line of defense, scanning configurations for errors and preventing them from reaching production. The errors found are not tucked away in an obscure location but put into context next to the error.

In the above screenshot, the analyzer running as a GitHub Action catches a typo that would cause an outage in one of the Airline APIs by referencing a non-existent Kubernetes service underlying the API. The action prevents the pull request's merge and gives contextual help on the error to let it be corrected in a follow-up Git commit that is now safe to deploy into production.

The linter can check for various configuration errors, including duplicate, orphan, childless, and invalid resources; resource references; selector definitions; conflicting API paths; unknown operation sets; and service ports.

The analyzer can also do an impact analysis on the change set of a pull request preventing modifications with significant differences from being merged and ensuring the cluster remains stable.

The above screenshot shows the human-readable reports outlining the affected APIs and users of an API consumption rate limit change. It allows the pull request reviewer to confidently decide whether to merge the changes or refine them further.

API Incident Mitigation and Resolution with GitOps

In our second use case, we see how you can manage your APIs with the confidence that any misstep can be instantly undone.

With GitOps's magic touch to API management, rolling back a faulty update of an API to its previous, healthy state is a breeze. No more scrambling, no more downtime, no more complaints from API consumers—just a quick rewind via a click of a button and you're back in business.

But GitOps doesn't stop there. It empowers your developers to become debugging detectives. When an error pops up, the faulty state is preserved like a crime scene, waiting to be examined. This traceability buys precious time to delve into the root cause, craft a fix, and deploy it seamlessly through a new pull request.

Wait, there's more! OpenTelemetry metrics paint a vivid picture of your API landscape. They expose error rates, latencies, and request counts, all categorized by API, version, and user. Want to zoom in on a specific API or version? No problem! Drill down with ease and gain laser-sharp insights into user behavior.

And the best part? You don't need to rip and replace your existing tools. OpenTelemetry integrates seamlessly, acting as a common language for your event data. This opens the door to exciting possibilities—e.g., effortless event correlation with Git actions, AI-powered analysis, and a world of customizations tailored to your needs.

The above screenshot shows Traefik Airline’s monitoring dashboard with GitHub pull requests and Flux deployment events correlated to OpenTelemetry-based API metrics collected by Prometheus and visualized in Grafana. Whenever something goes bad, and the unexpected happens, it’s easy to see which change caused the problem and roll it back to restore healthy operations.

You can dive into the code and see how GitOps API management can transform your API Operations for yourself with our example repository.

Slash Costs, Boost Efficiency: GitOps API Management is Your ROI Goldmine

Renowned tech experts at Gartner have labeled GitOps as an emerging technology on the verge of widespread acceptance, marking its transition from early stages to the brink of mainstream adoption. Time is money, and GitOps saves both. Forget the days of API Management costs draining your budget. Get ready to unlock efficiency, agility, and massive cost savings with the revolutionary power of GitOps for your API Operations.

Accelerate releases: Get changes from dev to production faster than ever. Save precious time and human effort across your team.
Avoid slashing SLAs: No more costly downtime due to faulty updates. With a single click, rewind to a stable state and keep your APIs humming.
Solve problems efficiently: Drill down into performance metrics, pinpoint issues before they snowball, and avoid costly outages.
Work on what matters: Automate repetitive tasks and eliminate manual interventions, freeing up your team for higher-value work.
Pay only for what you use: Meet demand fluctuations effortlessly with dynamic infrastructure provisioning, scaling up and down gracefully.
Eliminate waste: Say goodbye to siloed data, gain a holistic view of your API landscape, and optimize resource allocation.

Reduced effort, faster cycles, and fewer outages translate to measurable financial benefits. GitOps-driven API management is not just technology, it's a philosophy and a financial game-changer.

Wrapping It Up

In conclusion, the GitOps approach represents a significant paradigm shift in API operations. Grounded in the declarative principles of Kubernetes, it offers a sophisticated framework that empowers a new era of API management focused on collaboration, transparency, and control. This evolution is particularly relevant in an era dominated by the widespread adoption of Kubernetes and the integration of machine-to-machine communications.

With the operational benefits of GitOps-based API operations, you can achieve faster iterations, enhanced security, and improved overall API governance, unlocking new levels of agility and efficiency for your organization.

]]>

Why Modern API Management Needs an End-to-End GitOps Strategy

Immánuel Fodor — Fri, 01 Mar 2024 19:44:10 GMT

GitOps isn’t just another IT buzzword; it’s quickly transforming how software development operates, making deployments more reliable, repeatable, and automated. GitOps is used to manage infrastructure and workloads, but it hasn’t taken hold in managing APIs. While this modern operations model is used in API development, API management has yet to benefit from GitOps-driven processes, and it’s sorely needed.

With the rise of cloud-native development and microservices, a growing majority of development involves APIs, and managing them is key to both development velocity and application security.

By including GitOps throughout the API management process, DevOps and platform engineers are able to enforce consistency and make it possible to scale operations, all while enhancing the developer experience, fostering collaboration, and bolstering security. Let’s take a deeper look at how GitOps enables better API management.

Streamlined Workflows: Orchestrating Deployments in the GitOps Era

In the GitOps landscape, every deployment is a reflection of the desired state defined within a Git repository. This declarative approach, where the desired state of infrastructure and applications is codified, demands seamless and efficient workflows.

GitOps-driven API management ensures that as changes are pushed to the Git repository, they are mirrored in real-time across the infrastructure. This real-time synchronization not only accelerates deployment cycles but also minimizes manual intervention, a cornerstone of the GitOps philosophy.

But it's essential to understand that the role of API management isn't limited to just facilitating deployments. It extends to cataloging and securing both internal APIs that drive microservices and external APIs that interface with third-party services.

Most of all, by automating tasks that were traditionally manual, GitOps-driven API management reduces the scope for errors. For example, when a new API endpoint is added to a service, the management tool can automatically catalog it, apply predefined security policies, and ensure it aligns with the desired state in the Git repository.

Consistency: Maintaining Equilibrium in the GitOps Landscape

When it comes to GitOps, consistency is more than a best practice—it's the point. With every piece of infrastructure and application configuration codified within a Git repository, GitOps is about ensuring that the live state mirrors this codified state.

One way in which GitOps enhances consistency in API management is through the use of validation of changes or “early checks.” These early checks can compare changes against a set of rules, failing violations to ensure that nothing questionable is implemented. If an issue should arise, you have the ability to add any knowledge gained through fixing the problem to the set of rules, preventing it from happening again.

API management tools should ensure that as configurations are updated or new ones are added to the Git repository, they are consistently applied across all environments. Whether it's a development sandbox, a staging area, or a production setup, the behavior and configuration of APIs remain uniform. This uniformity is achieved through centralized configuration management, where API configurations are stored, versioned, and synchronized with the Git repository, along with the stateless nature of Kubernetes and its underlying immutable infrastructure.

Such centralization eliminates the discrepancies that often arise between environments, ensuring environment parity—a core tenet of GitOps. With it, teams can be confident that an API's behavior in a development environment is replicated identically in production, reducing "it works on my machine" issues.

More than that, in the event of an outage, not only can changes be rolled back in a matter of minutes, but you can duplicate the environment in order to debug. So instead of your engineers getting jolted out of bed at 2am on a Sunday while you lose money every minute you’re down (the Ponemon Institute estimated the average cost of downtime at $9000 per minute!), the system can rollback automatically, and the engineers can come in on Monday to an error report they can debug at their leisure before reapplying.

Furthermore, in a GitOps-driven world, changes are continuously integrated and deployed. This ensures that every deployment, every change, and every API call aligns perfectly with the state defined in Git. And if the API management tool is based on Kubernetes, it reduces the learning curve, because developers are already (by and large) familiar.

Monitoring & Analytics: The Pulse of GitOps Operations

The continuous flow of changes within an environment makes it necessary to keep an eye on the system's health and performance. These observability capabilities are where API management's monitoring and analytics capabilities shine, acting as the pulse-check for GitOps operations.

API management tools provide real-time insights into the performance and health of APIs. They track crucial metrics such as response times, error rates, and throughput. In a GitOps-driven environment, where deployments can be frequent and automated, understanding these metrics becomes vital. It helps teams identify potential bottlenecks, optimize performance, and ensure that the system remains resilient even as changes are continuously integrated.

Beyond performance metrics, analytics delve deeper into API usage patterns. They reveal which APIs are most accessed, which ones might be underutilized, and even which ones could be potential points of failure. In the context of GitOps, where the desired state of the system is codified, these insights help in refining this state, ensuring it aligns with actual usage patterns and needs.

Additionally, in the event of discrepancies between the Git-defined state and the live state, monitoring tools can trigger alerts. These alerts can be invaluable in a GitOps setup, allowing teams to quickly identify and rectify deviations, ensuring that the system remains consistent and aligned with the repository.

It’s also important that the tools involved can easily integrate with other systems. For example, a tool that provides observability based on OpenTelemetry can send data to dozens of other tools that can easily ingest it.

Scalability: Growing with GitOps

As organizations evolve, so do their digital needs. The number of services, applications, and consequently, API calls can expand exponentially. In the GitOps paradigm, where infrastructure and application configurations are continuously synchronized with the state defined in Git, ensuring that this growth is managed efficiently is crucial.

API management is the linchpin in this growth story. It provides the tools and mechanisms to handle an increasing volume of API calls. For example, it can provide load balancing, which ensures that incoming API requests are appropriately distributed across servers so that no single server or service becomes a bottleneck and performance is optimized.

But sometimes it’s not about distributing numerous requests, but limiting them. In a GitOps-driven environment, where changes can be continuously deployed, it's vital to ensure that no service or consumer overwhelms the system with excessive requests. Rate limiting sets a cap on the number of requests a consumer can make in a given time frame, ensuring system stability.

Furthermore, as the infrastructure grows, the Git repository becomes a more extensive catalog of configurations and desired states. API management ensures that, regardless of the scale, every API call, every service, and every microservice aligns with this cataloged state, ensuring that scalability doesn't come at the cost of consistency or performance.

Using GitOps with API management also enables you to scale teams as easily as you scale your infrastructure, because the Pull Requests that enable GitOps to work, enable easy collaboration between multiple developers working on a single system.

Developer Experience: Thriving in a GitOps Ecosystem

In the GitOps landscape, where infrastructure and application changes are driven by Git repositories, the developer's role is pivotal. They're not just coding features or fixing bugs; they're actively participating in the deployment and operational processes. In this context, enhancing the developer experience becomes paramount, and API management plays a crucial role in this enhancement.

API management tools offer a suite of features tailored for developers. One of the standout features is the automatic generation and updating of API documentation. In a GitOps-driven environment, where changes can be frequent and continuous, having up-to-date documentation is invaluable. It ensures that developers, regardless of their familiarity with an API, can quickly understand its functionalities, parameters, and expected behaviors.

Moreover, API management tools often provide sandbox environments. These are isolated spaces where developers can test new API integrations or changes without affecting the actual production environment. In the GitOps world, where changes in the Git repository can trigger deployments, having a safe space to test and validate these changes before they're merged into the main branch can save a lot of time.

And GitOps-enabled API management provides benefits to both sides of the API equation. The API publisher/producer side gets a way to abstract the APIs themselves, as well as a declarative way to deploy them that includes error checking and code review. For the API user/consumer side, in addition to up-to-date documentation, the API developer portal shows what APIs that are available and how to call them.

Auditability and Security: Safeguarding the GitOps Pipeline

In GitOps, every change to infrastructure or applications is initiated through a Git commit, making the Git repository a chronological record of all changes. This inherently provides a level of auditability. However, when APIs come into play, especially in a microservices architecture, the complexity multiplies, and GitOps can help to manage that complexity.

API management tools provide comprehensive logging and tracking capabilities, tracking every API call, every data fetch, and every modification. This granular logging ensures that organizations have a clear trail of who accessed what data, when, and for what purpose. In the context of GitOps, where the desired state of infrastructure is defined in Git, having a clear understanding of how APIs interact with this state is invaluable. It provides insights into potential discrepancies and aids in forensic analysis.

Security, on the other hand, is non-negotiable. As the gateways to data and services, APIs are prime targets for malicious attacks, and centralized API management plays a pivotal role in uniformly enforcing security policies. But without the centralization provided by GitOps, there’s the opportunity for different teams to manage things in their own way, eliminating many of the benefits a GitOps-enabled system provides. Access controls can be defined and enforced, ensuring that only authorized entities can access specific APIs. Moreover, advanced threat protection mechanisms can identify and block malicious patterns or anomalous behaviors, safeguarding the system.

Incident Management: Navigating Challenges in the GitOps World

Even with the most meticulous planning and robust systems in place, incidents are an inevitable part of the software lifecycle. In the GitOps paradigm, where infrastructure and application states are continuously synchronized with Git repositories, the ability to respond swiftly and effectively to incidents is crucial.

GitOps-driven API management plays a pivotal role in this rapid response mechanism. One of the standout features of modern GitOps-driven API management tools is the capability for quick rollbacks. If an API change leads to unexpected behavior or a system outage, teams can use the same Git-based workflows to roll back the entire API platform to a previous, stable state. This rollback isn't just about reverting code; it's about ensuring that the entire ecosystem, from infrastructure to data flows to service integrations, is restored to a known good state.

Furthermore, in the unfortunate event of an incident, real-time monitoring and analytics provided by API management tools become invaluable. They offer insights into the root cause, be it an API overload, a data breach attempt, or a faulty integration. By pinpointing the issue, teams can not only address the immediate concern but also implement preventive measures for the future.

Another crucial aspect of incident management is communication. In the heat of a situation, effective communication with stakeholders, be it developers, operations teams, or end-users, is vital. API management tools often come equipped with notification systems that can alert relevant parties about the incident, its impact, and the steps being taken for resolution, but because GitOps tools already handle incident management for software, applications, and services, adding an additional layer is an unnecessary level of complexity. Ideally, your API management tool will integrate with GitOps to simplify matters.

Embracing the Future with GitOps and API management

In the evolving landscape of software deployment, GitOps has firmly established itself as the blueprint for operational excellence. Yet, it's the seamless integration with API management that truly amplifies its potential. Together, they form a powerful duo, streamlining workflows, ensuring unwavering consistency, and offering unparalleled insights into system performance.

As organizations continue to navigate the complexities of modern development, the harmony between GitOps and API management will be instrumental in driving success. By embracing this synergy, businesses are not only future-proofing their operations but also setting the stage for innovation, growth, and resilience in the face of ever-changing technological challenges.

]]>

Traefik Proxy v2.11 is Now Available! Here are the Latest Updates.

Emile Vauge — Thu, 22 Feb 2024 17:24:58 GMT

On the heels of our announcement last week regarding the new release candidate of Traefik 3.0, we are excited to bring to you Traefik Proxy 2.11, an important update to the 2.x branch underscoring our ongoing commitment to support this branch based on continuous feedback from the community and our customers.

In a nutshell, Traefik Proxy 2.11 extends Redis support to Redis Sentinel, is more efficient dealing with open connections, is safer with sticky cookies, and adds a migration path to Go 1.22, along with several bug fixes.

Let's dig deeper into this latest release and the details of your new features and how to use them.

Redis Sentinel Support

Traefik Proxy has a unique configuration model that allows runtime changes to its dynamic configuration, so you can maintain high availability without dropping open connections by an unnecessary full proxy reload. From the multiple providers available, the Redis provider has been around since Traefik Proxy v2.2.

This release adds support to Redis Sentinel (PR#10245), which provides high availability for Redis when not using Redis Cluster. One of our Traefik Enterprise customers uses it for storing OIDC tokens, which we thought could could benefit the wider community, so we brought it to the open-source version.

Open Connections Improved

Before Traefik Proxy v2.11, HTTP connections would stay open until clients closed them. This behavior could lead to more open connections and higher memory usage. Furthermore, when coupling Traefik Proxy with an upstream Load Balancer, this connection management method prevented an efficient traffic distribution when scaling up Traefik Proxy instances. The Load Balancer would stick to the same instance rather than ensuring a more even traffic distribution by sending requests to the newest instances.

To tackle this, we’ve introduced two new transport options for entrypoints (PR#10247). You can use both options together or choose one or the other:

keepAliveMaxRequests: limits the number of requests each client can make within one connection. Whenever a client sends a request within an open connection, if this request number exceeds the keepAliveMaxRequests threshold, Traefik Proxy will ask for a connection close (sends header Connection:close).
keepAliveMaxTime: limits the time a client can keep reusing the same connection for its requests. Whenever a client sends a request within an open connection, if the elapsed time between the first and current requests exceeds the keepAliveMaxTime threshold, Traefik Proxy will ask for a connection close (sends header Connection:close).

Load balancing is a big topic in networking, we even have a free Traefik Academy video course on it if you want to learn more about the ins and outs of load balancing. There are different ways to achieve it, Weighted Round Robin (WRR) is one of them. With the File or IngressRoute providers, you can assign weights to services and let Traefik Proxy load balance the requests between these available services based on the weights.

When using WRR, a sticky session cookie is used to let the client know which server handles the first response. On subsequent requests, the client should send the cookie with the value set to keep the session alive with the same server. However, the contents of this cookie were previously not encrypted, which could expose the internal IP address of the proxied service.

We’ve listened to the community's voice and provided a fix (PR#10243) to hash the contents of the WRR sticky cookies. This way, it’s not possible anymore to deduct information about the server handling the request.

Other Changes

The / character in router names can now be used through URL encoding/decoding the YAML keys. This change helps avoid 404 errors on the dashboard for aptly named routers (PR#10292).
The ReadHeaderTimeout has been fixed for PROXY protocol by using the ReadTimeout configured on the entrypoint (PR#10320).
We have fixed Kerberos and NTLM authentication with IIS server by “sticking” to the TCP connection with the backend for a single client TCP Connection when it’s necessary (PR#10405).
The middleware IPWhiteList has been deprecated in favor of the new IPAllowList middleware (PR#10249).
The Go library used has been updated to take advantage of the new features embedded in Go v1.22 as well as benefiting from security fixes. This update impacts both the cipher suites and TLS minimum version management, for more information, please read the migration notes.
Our documentation’s readability, wording, formatting, link accessibility, and examples have also been improved.

Wrapping It Up

With 2.11, Traefik is smarter, safer, more efficient, and future proof. We want to thank all community members who gave feedback, asked questions and challenged the team. This makes Traefik much better.

If you are interested in contributing to Traefik Proxy, join us on GitHub, where you can request features/enhancements, help us design the next version, or even get your hands dirty by creating or reviewing pull requests. For more information, check out the Release Notes and updated Docs. If you have questions, pop over to our Community Forum, where you can work with other community members to find your answers. And as always, we look forward to your feedback.

]]>

Announcing Traefik Proxy v3.0 RC1

Emile Vauge — Wed, 14 Feb 2024 17:10:51 GMT

Almost five years ago, in September 2019, we released the previous major version of Traefik: v2.0. Since then, Traefik has been more active than ever. It has achieved a unique, long term traction with more than 3 billion downloads and 700+ contributors. Traefik is ranked in the Top 15 on DockerHub, has 46,000 GitHub stars, and continues a five-year streak as the #1 API Gateway according to OSSInsight.io. What a ride!

We have designed this major release around a few key aspects. First of all, we added support for popular, emerging technologies—WebAssembly (Wasm), OpenTelemetry, and Kubernetes Gateway API. And then, of course, because the cornerstone of a proxy is the routing and the security, we revamped some key parts of the routing rules and added support for some leading edge technologies like HTTP/3, SPIFFE, and Tailscale.

Another critical aspect of any major release, which by design contains breaking changes, is to provide the smoothest user experience migrating from the previous version. Not only are we providing a complete migration guide, but we ensured that the transition process from the previous v2 to the new v3 has been streamlined, by ensuring backward compatibility with v2 syntax while offering a progressive path for adopting the v3 syntax.

Finally, Traefik is and has always been a community-driven project. A huge effort has been made on improving the user experience for contributors with the help of strong and clear processes within a welcoming and respectful environment.

I’m extremely proud to see that this project, born 8 years ago, has become such a critical piece of the modern cloud-native infrastructure stack, with the help of such an amazing community.

Alright, let’s jump into the details of v3 🙂

Ease of Migration

A few years back, when we introduced Traefik v2 to the world, the team was so excited to improve the project that we underestimated one thing: migration experience from v1.x to v2.x. Not that our migration plan was empty, far from that: we wrote an extensive migration guide, we even provided a migration tool to help users convert their configuration to the new format. But that wasn’t enough and let’s be honest, the adoption of this new version took longer than what we expected. Clearly, this wasn’t something we wanted to see again with Traefik v3. A major version means breaking changes obviously, but that shouldn’t imply a painful migration experience.

With Traefik v3, we are introducing a streamlined transition process from v2, ensuring backward compatibility with v2 syntax while offering a progressive path for adopting the v3 syntax, effectively overcoming the challenges encountered in previous migrations.

Here are the guidelines we followed:

Obvious first thing is to provide a detailed migration guide describing every change that need to be made from v2.x to v3
Breaking changes on the static configuration are OK as soon as we provide user friendly logs when v2.x options are still used and link to the new v3 format
The dynamic configuration will support both v2.x and v3 configuration formats. Imagine a scenario with 200 ingress resources to migrate. You will be able to use Traefik v3 without any change, and you will be able to progressively upgrade your ingresses to the new format.

All in all, this should be pretty straightforward for Traefik users to adopt this new major version and we will carefully listen to your feedback to further improve upon the migration guide.

Routing & Security

One of the fundamental purposes of Traefik is to route network traffic in a secure and intelligent manner. This is the core part of any proxy, but Traefik has always been the cutting edge. And this v3 release doesn’t disappoint with a huge number of new features and improvements.

Let’s start with HTTP, the backbone of the Internet. Since HTTP/2 was approved in 2015, a lot of work has been done on defining the next milestone: HTTP/3. And this one is big as it doesn’t rely on the good old TCP anymore, but instead UDP (being based on QUIC, a new transport protocol developed by Google in 2012). And it comes with many benefits like enhanced performance, faster connection establishment, simpler encryption, etc. HTTP/3 is now out of experimental and fully production ready in Traefik! Thanks to the maintainers of the Quic-Go Project, and especially Marten Seeman, for their efforts in developing this library and enabling this advancement.

Another key technology that is being widely used lately in cloud-native environments is gRPC. gRPC is a modern Remote Procedure Call (RPC) framework that can run in any environment. It uses Protocol Buffers as the interface description language, and provides advanced features such as bidirectional streaming and flow control, cancellation, and timeouts. Traefik v3 now supports gRPC-Web, which adds the capability to call a gRPC service from a web app within a browser. Additionally, Jeremy Jacque added gRPC healthchecks support to Traefik.

Deploying dozens or hundreds of services in production comes with many challenges. One of them is enabling workloads to prove their identity to trust each other. Enter SPIFFE, the Secure Production Identity Framework For Everyone Project defines a framework and set of standards for identifying and securing communications between application services. Thanks to Julien Levesy, Traefik now supports SPIFFE mTLS with its backend servers!

Traefik had a routing syntax that was already very powerful, but we figured out that some parts could be further improved and unified to make the user experience even better.

If you need to protect websites within a Tailscale network, Traefik can now request TLS certificates from the Tailscale API (provisioned with Let’s Encrypt)! The long-awaited Brotli compression algorithm support was added by Greg Linton which in short means quicker website loading. Michael Kuhnt brings SNI routing with Postgres STARTTLS. And finally, you are now able to fully configure your TCP server transports.

As you can see, the work that has been done is just insane and reinforces the pioneering routing & security capabilities of Traefik while sticking to the original virtues of a smooth user experience and top-notch performance.

Augmented Traefik

Back in 2019, with the release of Traefik v2, we had introduced an extension engine, Yaegi, that enabled many to develop middleware (or providers) for their context. To this day, we can count more than a hundred middlewares made available through the catalog, plus many more that are kept private.

Yaegi is an extremely powerful engine whose boundaries go beyond Traefik itself. Given its nature though, Yaegi still requires much work to support the full Go specification and to keep up with every release of the language. This can sometimes result in difficulties in adding very challenging features.

Another extension technology we have been looking at for quite some time is WebAssembly (Wasm). Until today, we were not satisfied with current ABIs (Application Binary Interface) like proxy Wasm because they were closely designed with other technologies in mind (Envoy), making it hard to leverage in our context. With the release of http-wasm providing a standard ABI designed to match the HTTP handler mechanism built-in with Go, the game changed. Thanks to this technology, in less than 3 months, a contributor jumped in and tackled the necessary work.

You read it correctly: you can now leverage Wasm to develop plugins in traefik, removing any existing hurdle and expanding the field of possibilities.

Many thanks go to Jesse Haka for the bigger chunk of the work involved, and José Carlos Chávez for the review effort.

Everything, Everywhere, All-at-Once Observability

With its central place in your architecture, making decisions on your behalf about where to route requests, Traefik made observability one of its core features since v1, supporting many vendors for metrics and tracing analysis.

With the advent of OpenTelemetry, it sounded natural that Traefik would pave the way for adoption and encourage our users to migrate to state-of-the-art toolings. Once again, we can thank Jesse Haka for this contribution.

For Metrics, where Traefik already supports a large panel of vendors with wide adoption (Datadog, Prometheus, and others), we added OpenTelemetry as a cherry on the cake: you can start migrating to a newer stack without compromising your well-established monitoring toolchain.

Tracing was a different story: Traefik had been an early supporter of two standards (OpenSensus and OpenTracing) that have now merged into one. The adoption is already there, and the migration path is well known. As a result, we dropped support for the previous standards in favor of OpenTelemetry.

But OpenTelemetry is not the only trick Traefik 3 has in its hat around observability. For gRPC enthusiasts, we added support for proper health checks. A one-liner that will change the lives of many, at least at work or in the lab. Thank you Jeremy Jacque.

And my own personal favorite, speaking to the geek side of us, we changed the logger system for a zero-allocation (read “better performing”), structure-friendly, rotation-aware logger. This apparent small tweak will make the whole experience of understanding and debugging your system much easier.

Team Player Traefik: Kubernetes & Friends

For those of you Kubernetes heads, chances are you’ve always seen Traefik as your ingress controller of choice, and that you have adopted IngressRoutes as a clean way to get rid of the limitations of the good old (especially old) Ingress specification. This opened the door to many benefits: declarative and more routing options to name two.

Still, Kubernetes is evolving, and Traefik was part of the Gateway API vanguard, supporting the first alphas. It’s no surprise that we’re bringing full support of the Gateway API specification that is now officially out.

The power of Kubernetes also lies in its vendor ecosystem, allowing DevOps to pick the best tools available. In that regard, Traefik has always been a team player, supporting many vendors, third-party orchestrators, observability components, and key-value stores to name a few. Based on popular demand amongst users and customers alike, we are thrilled to share that Traefik v3 comes with better support for Hashicorp’s Consul (for meshing that requires special TLS management) and Nomad (for namespace support). On the latter, we can thank Thomas Harris.

Strengthen Community & Open Source Values

Maybe the best way to end the list is a non-feature that maintainers adopted wholeheartedly: our commitment to make the Traefik v3 experience the best it can be, from using Traefik to actively contributing to the project.

Yes, Traefik has always been open-source and contributions have been at the forefront of each version. Still, despite a strong commitment to sorting each incoming issue and PR, we concurred that maintainers could better document their thoughts around these contributions. Not only for us as a group of maintainers working async but as a testimony to enable contributors for action.

As a consequence, we reinforced our stance to ensure several key points:

Always spend the extra time so each interaction is welcoming, open, and clear
Always prioritize contributions over our own agenda
Always document the ongoing work for easy collaboration
Always create proposals for upcoming features

These small tweaks almost instantly empowered some key contributors who brought two of the biggest features of Traefik v3, namely Wasm support and OpenTelemetry.

We just can’t wait to welcome even more contributors and additional features in the near future.

Next Steps

Like in the previous major release v2, Traefik v3 shows the importance of the community. Every feature that has been discussed today was born from a user asking for it, reporting a bug, or just asking for something “better”. Critical new features have been completed by external contributors, and, to me, this demonstrates a healthy and vibrant open source project.

A release candidate is the beginning of a release process. To make it to GA, we will need your feedback, comments, and ideas to make this landing as smooth as possible. To get involved, you can download Traefik v3 from DockerHub and help us shape it via our OSS Project on GitHub.

Additionally, if you’d like to receive regular updates via email as v3 heads towards GA, sign up here.

Finally, I want to send a huge thank you to all contributors. Your help is invaluable.

See you on GitHub!

Useful Links

Traefik 3.0 RC1 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Enhancing API Observability: Traefik Hub, OpenTelemetry, and the New Era of Data-Driven API Management

Immánuel Fodor — Thu, 07 Dec 2023 20:30:59 GMT

In today's interconnected digital landscape, application programming interfaces (APIs) serve as the invisible architects, shaping the way we interact with technology. They are the unsung heroes behind the scenes, propelling innovation, driving new business models, and seamlessly delivering products and services across diverse channels. As the backbone of the digital era, APIs command a staggering 54% of total requests and are growing twice as fast as web traffic, underscoring their pivotal role in the evolution of technology.

The API market, once valued at $5 billion in 2022, is forecasted to skyrocket to $40 billion by 2030. This meteoric rise is fueled by the fact that two-thirds of companies are either embracing or fully adopting API-first strategies. As the world becomes increasingly connected, APIs are not just a technological necessity but a strategic imperative for businesses looking to thrive in the digital age.

However, with great impact comes the need for visibility and control. Observability has become paramount in the era of modern and distributed systems, as well as for the APIs connecting them. This paradigm shift from monitoring with simple metrics to comprehensive telemetry analysis is vital for today’s issue resolution, performance optimization, and user satisfaction.

Enter OpenTelemetry, a game-changer in the realm of observability.

OpenTelemetry: The Observability Enabler

Designed to streamline telemetry data collection from distributed systems, OpenTelemetry replaces multiple vendor-specific agents, simplifying and unifying dynamic systems telemetry data. It provides a unified standard, ensures consistent instrumentation across various programming languages, and offers enhanced insights into application use, health, and performance. This becomes especially important as organizations deal with the complexities of microservices and cloud-native architectures.

In this blog post, we will delve into the critical role of OpenTelemetry in achieving comprehensive observability and explore how it addresses the challenges posed by modern software architectures and APIs. For context, API gateways play a pivotal role in managing and securing API traffic, together with the need for visibility into APIs’ health, usage, and performance. Whether with issue alerts, event correlation for rapid issue fault-domain isolation, or cross-domains causations and root-cause analysis, OpenTelemetry emerges as a key enabler in ensuring the reliability and performance of API-driven systems.

Understanding OpenTelemetry

OpenTelemetry is an open-source project that emerged in 2019 combining the OpenTracing and OpenCensus projects. It quickly became a game-changer unifying and streamlining the collection of telemetry data from distributed systems across tech stacks. Born out of the necessity for a unified approach governed to date by multiple proprietary approaches, OpenTelemetry provides a standardized, open telemetry framework for comprehensive visibility into health, usage, and performance across infrastructure, services, and applications.

Metric types in OpenTelemetry serve as the bedrock for understanding system performance and behavior:

Counter: Tracks continuously increasing numeric values, ideal for counting occurrences of events.
UpDownCounter: A versatile counter that accommodates both positive and negative changes, offering flexibility in tracking fluctuations.
Gauge: Offers a snapshot of a value at a specific moment, providing insight into the current state of a system.
Histogram: Captures the distribution of observed values over time, offering a nuanced view of data spread and central tendency.

On the other hand, metric labels play a pivotal role in enhancing the granularity and context of metrics. Labels are key-value pairs associated with metric data, allowing for more nuanced and specific insights into system behavior.

Let's consider a practical example in the context of an airline booking system. Imagine you want to monitor the usage of the APIs with a metric called API Request Total (api_requests_total) which can be an instance of the counter metric type.

The following labels could come in handy:

Code: To categorize requests based on the HTTP response code (e.g., "200" for successful requests, "404" for not found, etc.)
Method: To differentiate requests based on the HTTP method used (e.g., "GET," "POST," etc.)
API Name: To specify the name of the API being accessed (e.g., "flight-api")
API Version: To track the version of the API (e.g., "v2")
Token Name: To identify the API key used for authentication (e.g., "my-test-token")

Example metric labels in OpenTelemetry for this use case:

For a specific metric instance, you might have labels like {"code": "200", "method": "GET", "api_name": "flight-api", "api_version_name": "v2", "token_name": "my-test-token"}
Another instance might have labels like {"code": "404", "method": "POST", "api_name": "ticket-api", "api_version_name": "v1.1", "token_name": "prod-app-key"}

Now, when analyzing the example api_requests_total metric, you can gain specific insights thanks to labels:

HTTP Code Analysis: Compare the total count of requests based on HTTP response codes to identify patterns in successful or error-prone API interactions.
HTTP Method Distribution: Understand the distribution of requests based on HTTP methods to optimize API resources accordingly.
API Name and Version: Monitor the usage of different APIs and versions to plan for updates, deprecations, or optimizations.
Token Usage: Track the utilization of different API keys for authentication, identifying potential security concerns or optimizing key management.

At the end of the day, the more metrics and labels you have, the more analysis you can do.

OpenTelemetry Metrics in API Management

OpenTelemetry metrics has great potential to better observe APIs, however, the current adoption seems to lag.

Moreover, even in instances where API management solutions do offer observability features, the availability of regular metrics remains limited. Many solutions rely on user interface (UI)-based interactions, often leading to what can be termed as "ClickOps"—a reliance on manual, point-and-click operations for metric visibility. This approach, while providing some insights, falls short of the automations and cross-domain analysis promised by OpenTelemetry.

In the API management landscape, where the reliability and performance of APIs are mission-critical to the business, the gap in comprehensive observability can hinder proactive issue resolution, optimization, and strategic decision-making.

Do not fly blind!

Use Cases of OpenTelemetry Metrics in API Management

Real-Time Monitoring and Alerting

Leverage OpenTelemetry metrics to establish thresholds for vital parameters such as API response times, error rates, and request counts. This enables real-time monitoring, allowing organizations to promptly identify deviations from expected behavior.

Additionally, integrating OpenTelemetry metrics with GitOps workflows to correlate API performance changes with code deployments facilitates the swift identification of issues stemming from recent changes. This streamlines troubleshooting and ensuring the reliability of API services.

Capacity Planning and Scaling

Utilize OpenTelemetry metrics to gain comprehensive insights into resource utilization, enabling effective capacity planning. Identify bottlenecks, predict resource demands, and optimize infrastructure to meet current and future workload efficiently.

By analyzing historical OpenTelemetry metrics, organizations can forecast trends and plan for future workloads. This proactive approach ensures that API management systems are adequately prepared to handle increased demand, preventing potential performance issues.

Performance Optimization

Dive deep into OpenTelemetry metrics to identify patterns of errors and latency issues. Whether it's pinpointing error-prone API endpoints or analyzing the performance impact of specific client types, OpenTelemetry metrics provide the granularity needed for precise optimization.

OpenTelemetry Metrics in Traefik Hub

Traefik Hub showcases a wealth of OpenTelemetry metrics and labels that redefine how organizations monitor, manage, and optimize their API infrastructure. The expansive range of metrics, totaling over 20, coupled with more than 15 labels, places Traefik Hub at the forefront of empowering users with unparalleled insights.

The four key categories of metrics exposed by Traefik Hub:

Dataflow-related metrics of exposed APIs: These metrics offer insights into the health, performance, and interactions of APIs, allowing organizations to precisely monitor and optimize the flow of data through their infrastructure.
API management object metrics: A holistic view of key objects including but not limited to the number of APIs, users with API access on gateways published by the agent, API keys, and more. This category of metrics allows organizations to track the usage and impact of API-related objects, enabling efficient management and strategic decision-making.
Licensing metrics for usage tracking: This category enables organizations to track their own usage against the object counts included in the license. This functionality empowers users to manage licensing efficiently, ensuring compliance while optimizing the utilization of available resources.
Ingress-level metrics inherited from Traefik Proxy: Building upon the capabilities of Traefik Proxy, Traefik Hub inherits ingress-level metrics that provide a comprehensive understanding of network traffic patterns and behavior.

We recently enhanced our OpenTelemetry metrics by adding support for API versions, so you can stay on top of your API version proliferation. Learn more here.

Your APIs, Your Metrics, Your Tools

We don’t believe in vendor lock-in. In the quest to embrace a flexible, interoperable observability landscape, OpenTelemetry emerges as a beacon of liberation. Leveraging the rich ecosystem around OpenTelemetry not only ensures adaptability but also opens up a realm of possibilities for metric manipulation, integration, and visualization.

The OpenTelemetry Collector stands as a versatile tool in the arsenal of observability enthusiasts. It goes beyond collection, offering the capability to manipulate metrics dynamically. Through transformations, conversions, filtering, and enrichment, organizations can tailor their telemetry data to specific needs. This flexibility allows for seamless adaptation to evolving requirements and diverse use cases.

Image Source: https://opentelemetry.io/docs/collector

The latest release candidate of Prometheus marks a significant stride towards embracing open standards. With native OpenTelemetry metrics ingestion support, Prometheus becomes a part of the larger OpenTelemetry ecosystem. This integration not only simplifies the process of incorporating OpenTelemetry metrics into existing setups but also lays the foundation for a more unified and standardized observability approach usable by anyone since open-sourced.

To truly liberate observability, visualization tools play a crucial role. For example, users can leverage the full spectrum of Grafana's battle-tested features to visualize, analyze, and get insights from their APIs metrics data. Additionally, the embrace of open standards by 3rd party tools ensures that organizations are not tethered to proprietary solutions, fostering a vibrant ecosystem of choice and flexibility to cater to multiple teams and organizations needs.

Grafana Dashboards Example and Other Resources

We’ve made Grafana dashboard examples that you can extend public, delivering insights with a high-level API overview, a detailed per-API, or per user-specific drill-downs.

All API Overview: Get a comprehensive overview of your entire API landscape, including traffic, response times, error rates, and more. Monitor the overall health and performance of all your APIs, enabling swift identification of global trends and anomalies and where to look deeper.

Comprehensive overview of API traffic, response times, error rates, etc.

Per-API View: Dive deep into the specifics of each API, analyzing traffic patterns, latency, and error rates for targeted understanding and optimizations, with full visibility into API versions.

Per-API view of traffic patterns, latency, and error rates.

Per-User/Per-Token Drill-Down: Track the usage patterns of individual users or API tokens, gaining granular visibility into their interactions with your APIs. Analyze the usage and health of specific API consumers on API requests and responses, ensuring optimal security and resource allocation.

Per User/Per Token drill-down into API interactions.

Kick start your observability journey with our step-by-step tutorial, incorporating the Grafana sample dashboards and our static CRD analyzer for early issue detection, embracing a GitOps approach to modernize API operations. And for in-depth information, see our documentation.

Wrapping It Up

Metrics are a key pillar of modern API observability. OpenTelemetry, thanks to its standardized approach, stands as a game-changer, providing invaluable visibility into API health, usage, and performance. Looking ahead, Traefik Hub is committed to extending OpenTelemetry to traces and logs, promising even more complete observability.

]]>

API Versioning with Traefik Hub: Smooth Transitions, Seamless Innovation

Immánuel Fodor — Mon, 20 Nov 2023 16:13:03 GMT

API versioning is a fundamental practice in software development that involves the creation and maintenance of distinct versions for an application programming interface (API). The primary objective is to safeguard the functionality of applications dependent on the API from disruptions caused by changes or updates.

This practice is paramount for preserving backward compatibility, ensuring that applications using older API versions remain functional. Meanwhile, it facilitates the seamless integration with newer clients, allowing them to leverage the enhanced features of the updated API.

However, API versioning is not just a technical concern; it has direct implications for the success and growth of your business. It helps you maintain developer satisfaction, support diverse user needs, innovate, and manage risks effectively. By providing a reliable and consistent experience for API consumers, you can build a loyal user base, foster a thriving ecosystem, and ultimately drive business growth and revenue.

Before we dive into how Traefik Hub improves API versioning, let’s first take a look at why it’s important for both API producers and consumers.

The Importance of API Versioning

For API Producers

In the fast-paced world of API development, the challenge lies in innovating with velocity while avoiding disruptive changes. For API producers, the absence of a common language among different teams discussing changes can result in a lack of context, leading to errors as interpretations vary. Accelerating the development process may intensify these errors, while slowing down compromises the speed of launching new features. In this regard, API versioning is a crucial solution to strike a balance between innovation and resilience.

For API Consumers

On the consumer side, API versioning addresses the challenges of keeping pace with releases and changes that can lead to integration issues and potential disruptions. With a well-defined versioning strategy, API consumers are more informed and can adapt to new releases at their own pace. This not only keeps existing users satisfied but also attracts new users who seek a reliable and future-proof API.

Ultimately, API versioning acts as a form of insurance for both producers and consumers. It mitigates business risks by providing a controlled mechanism for introducing changes, minimizing the chances of costly outages and revenue loss. Additionally, adhering to versioning practices helps ensure compliance with legal and regulatory requirements, safeguarding the business against potential legal challenges and associated penalties.

How Traefik Hub Solves API Versioning Challenges

Traefik Hub transforms the API versioning landscape by providing a unified solution that caters to both API producers and consumers. For API producers, Traefik Hub offers a common actionable language, streamlining the understanding, expression, and management of change through innovative approaches like CRDs—a.k.a., Custom Resource Definitions—and GitOps. This empowers developer teamwork at scale and speed while minimizing risks of breaking changes.

On the other side of the spectrum, Traefik Hub prioritizes the needs of API consumers by emphasizing ease of use and clarity. The platform introduces API Version CRDs coupled with internal semantic versioning and external flexible version matchers, offering consumers a straightforward way to comprehend and interact with APIs. This not only enhances the developer experience but also ensures that changes are clearly communicated and can be adopted without friction. Traefik Hub’s API Developer Portals expose all API versions clearly, promoting efficient information flow between API producers and consumers.

In order to facilitate efficient communication and collaboration, CRDs are used to standardize changes. This common language ensures that everyone involved understands the modifications, reducing the risk of errors and misinterpretations. As a result, teams can move faster without sacrificing the quality and stability of the API. This aligns with the need for speed without compromising accuracy, enabling a smoother development process.

API Versioning Your Way

We believe in letting developers seize control, making their own choices. We don't impose a rigid versioning structure; we empower you to define the version matchers that suit your unique needs—also ensuring that versioned and unversioned APIs can coexist harmoniously.

Traefik Hub offers four API versioning methods, giving you the freedom and flexibility to choose the best approach for your particular situation:

Path-based (/v3): The traditional route for versioning, with a clear, user-friendly path to your API versions.
Query-based (?v=3): A versatile option that embeds the version directly into the API request, simplifying integration.
Media type-based (Accept/Content headers): This approach leverages headers, offering precise control over content negotiation and version selection.
Header-based (any custom header): The ultimate in customization, letting you set up your own headers to manage API versions.

To provide the best possible product, we looked around on the market, and realized most other API management tools offer only a subset of these methods or even stick to just one, limiting you to follow the path they chose.

Mix and Match Versioning Methods

For Traefik Hub, it's not just about having multiple options. You can combine them in ways that make sense for your project. Traefik Hub provides the unique capability to mix and match these versioning methods with AND/OR relations, giving you the power to craft the perfect strategy to meet your specific needs.

Tired of using numbers for publishing versions? Use calendar dates, cheese names, or anything you like. We don’t limit you; it’s your API, it’s your API governance, and your versioning convention.

Structured Flexibility

Though the offered freedom is great, it shouldn’t sacrifice structure and maintainability. Traefik Hub uses semantic versioning internally to keep track of the version changes and to allow you to mark what is the current API version, what are the older ones, and what are future releases.

This flexibility in interacting with your APIs on the request level paired with the structured internal representation allows you, your team, and your clients to always stay on top of the API version proliferation.

And in case you want to expose an always up-to-date version or a fallback version, a catch-all route can be marked for a versioned API, which still allows unversioned access to the underlying service. It can be beneficial to test the latest functionality or to provide a fallback version to clients that access the API without any version matcher (i.e., access a version without explicitly specifying the version anywhere in the request).

Version Control for Headers

Finally, Traefik Hub’s API and API Version CRDs allow you to seamlessly alter regular and security headers on both APIs and API versions. Add, remove, and manipulate request and response headers on a per-version basis to fortify your API infrastructure.

Use Case: Traefik Airlines

Let’s take a look at Traefik Hub’s API versioning capabilities in action. Traefik Airlines, our fictitious company, is a modern airline company that offers a variety of digital services to its customers and partners. To meet the evolving needs while maintaining business stability, Traefik Airlines must implement a robust API versioning strategy. Let's explore how this can be achieved by using Traefik Hub API management.

Traefik Airlines provides a RESTful API for customers to book flights and manage reservations. Customers can retrieve real-time flight information such as schedules, delays, and gate changes. The API offers mobile check-in, boarding pass retrieval, and gate information. Customers can also interact with an API that manages loyalty points and rewards.

Additionally, third-party travel agencies integrate with Traefik Airlines to offer flight booking services to their customers. They also need to maintain integration with airport systems for baggage handling, gate allocation, and passenger tracking. Collaboration with payment providers is with the utmost importance to process transactions securely.

As we can see, there are many clients, many APIs and many developers supporting the company’s digital infrastructure. The demanding environment screams for using API versioning to satisfy all needs, provide business continuity and let the airline expand at the same time, deploying new API releases on a regular basis.

Flexible API Versioning with CRDs

Let’s say, for example, that Traefik Airlines has already deployed Traefik Hub to their Kubernetes clusters running their microservices, and they are now ready to define APIs and API versions.

Let’s pick the Customer API as an example.

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: customer-api
  namespace: apps
  labels:
    area: customers
    module: crm
spec:
  pathPrefix: "/customers"
  currentVersion: customer-api-version-one

---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: customer-api-version-one
  namespace: apps
spec:
  apiName: customer-api
  release: 1.0.0
  title: "v1 with URI path"
  routes:
    - pathPrefix: /v1
  stripPathPrefix: true
  service:
    name: customer-app-v1
    port:
      number: 3000
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000

Here they define the first version of the Customer API.

The API CRD is labeled for later reference in an API developer portal providing documentation, API access policies to allow users API consumption, API gateways to publish it to the defined audience, API collections to create virtual API groups, or API rate limit policies to secure the backends from rouge clients or malicious actors. The CRD also defines a path prefix for the whole API containing all versions, and references the current version.

The API Version CRD defines the first version of the API, internally versioned as 1.0.0 and exposed as /customers/v1 in the URI path, from which the version is stripped when passing the request to the microservice exposing the API. Here, they could have used any text, date, or even a cheese name to publish the API. If they completely omitted the routes block, the version would have become a catch-all route, accepting any unversioned traffic. The service also exposes an OpenAPI specification which will be attached to the API version on the developer portal.

Let’s say they would have opted for another API versioning method. How could they implement it with the provided flexibility but still maintaining a clear structure?

routes:
    - queryParams:
        my-version-param: "1"

In this case the API version is exposed at /customers?my-version-param=1. Need to expose a version for another language, but the two should be applied at the same time?

routes:
    - queryParams:
        my-version-param: "1"
        lang: "hu"

Now the version is available at /customers?my-version-param=1&lang=hu. Add as many parameters with any value as you like.

Need to expose the same version with different methods? We’ve got you:

routes:
    - queryParams:
        v: "1"
        lang: "hu"
    - headers:
        v: "1"
        lang: "hu"
    - headers:
        Content: "application/vnd.example.v1+json"
        lang: "hu"
    - pathPrefix: /v1
      queryParams:
        lang: "hu"

Now the version is available at the following ways:

Using query params at /customers?v=1&lang=hu
Using the custom headers v: 1 and lang: hu at /customers
Using media-type type versioning with extending the standard content negotiating header Content: "application/vnd.example.v1+json" and a custom lang: "hu" at /customers
Using the path prefix and a query parameter at /customers/v1?lang=hu

You can mix and match all these route options in an API version, both the CRD and the web user interface displays it in a structured, easy-to-digest manner.

In case Traefik Airlines need to publish the 2nd version of the API, they deploy the new supporting service with the corresponding new OpenAPI specification, reference it in a new API Version CRD, and update the current version in the API CRD. And that’s it, the new version is published. The main changes to the first version are emphasized below:

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: customer-api
  namespace: apps
  labels:
    area: customers
    module: crm
spec:
  pathPrefix: "/customers"
  currentVersion: customer-api-version-two   # points to new version
  
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: customer-api-version-two             # new version CRD
  namespace: apps
spec:
  apiName: customer-api
  release: 2.0.0                             # bumped internal version
  title: "v2 with URI path"                  # new title
  routes:
    - pathPrefix: /v2                        # new version matcher
  stripPathPrefix: true
  service:
    name: customer-app-v2                    # points to the new service
    port:
      number: 3000
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000

None of the configuration building on top of the API needs to be altered, the API access policies, rate limits, collections, etc. stay the same. Publishing the new version is propagated to the API gateway and the portal transparently. Now clients have a backward-compatible way to adapt to the new version without any service disruption to the existing consumer configuration.

You can find even more information about API versioning in our documentation.

Wrapping It Up

In today's fast-paced software development landscape, API versioning is not just a technical necessity but a strategic imperative for a thriving and sustainable API ecosystem. As software development velocity accelerates, API versions provide stability, ensuring existing integrations remain intact with backward compatibility. While you evolve and improve your APIs, it gives your consumers a predictable environment to adopt innovations at their own pace.

Traefik Hub API management simplifies transitions, minimizes disruptions, and offers a smooth migration path for legacy APIs by extending rather than replacing non-versioned APIs, allowing coexistence. With this internal semantic versioning system and flexibility in versioning methodology, Traefik Hub accommodates diverse client needs and enhances the overall experience for both developers and clients.

]]>

How to Install Traefik via Azure Marketplace

Nikolas Sachos — Mon, 26 Jun 2023 08:27:59 GMT

If you're a developer or DevOps engineer looking to integrate Traefik Proxy and Traefik Enterprise with Azure Kubernetes Service (AKS), you're in the right place! This guide provides a simple and intuitive walkthrough on how to install Traefik Proxy and Traefik Enterprise through the Microsoft Azure Marketplace, granting you access to the many benefits of the Traefik product suite.
Traefik is one of the preferred choices for running an Ingress Controller and API Gateway on Azure. Recommended by Microsoft in their baseline architecture for Azure Kubernetes Service (AKS), Traefik offers dynamic configuration, middleware support, and seamless compatibility with AKS workloads.

Step-by-Step Instructions for Installing Traefik Proxy via Azure Marketplace

Before you get started, ensure that you have the following prerequisites:

An active Azure subscription.
An existing Azure Kubernetes Service (AKS) cluster.
Helm installed on your local machine.
Azure CLI installed on your machine (for cluster access).

After ensuring you meet the necessary prerequisites, you can move forward with the installation process:

1. First, navigate to Microsoft Azure Portal. Use the top search bar, search for Traefik Proxy, and click on the correspondent link under the Marketplace section.

2. Use the top search bar, search for Traefik Proxy, and click on the correspondent link under the Marketplace section.

3. Next, click the Basics tab, and choose your Subscription, Resource group, and Instance details.

4. Click on Cluster Details, and choose your AKS Cluster Name.

5. In the Cluster Extension Details tab, fill in your Cluster extension resource name by filling in a numeric 6 to a 30-character-long alphanumeric value.

Note: In the Cluster Extension Details tab, fill in your Cluster extension resource name by filling in a numeric 6 to a 30-character-long alphanumeric value.

6. Finally, on the "Review + create tab", click the Create button at the bottom left of the page.

7. After completing the installation process, the following confirmation page will be displayed showing that the deployment is complete and one Traefik Proxy instance is running in your cluster.

To verify that the installation runs smoothly, we can list the pods in the Traefik namespace by running the command:

kubectl get pods -n traefik

The output verifying the Traefik pod is running as expected should be similar to:

NAME                        	            READY   STATUS	RESTARTS      AGE
traefik-5dbc588874-bfbpv    1/1 	   Running            0      	  5m

In order to get the Loadbalancer IP and verify that our services will be accessible from the outside world, we can list the Traefik services by executing the command:

kubectl get services -n traefik

Your output should show an External IP address similar to:

NAME        TYPE               CLUSTER-IP      EXTERNAL-IP    PORT(S)          AGE
traefik     LoadBalancer   10.0.93.113        20.4.131.79	80:32159/TCP,443:32202/TCP   5m

Your Traefik Proxy deployment is now active and operational. It's ready to accept connections on the Loadbalancer External-IP on ports 80 and 443.

8. Once your Traefik installation is running smoothly, the next step is to set up your routing rules and test them. This can be achieved by using the IngressRoute CRD to define your routes, services, and ports. The code below demonstrates how a Service and an IngressRoute can be applied to a cluster with a single command:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: apps
---
apiVersion: v1
kind: Service
metadata:
  name: my-service-name
  namespace: apps
spec:
  selector:
    app: my-app
  ports:
    - name: http
      protocol: TCP
      port: 3000
      targetPort: 3000
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: myIngressName
  namespace: apps
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: my-service-name
          port: 3000
  tls:
certResolver: letsencrypt

Once you run the command above, the output will be

namespace/apps created
service/my-service-name created
ingressroute.traefik.containo.us/my-ingress created

and it will create a new namespace named apps, create a service in that namespace, and set up the routing rules for incoming traffic to this service.

9. Now you can configure access to the Traefik dashboard that will provide a visual insight into the health, performance, and routing configuration of your Traefik instance. To achieve this, we can use an IngressRoute CRD and define the appropriate metadata, entryPoints, and routes. Create a YAML file named dashboard-ingress.yaml, and paste the following content:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefik
spec:
  entryPoints:
	- web
	- websecure
  routes:
  - kind: Rule
	match: Host(`example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
	services:
	- name: api@internal
  	kind: TraefikService
  tls: {}

Do not forget to replace example.com with your preferred domain name.
Now to create the routing rules, apply the file by executing the command:

kubectl apply -f dashboard-ingress.yaml

To confirm that the dashboard has been deployed, your output should be:

ingressroute.traefik.containo.us/traefik-dashboard created

You can access the dashboard at https://example.com/dashboard/. Below is a screenshot highlighting the various HTTP, TCP, and UDP Routers, Services, and Middlewares running across a single cluster.

An interesting feature of the Dashboard is that when an error occurs in a Pod (referred to as Server within the interface), then automatically, a part of the graph will become red, notifying you about the issue while also giving you the exact error message.

10. With our Dashboard operational, let’s now use the Traefik automated service discovery. This is a built-in feature that dynamically discovers and configures routes to services as they are created (or destroyed) in your environment. We can deploy a simple Go-based webserver application and verify the service discovery by watching the changes live on the Dashboard.
Go ahead, and create a file my-app.yaml, and paste the code below:

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: my-namespace
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: my-namespace
spec:
  selector:
    app: whoami
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: my-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        ports:
        - containerPort: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: my-namespace
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: whoami
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls: {}

And apply the manifest by executing the command:

kubectl apply -f my-app.yaml

Your output should look similar with:

namespace/my-namespace created
middleware.traefik.containo.us/prod-rate-limit created
service/whoami created
deployment.apps/whoami created
ingressroute.traefik.containo.us/prod-route created

The namespace, middleware, service, deployment, and ingress routes are now created, meaning your application is now deployed and ready to be used. After a few seconds, you will notice that your Dashboard interface displays two new Routers, one new Service, and one new Middleware in the HTTP section.

By clicking on the Explore link on the Services box, you will see the new Service name my-namespace-prod-route-(randomid)@kubernetescrd, with Type loadbalancer having one Server (the Nodejs pod) using the kubernetescrd Provider. The randomid is a random string that Traefik produces internally to avoid possible conflicts.

At this point, we have completed all necessary steps to enable our Traefik dashboard and deployed an example application to see its effects displayed live on the screen. This will conclude our installation guide, and from this point on, you may apply any other manifests like you normally would.

Now that we've covered the installation process for Traefik Proxy via Azure Marketplace let's explore why you might want to consider Traefik Enterprise for your AKS deployment. If you're running mission-critical applications in production, we strongly recommend considering Traefik Enterprise. Why? Traefik Enterprise offers a robust suite of features designed specifically to meet the demands of such important environments. It provides an array of advanced security measures, sophisticated traffic management capabilities, high availability, scalability, and comprehensive monitoring and troubleshooting tools.

Why Choose Traefik Enterprise for Your AKS Deployment?

Traefik Enterprise is a robust and versatile API Gateway and Ingress solution designed to cater to the needs of enterprises. Its numerous features and benefits make it an ideal choice for Azure Kubernetes Service (AKS) deployments:

Security: Traefik Enterprise offers security features such as IP Allow List, Access Control protocols support, and Distributed Let’s Encrypt. It separates responsibilities between two planes for enhanced security.
Traffic Management: Traefik Enterprise provides dynamic configuration, service discovery, traffic mirroring, deployment strategies, health checks, rate limiting, and service mesh capabilities for advanced traffic management.
High Availability: Traefik Enterprise ensures continuous service availability with its fault-tolerant design. It runs natively in cluster mode, storing all cluster data, and works with AKS for seamless application scaling.
Scalability: Traefik Enterprise scales with your applications and supports various deployment methods for easy replication across different environments.
Monitoring and Troubleshooting: It offers a dashboard, proxy health checks, and support for tracing and real-time monitoring tools. Integrated with AKS, it helps understand traffic flow and service dependencies for efficient troubleshooting

Step-by-Step Instructions for Installing Traefik Enterprise via Azure Marketplace

To get started with the installation of Traefik Enterprise in AKS, make sure you first request a Traefik Enterprise trial.

1. Visit Microsoft Azure Portal and type Traefik Enterprise in the top search bar. Then, click on the correspondent link under the Marketplace section.

2. Click on the Basic tab, then choose your Subscription, Resource group, and Instance details.

3. In the Cluster details tab, choose your AKS Cluster name.

4. Next, in Cluster Extension details, choose your Cluster Extension Resource name, and the number of Proxy instances.

5. By clicking on the Review + Create tab, you will see the summary of your Traefik Enterprise installation on AKS. Next, click the Create button.

6. After completing the installation process, the following confirmation page will be displayed showing that the deployment is complete and one Traefik Enterprise instance is running in your cluster.

Now you can list Traefik Enterprise pods in traefikee namespace to verify that they are running, by executing the command:

kubectl get pods -n traefikee

Your output should list all running pods in traefikee namespace:

NAME                         	          READY   STATUS    RESTARTS    AGE
default-controller-0        	            1/1 	 Running            0	       10m
default-plugin-registry-0   	            1/1 	 Running   	   0      	       10m
default-proxy-d4c5549fc-7sl4f       1/1  	 Running   	   0      	       10m
default-proxy-d4c5549fc-ftgnj        1/1 	 Running   	   0      	       10m

Traefik’s services can be listed by executing the command:

kubectl get services -n traefikee

Your output should be very identical with the one below:

NAME                                         TYPE        CLUSTER-IP   EXTERNAL-IP  PORT(S) AGE
default-ctrl-svc                       ClusterIP        None             <none>     	<none>   20m
default-plugin-registry-svc   ClusterIP  	 10.0.52.229  <none>          443/TCP 20m
default-proxy-svc         	LoadBalancer 10.0.15.250  20.4.133.168 80:30876/TCP,443:31247/TCP   20m

The service default-proxy-svc was assigned an external IP by the provisioned Azure load balancer, and now can accept traffic in ports 80 and 443.

7. Traefik Enterprise is now running, and the next step is to configure it. The configuration can be done using teectl, a comprehensive command-line tool designed to interact with Traefik Enterprise. Follow the installation instructions in order to generate the controller configuration. This configuration is essential for Traefik Enterprise to handle incoming traffic, enable the Dashboard, and define certificate resolvers. The series of commands described below can be used to generate the necessary credentials to interact with your Kubernetes cluster using Traefik's teectl tool, and to configure teectl to use those credentials.
Continue by executing the following commands:

kubectl exec -n traefikee default-controller-0 -- /traefikee generate credentials --kubernetes.kubeconfig=/home/username/.kube/config --cluster=default > tee_config.yaml
teectl cluster import --file=tee_config.yaml
teectl cluster use --name=default

Remember to replace username with your actual user.
The output of the commands above will display your default pod that teectl interacts with, the cluster name, and the context namebe:

Defaulted container "default-controller" out of: default-controller, wait-dns (init)
Cluster "default" imported
Cluster context has been set to "default"

8. Now you must generate the static configuration. This is crucial because it sets up the foundational settings for your Traefik instance. The configuration will include the Providers, entryPoints, Dashboard enablement, and certificatesResolvers and is essential because it provides the foundational settings for your Traefik instance. The static configuration contains settings that define how Traefik will run and interact with your environment, and once set; these settings are not dynamically reloadable.

Create a file named static.yaml, and paste the following content:

providers:
  kubernetesCRD: {}

entryPoints:
  web:
    address: ":80"
  websecure: 
    address: ":443"
  internal:
    address: ":8888"

api:
  dashboard: true

certificatesResolvers:
  letsencrypt:
    acme:
      email: your_email_address
      tlsChallenge: {}

Replace your_email_address with your email address.

Now apply the static configuration by running:

teectl apply –file=./static.yaml

The command above does not produce an output, and at this step, you should have Traefik Enterprise correctly configured. The next step will involve creating the Ingress route for the Dashboard.
Create a file called dashboard-ingress.yaml, and the content:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefikee
spec:
  entryPoints:
	- websecure
  routes:
  - kind: Rule
	match: Host(`example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
	services:
	- name: api@internal
  	kind: TraefikService
  tls:
	certResolver: letsencrypt

Replace example.com with your domain name.
10. Create your routing rules by applying the file dashboard-ingress.yaml.

kubectl apply -f dashboard-ingress.yaml

The output below displays the created ingressroute resource:

ingressroute.traefik.containo.us/traefik-dashboard created

This means that your routing rules to the api@internal service are now applied.
Visit the URL https://example.com/dashboard/. You will be able to access the dashboard.

Your Traefik’s Enterprise dashboard is now fully functioning!

At this point, you can begin deploying your applications as you normally would. For instance, you can apply the following manifest that will create a namespace, service, deployment, and an IngressRoute, as seen below:

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: my-namespace
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: my-namespace
spec:
  selector:
    app: whoami
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: my-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        ports:
        - containerPort: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: my-namespace
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: whoami
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls:
    certResolver: letsencrypt

Replace example.com with the domain name of your choice. In the example code above, we used the image traefik/whoami, which is a tiny Go webserver that prints OS information and HTTP requests to output. So When you visit https://your-domain.com, you will see such information displayed on your screen.
This last step has our deployment guide completed.

Enhancing Your Setup for API Management on Top of Traefik Proxy or Traefik Enterprise

Once you have deployed Traefik Proxy or Traefik Enterprise on AKS, you might consider enhancing your setup with Traefik Hub. Traefik Hub is a Kubernetes-native API management platform that integrates seamlessly with Traefik Proxy and Traefik Enterprise, offering a streamlined solution for publishing, securing, and managing APIs.To get started with Traefik Hub, you can request a free trial here. You'll be able to deploy the Traefik Hub Agent in your Kubernetes cluster, following the guidelines provided in the official documentation. Traefik Hub offers a dashboard and onboarding steps making it easy to get started as demonstrated below.

Wrapping it Up

Traefik Proxy, Traefik Enterprise, and Traefik Hub provide enhanced connectivity and API management for AKS environments. Offering advanced security, traffic management, scalability, and monitoring, they help developers and enterprises streamline their application publishing and security. By following our installation guides and Traefik Hub introduction, you can easily tap into these additional features to optimize your AKS setup.

Interested in getting started with Traefik in AKS? Schedule a personalized demo of Traefik Enterprise with us. We'll guide you through its main features and demonstrate how it can fit into your specific AKS use case.

]]>

Harnessing the Power of Kubernetes-native API Management with Traefik Hub and Rancher Prime

Erwan Paccard — Fri, 16 Jun 2023 13:32:34 GMT

As digital transformation accelerates at an unprecedented pace, several design patterns and architectural choices have emerged to keep up with the rapid change. Microservices, multi-cloud deployments, Kubernetes, and APIs are not just buzzwords; they are critical components of the contemporary digital landscape, driving business agility, application agility, digitization, and B2B value creation.

The surge in digital services has led to a scenario where API requests comprise over 80% of all traffic types and continue to grow, with 90% of digital services being built using public & internal API-delivered services. APIs, therefore, form a critical component of digital transformation, and participating in the API economy has become a top priority for most organizations.

Amid this backdrop, today we’re going to explore the joint value proposition of Traefik Hub and Rancher Prime, two powerful tools that, when combined, can supercharge your digital transformation journey.

Embracing Kubernetes-native API Management

Traefik Hub is the industry’s first Kubernetes-native API management solution. It extends the Kubernetes capabilities with API Management through CRDs, seamlessly integrates with its API, enabling automatic discovery of services, and delivers Kubernetes native ways to publish and manage APIs. This provides a significant operational advantage over proprietary solutions that are not Kubernetes-native. Traefik Hub Kubernetes-native approach based solely on CRDs unlocks operational excellence with scalable and highly available deployments, a full GitOps compliance for powerful automation with tools like Helm, Terraform, and ArgoCD for repeatability and auditability. It also eliminates vendor lock-in by supporting any Kubernetes distribution and ingress controller and offers a lightweight, portable, and efficient solution with separate control and data planes.

Ensuring High Availability & Security

Coupling Traefik Hub with Rancher Prime, the leading Kubernetes management platform ensures high availability with built-in HA with replicas. This redundancy is a fundamental component of high availability, as it ensures that even if one node goes down, the service remains accessible. The Traefik Hub platform, as part of its core offering, allows for easy and automatic configuration of service replicas. Therefore, if one instance faces an issue, another can seamlessly take over, reducing downtime and maintaining service reliability.

Traefik Hub and Rancher Prime also help users extend security with seamless third-party integrations. Rancher Prime provides comprehensive security features including RBAC, SSO, AD/LDAP integrations, and secrets management. Combined with Traefik Hub’s built-in high availability and security features, you get a secure, robust, and reliable application deployment pipeline.

CI/CD Compliance

The combination of Traefik Hub and Rancher Prime provides a powerful toolset for those embracing the GitOps model and continuous integration/continuous deployment (CI/CD) methodologies. They offer a streamlined workflow that seamlessly integrates with your CI/CD pipeline, making it an ideal solution for DevOps teams. Rancher Prime provides built-in CI/CD pipelines that can be configured to trigger deployments based on changes to a Git repository. Rancher Prime’s multi-cluster management capabilities allow CI/CD pipelines to deploy applications across multiple Kubernetes clusters. This feature is particularly beneficial in a multi-cloud or hybrid-cloud environment. With Traefik Hub providing API management across these clusters, you get a unified, efficient, and scalable deployment process.

Custom Resource Definitions (CRDs) in Kubernetes provide a way to define custom resources with your own API types, extending the Kubernetes API. Traefik Hub, being a Kubernetes-native solution, treats CRDs as first-class citizens. This means that it provides native support for CRDs, allowing developers to create and manage custom resources easily.

Deploy Anywhere, with Any Ingress Controller

The Traefik Hub and Rancher Prime solution supports any deployment environment and any ingress controller. This flexibility is crucial for organizations looking to deploy APIs across diverse environments, including on-premises, hybrid cloud, multi-cloud, and edge deployments.

Modern, lightweight, and open architecture

Traefik Hub and Rancher Prime boast a modern, lightweight, and open architecture. They unlock and unleash the full potential of Kubernetes for API Management. They are lightweight, so they won’t weigh down your systems, but they are feature-packed to keep your operations running smoothly. Their open architecture speaks the language of Kubernetes, making integration a breeze. And because they are modern, they are constantly evolving to meet the ever-changing demands of the DevOps world.

Embracing Edge Computing

Edge computing, another significant trend, is fueling new edge applications and creating infinite possibilities. The broad spectrum of edge use cases can be divided into the Near Edge, Far Edge, and Tiny Edge (autonomous vehicles, industrial robotics, drones/UAV, smart grid), with each having specific requirements in terms of devices and proximity to data centers or users.

As applications move to the edge, so do APIs. Specialized Edge APIs are emerging with the same drivers around business value and time. Traefik Hub, Rancher Prime, RKE2, and K3s are well-positioned to cater to these needs due to their flexible deployment capabilities, GitOps compliance, and support for any ingress controller.

Architecture diagram Rancher Prime + Traefik Hub

In conclusion, Traefik Hub and Rancher Prime offer a compelling value proposition for organizations looking to streamline their API management and Kubernetes operations. By embracing these tools, you can not only keep pace with the accelerating digital transformation but also lead, compete, and thrive in the digital economy. The future of API management is Kubernetes-native, flexible, and secure, with full CI/CD compliance. Embrace this future with Traefik Hub and Rancher Prime, and unlock the full potential of your digital transformation journey.

Traefik Labs is excited to announce our sponsorship of SUSECON 2023. We invite everyone to join us at the conference, in person in Munich or online, from June 20th to June 22nd. From Traefik Labs, Sudeep Goswami, Chief Revenue Officer, and Maytham Alfouadi, Solutions Architect will be presenting "Kubernetes-native API management at the K3s edge: The GitOps way" on Thursday, June 22 at 9 am CEST.

Join us to learn about the latest developments in open source technology and API management strategies. We look forward to seeing you there!

]]>

Kubernetes CRDs in Traefik: Overcoming the Limitations of Kubernetes Standardized Objects with Custom Resource Definitions

Nikolas Sachos — Tue, 06 Jun 2023 14:48:46 GMT

While Kubernetes provides built-in API objects and resources, the Kubernetes API can be extended to any object you like using Custom Resource Definitions (CRDs). This allows you to capture the subtleties of your use case and tap into the flexibility Kubernetes offers.

As container orchestration continues to gain traction, Kubernetes has become a (if not the) standard for Developers and DevOps engineers: they’re now used to declaring the desired state of their system instead of manually deploying software all over the world.

But how does one declare this desired state? And what if Kubernetes doesn’t fully capture the subtleties of your use case and you need to be a bit more descriptive than what it is aware of?

The goal of this post is to answer this very question. In the context of Ingress Management (how your cluster accepts and reacts to incoming requests), we’ll walk you through the standardized objects, use them with Traefik, understand their limitations, and show you how to overcome them. (Spoiler, we’ll use Traefik CRDs.)

Understanding Kubernetes Objects

In a world where APIs are more and more prevalent, it’s no wonder that Kubernetes introduced a way to describe how your cluster must handle incoming requests: the Ingress object.

But if you look at the bigger picture, other objects are involved in the process, let’s see which ones:

Ingress: As we just said, Ingresses manage external access to services within a cluster. You’ll describe HTTP/HTTPS request patterns and which services are responsible for handling these.
Pods: The smallest Kubernetes unit, and most of the time a single container (could be more). You can see a Pod as a running instance (replica) of your application.
Services: Because Kubernetes decides where it deploys a Pod, and because it might kill and respawn a Pod based on the health, workload, and lifecycle of the whole system, each Pod has its own ephemeral ID. Services are a construct that provides a stable logical name for accessing the Pods.
Deployments: Deployments are where you tell Kubernetes what containers you want to run in your Pods, and how many replicas you want it to run. There’s of course more to say, but for the sake of our tour, this is enough.
ConfigMaps: ConfigMaps store non-sensitive, configuration-related data as key-value pairs. They allow you to decouple configuration data from container images, making managing and updating your applications easier.
Secrets: Similar to ConfigMaps, Secrets store sensitive data such as passwords, tokens, and keys. They help protect sensitive information by allowing access control and keeping it separate from your container images and application code.

Each object serves a specific purpose and contributes to the overall architecture, but if you have a good understanding of the above concepts, you can already take full advantage of Kubernetes’ powerful features and create robust and scalable applications.

Using Kubernetes Ingress Object with Traefik

So if the Kubernetes Ingress Object describes the external access to your services, what handles it? The answer is the Ingress Controller, and Traefik is one of them (and we like to believe one of the best).

When you run Traefik on your K8S cluster, it reads the Ingress objects to understand the routing you want to achieve, then routes accordingly. It’s as simple as that.

Basic Example: TLS and Two Paths

Basic Example: TLS and Two Paths

Let’s define two different paths (/foo, /bar), routing to two different services (foo-service, bar-service).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: IngressFooBar
  namespace: production
annotations:
   kubernetes.io/ingress.class: traefik
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls-secret
  rules:
    - host: example.com
      http:
        paths:
          - path: /foo
            pathType: Exact
            backend:
              service:
                name: foo-service
                port:
                  number: 80
          - path: /bar
            pathType: Exact
            backend:
              service:
                name: bar-service
                port:
                  number: 80

In the above example, you can see the Ingress resource with a specific host (example.com) and two paths. The first path, /foo, routes traffic to the backend service foo-service. The second path /bar routes traffic to the backend service bar-service.

Apart from the obvious, both services are listening on port 80. Additionally, the tls section specifies that TLS should be used for the host example.com and that the TLS certificate and private key are stored in a Secret named example-tls-secret.

To create this secret, you can run the command:

kubectl create secret tls example-tls-secret --cert=path/to/tls.cert --key=path/to/tls.key -n production

When you have simple needs, you need a simple solution, and we saw that Traefik is an excellent fit in that scenario. Traefik is fully compatible with the Ingress specification and will handle any request you can describe using that standard.

Limitations of Kubernetes Ingress

The premise of the Ingress object was to define a standard capable of expressing every routing need the user would face. Then, users would pick their Ingress Controller of choice. Unfortunately (or fortunately), there is more to routing than just paths, and chances are your ingress controller of choice can do (much) more than path-based routing, which is especially true if you’re used to full-featured solutions like Traefik.

These limitations include:

Limited Routing Rules: As we said, the Ingress object, while offering basic routing rules based on the request host and path, falls short in providing advanced routing capabilities such as header-based routing or traffic splitting.
HTTP Only: The primary function of the Ingress object is to route HTTP and HTTPS traffic. Support for other protocols, such as TCP and UDP, simply doesn’t exist.
Limited to routing: the Ingress object does not include features that play along well with your ingress controller, like tracing, logging, rate limiting, circuit breakers, or many other use cases.

Annotations to the rescue

To support these extra features, Ingress Controller vendors started to leverage the annotation system in Kubernetes, where one can add any text information and attach it to the Ingress object. Unfortunately, no standard has emerged for these annotations, and these annotations themselves are limited:

No validation: Because the annotations are “free text”, there is no safety net for users so that they can avoid typos or syntax errors while describing advanced features they want to attach.
No standard: Each vendor has their own set of features and options, defeating the purpose of having a standard in the first place.
Too broad: Because annotations are attached to the Ingress itself, it’s difficult to describe a feature on a finer-grained level (e.g., the path or the service).

Introducing Traefik Kubernetes CRD: A better alternative

So you chose Traefik for a reason, and that reason is you want to leverage every feature it has to offer.

Since annotations already force you to add specific configuration instructions that break the premise of compatibility with other ingress controllers and don’t overcome every limitation, you need a better alternative.

This is where CRDs (Custom Resource Definitions) come into play.

Not only Kubernetes has built-in objects, it also allows you to define your own, extending its description capabilities.

Traefik Kubernetes CRD (Custom Resource Definition) is a powerful tool that overcomes the Ingress specification limitation and allows for more options on top of providing a clear structure, for example:

IngressRoute: This is the extended equivalent of the Ingress object. It adds support for various options, such as load balancing algorithms.
Middleware: Middleware are a concept that you can mix and match for everything that goes beyond routing, whether it’s Access Control, header updates, circuit breakers, path manipulation, error control, redirection, and many others.
TraefikService: Provides an abstraction for HTTP load balancing and mirroring. It allows you to define how traffic should be distributed among multiple services in your cluster.
IngressRouteTCP & MiddlewareTCP: As their name implies, they allow you to define TCP routes and tweak the request in the process.
IngressRouteUDP: Once again, adding support for a different protocol, this time UDP.
TLSOptions: To fine-tune TLS connection parameters such as the minimum TLS version and the cipher suites that should be used.
ServersTransport: To tweak communication between Traefik and the services.

To read more details about these CRDs, visit Traefik CRDs documentation.

Example: Achieving Rate Limiting with Traefik CRDs

Now that we know what Traefik CRDs are, let’s leverage them to add rate limiting to our example, setting up a rate limit of 30 average requests per second with a burst of 50 for a nodejs service, accessible through the example.com host.

In the process, we’ll also ask Traefik to automatically handle certificate generation using Let’s Encrypt, that is is a free, automated, and open Certificate Authority (CA) that provides digital certificates required for HTTPS.

apiVersion: v1
kind: Namespace
metadata:
  name: nodejs
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: nodejs
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: nodejs
  namespace: nodejs
spec:
  selector:
    app: nodejs
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-deployment
  namespace: nodejs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodejs
  template:
    metadata:
      labels:
        app: nodejs
    spec:
      containers:
      - name: nodejs
        image: admintuts/expressjs-hello-world:latest
        ports:
        - containerPort: 3000
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: nodejs
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: nodejs
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls:
    certResolver: letsencrypt

To apply this configuration, save the above content into a YAML in a file, say nodejs-rate-limit.yaml, and apply it to your Kubernetes cluster with the following command:

kubectl apply -f nodejs-rate-limit.yaml

The command output will be:

middleware.traefik.containo.us/prod-rate-limit created
service/nodejs created
deployment.apps/nodejs-deployment created
ingressroute.traefik.containo.us/prod-route created

This will create the resources in your cluster, and a nodejs service will be subject to the rate limit rules defined by the prod-rate-limit middleware. Now you can test your setup, and you should observe that any requests beyond that limit are appropriately throttled.

Bonus: Testing Traefik Rate Limiting

To validate the rate limiting abilities of the configuration we applied above, we can use the hey tool, a popular HTTP benchmarking and load-testing utility. With hey, we can simulate multiple concurrent requests and observe how Traefik enforces the rate limits we have set. Assuming you have hey installed, let's run a test with the following command:

hey -n 100 -c 10 -q 10 http://example.com

Command output:

[...]
Status code distribution:
 [200]    75 responses
 [429]    25 responses

As you can see above, from our 100 requests made to the host, 25 requests were throttled, effectively proving that our rate limiting strategy is working as expected.

Let's break down the parameters used in the hey command:

-n 100: Specifies the total number of requests to be made, in this case, 100.
-c 10: Sets the number of concurrent connections, here we use 10 to simulate concurrent users.
-q 10: Defines the maximum number of requests to be made per second, limiting the request rate to 10 per second.
http://example.com: Specifies the URL endpoint we want to target with our requests.

By leveraging a Traefik Custom Resource Definition with the rateLimit configuration option and tools like hey, you can confidently design, test, and fine-tune your rate limiting strategy to achieve the desired level of control and protection for your applications.

Benefits of Using Traefik CRDs

Traefik CRDs deliver a host of benefits for your Ingress and API management. Here are a few reasons to incorporate them into your workflow:

Flexibility: With Traefik CRDs, you get unparalleled flexibility and fine-grained control over your Ingress traffic. Leverage advanced routing, traffic management, and security features tailored to your bespoke application requirements.
Compatibility: Traefik CRDs work seamlessly with Traefik’s ecosystem of tools and features, making them an ideal fit for your existing Traefik setup.
Advanced Use Cases: Traefik CRDs enable you to implement advanced use cases, such as Canary Deployments, A/B testing, and rate limiting that the standard Kubernetes Ingress object may not support. This allows you to implement complex traffic management strategies to ensure smooth application operation.
Ease of Use: It's easy to define, configure, and manage with standard Kubernetes manifest files, following the same declarative approach as other Kubernetes resources.

Wrapping It Up

In this article, we delved into Kubernetes CRDs in Traefik. We discussed Kubernetes Ingress limitations and how Traefik CRDs overcome these, providing a practical rate limiting example. We highlighted the benefits of Traefik CRDs, including their flexibility and compatibility. Whether you're a software developer or a DevOps engineer, we encourage you to explore the power of Traefik CRDs in your Kubernetes deployments for a more robust and scalable application infrastructure. Remember, the Traefik community is always here to support you.

In addition, CRDs are not only used in Traefik Proxy. Our recent product Traefik Hub, a Kubernetes-native API Management solution that simplifies API publishing, security, and management, also leverages native Kubernetes constructs, like CRDs, labels, and selectors. It is also fully GitOps compliant providing quick time to value and increased productivity for users. In an upcoming article, we will cover how to use CRDs in Traefik Hub to simplify API Management. In the meantime, you can start your Traefik Hub trial here.

]]>

Ushering in a New Generation of API Management With Traefik Hub

Erwan Paccard — Wed, 17 May 2023 15:39:24 GMT

Today we are pleased to announce the general availability of Traefik Hub API management, a modern Kubernetes native solution for publishing, securing, and managing APIs, with industry-first support for major third-party ingress controllers including NGINX.

The ability to design, develop, and launch APIs has become a strategic imperative for organizations focused on cloud native, containerization, and digital transformation initiatives. APIs are the connective tissue that powers nearly every digital product and service today, representing ~83% of total requests on the Internet, and are growing 2X faster than all other Web traffic. Yet, existing API management solutions are struggling to provide a cloud native experience and quick time to value, with their users facing steep learning curves, growing operational complexity, and a lack of deployment flexibility.

API management, the origin story

Before we dive into the details of the announcement, we wanted to share a bit of the back story on how Traefik Hub came to be. Traefik Proxy, our first project, was born in 2015 to tackle the growing complexity of adopting cloud native technologies. With a maniacal focus towards simplification and automation, the Traefik Proxy open source project has achieved over 3 billion downloads and 43k GitHub stars to date. From advanced home lab users to Fortune 100 enterprise architects, many have been using Traefik Proxy as their unified ingress and API gateway, helping it reach and maintain a leadership position in this category for several years in a row, based on rankings published on Ossinsight.

Given our access to such a large and globally distributed install base of Traefik users, we have routinely heard and observed first-hand the key challenges associated with API management.

Large monolithic systems: the API management ecosystem is rife with large and proprietary monolithic systems that imbue an all-or-nothing mindset. This is in direct conflict with modern cloud native principles that promote choice, modularity, and lightweight footprints.
Misalignment with DevOps operating model: existing API management solutions are designed around UI point-and-click workflows rather than Kubernetes-native scripting and automation, thereby causing friction and inefficiencies when addressing the needs of organizations adopting CI/CD and GitOps-centric methodologies.
Limited deployment flexibility: existing API management solutions are tightly coupled to a limited set of deployment environments, creating operational constraints for organizations adopting hybrid, multi-cloud and edge strategies.

Through numerous discussions and user surveys, we validated and concluded the dire need for a modern API management solution that caters to the growing Kubernetes and cloud native user community. Given our company’s track record and DNA in solving similar intractable problems, we felt compelled to rise to this new challenge and deliver a best-in-class Kubernetes-native API management solution.

Introducing Traefik Hub, the first Kubernetes-native API Management solution

Traefik Hub is the industry’s first Kubernetes-native API Management solution for publishing, securing, and managing APIs, with support for both Traefik and major third-party ingress controllers including NGINX, HAProxy, Ambassador, and many others.

Traefik Hub has been built from the ground up to deliver outcomes aligned with the following design principles and goals:

1. Freedom of Choice

Traefik Hub offers choice across multiple dimensions: users can choose any leading ingress controller, including Traefik Proxy or third-party solutions like NGINX. Users can also standardize their Traefik Hub installation across multiple deployment environments, including on-prem, managed Kubernetes in multi-cloud, and edge. Furthermore, the Traefik Hub architecture is both open and modular, providing seamless integration capabilities with third-party API design, testing, mocking, and security solutions, while leveraging industry best practices and standards such as OpenTelemetry and OpenAPI specification (OAS)

2. DevOps-first Mindset

Traefik Hub is a modern solution that embodies a DevOps-first mindset. Initial configurations and ongoing changes can be expressed as code and deployed through GitOps and CI/CD friendly workflows.

3. Security-first Mindset

Any discussion of a modern API management system would be incomplete without addressing the notion of APIs becoming a new attack vector and ways to mitigate them. Traefik Hub offers robust security features such as Role Based Access Control (RBAC), and support for modern industry-standard authentication and authorization mechanisms like OAuth 2.0 and JSON Web Tokens (JWT). Furthermore, Traefik Hub’s open and modular architecture offers integration capabilities with leading third-party API security solutions to provide additional layers of protection.

4. Kubernetes-native Approach

Traefik Hub provides a Kubernetes-native approach through deep integration with the Kubernetes API. Starting with the auto-discovery of existing microservices, Traefik Hub leverages familiar and powerful Kubernetes objects and primitives such as Custom Resource Definitions (CRDs), labels, and selectors to create, manage, and secure APIs at scale.

5. Quick Time to Value

Traefik Hub control plane provides a simple and intuitive point-and-click experience for users who are just getting started with publishing APIs in a single or multi-cluster Kubernetes environment, while also giving advanced Kubernetes users access to its full automation and GitOps capabilities. This dual operating model accelerates Kubernetes-native adoption and
maximizes business agility, giving organizations unprecedented speed and scale in launching new APIs and services.

Getting started with Traefik Hub

Traefik Hub addresses the challenges and limitations of traditional API management, by providing a true Kubernetes-native experience with full GitOps compliance for simplicity, automation, and security. With Traefik Hub, engineers can continue to use their preferred tech stack to build great APIs and applications. We believe Traefik Hub is the most open, modular, and intuitive platform for managing APIs across hybrid, multi-cloud, and edge environments.

If you are unsatisfied with your existing API management solutions, we would love to hear from you and encourage you to take Traefik Hub for a spin.

Sign up for a two-week trial here and experience a better way to manage your APIs.

]]>

ABAX Tracks Over 500,000 Assets with Entirely Cloud Native Strategy

Traefik Labs — Thu, 04 May 2023 23:18:53 GMT

At-a-Glance

Over the course of three years, ABAX entered a new industry and doubled the number of assets it tracks for customers. In that same period, it standardized its compute operations onto Kubernetes and successfully completed its migration to a fully cloud native environment.

Open source-based solutions, such as Rancher Prime, K3s, and Traefik Enterprise, played a central role in these achievements and continue to support the company’s next-generation operations, which run with less than five IT personnel at the helm.

Introducing ABAX

In 2007, ABAX opened its doors to deliver basic tracking solutions through high-quality GPS positioning. Since then, it has expanded services to help companies manage assets in new and better ways, leading it to become one of the largest telematics companies in Europe. Today, its sophisticated fleet and asset tracking solutions help clients reclaim stolen property, extend the lifecycles of expensive machinery, increase worker productivity, meet taxation requirements and even reduce carbon emissions.

In 2020, ABAX embarked on a journey to expand its offerings by developing mobility data solutions and entering new verticals, such as insurance and leasing. As a result, the company has become the preferred mobility data provider for the business-to-business industry on a broader scale.

ABAX's solutions empower customers to operate their businesses more intelligently and sustainably, making it simple to enhance efficiency, adhere to regulations and boost profitability. This approach is known as smart mobility.

The Journey to Cloud Native Operations

Combining telecommunications with information technology, ABAX is committed to using the latest technology advancements to send, receive, store and interpret data for its customers. In 2017, the company started using Kubernetes, Rancher Prime and public cloud providers to scale its operations for accelerated growth in the coming years.

Having proven to itself the viability of cloud-based container operations, it laid a plan in the Fall of 2020 to go fully cloud native by the end of 2022, but those plans went into hyperdrive mid-2021 when its on-premises data center provider announced closure by the end of the year. Thanks to the support of cloud native networking solution, Traefik Enterprise (an integrated API Gateway and ingress controller), ABAX seamlessly migrated its on-premises data center services to the cloud by directing traffic from one environment to the other. Originally planned as a multi-year migration project, ABAX was able to complete its migration in a quarter of the time and successfully went fully cloud native Dec. 30, 2021, one day before its on-premises data center pulled the plug.

Despite the accelerated timeline, SUSE solutions, Traefik Enterprise, and public cloud providers enabled ABAX to not only ensure continuity of services, but also reserve time for innovation. Remarkably, less than five people sit on the IT team that supports more than 70 developers and the environments of nearly 45,000 customers. For the team, maintaining these environments at scale, while having time to implement competitive infrastructure features, is vital for success.

“Providing software as a service for as much as we can is a requirement for us to be able to survive as an operations team due to resource availability,” says Thomas Ornell, principal system engineer (level IV) at ABAX.

A Modern, Open Source Infrastructure

ABAX processes a vast amount of data from over half a million devices every hour, making uninterrupted availability critical to the company's success. To ensure reliability, the company turned to Kubernetes as its primary compute platform, migrating numerous IT processes to the container orchestration system. ABAX runs most of its Kubernetes clusters on Google Public Cloud, with some operations on AWS and Microsoft Azure.

As a proponent of open source, ABAX relies on several open source solutions, including Rancher Prime, K3s and Traefik Enterprise, to optimize its infrastructure operations and deliver services to clients. These tools allow ABAX to manage and deploy its Kubernetes infrastructure efficiently and securely, maintain granular control over user access, and provide next-generation customer services.

“With Traefik Enterprise, everything is automated and easy to use. We maintain functionalities that our developers rarely see because they just work.”

Thomas Ornelll, Principal System Engineer (Level IV), ABAX

“Rancher Prime and Traefik Enterprise have become an integral part of the ABAX cloud native infrastructure,” says Ornell. “With Traefik Enterprise, everything is automated and easy to use. We maintain functionalities that our developers rarely see because they just work.”

Regardless of its dedication to open source, ABAX understands the importance of risk mitigation and utilizes enterprise support to ensure its systems remain stable and secure. By partnering with SUSE and Traefik Labs, ABAX can focus on building its future and delivering exceptional services to its clients with peace of mind.

The Impact

Maximizing Productivity with a Consistent Experience

ABAX, with a lean IT team handling various platform stacks and tools, aims to provide a consistent experience, performance and troubleshooting to its application developers. To achieve this, the IT team needs complete control over the platform stack. Rancher Prime and Traefik Enterprise make ABAX’s vision of a cloud native, infrastructure-agnostic platform a reality, providing developers with a single, consistent experience, regardless of the underlying cloud infrastructure.

The Rancher Prime interface allows developers and operations to work together in production with ease. The experience is the same, no matter which cloud a cluster is hosted on. “We don’t have to train our developers how to use the different user interfaces between the different cloud providers,” says Ornell. “Rancher Prime’s unified control system is key to maximizing productivity.”

Saving Time with Automated RBAC

Rancher Prime’s automated role-based access controls (RBAC) have also made a significant impact in enhancing the IT team’s productivity. Automated RBACs enable the team to grant and limit access to clusters and defined projects within Rancher Prime easily. If a developer works on a particular business domain, they can be granted access to only that business domain inside the production cluster, which is particularly important in highly regulated customer industries like insurance.

“What we're doing is giving developers direct access so they can manage their own workloads to a certain extent, saving us valuable time,” says Ornell. “This is key for us to be able to survive as such a small team managing so many different things.”

Simplifying Kubernetes Deployments with Fleet

Fleet, a feature of Rancher Prime, simplifies managing, deploying and scaling containerized applications across multiple clusters. It simplifies the process of managing distributed application deployments, allowing users to easily start, scale and keep track of the containers that make up their applications. It also saves time and resources by automating how users set up applications in different environments.

Fleet currently runs 51 infrastructure deployments and maintains more than 70 Kubernetes clusters for developers, which would be impossible for the IT team to manually manage otherwise. “We label our clusters with the features we want them to have,” says Ornell. “For example, we’ll label a cluster with Traefik Enterprise. Fleet then automatically deploys Traefik Enterprise to that cluster. Same with Datadog or any other infrastructure piece we might need in a cluster. This is key to being able to do a disaster recovery swap quickly because we wouldn’t have time to manually reconfigure everything when in a disaster scenario.”

Faster Disaster Recovery with a Powerful Trifecta

Speaking of disaster scenarios, ABAX was able to achieve an impressive disaster recovery time for its entire production compute environment (excluding the database environment) in just one hour 15 minutes, thanks to three features of Rancher Prime.

Rancher Prime's multi-cluster management enabled the team to efficiently monitor and control resources across multiple clusters through a single pane of glass, while RBAC and Fleet made deploying fine-grained control over user access to resources across multiple clusters significantly easier. With these three components, the team was able to relaunch its operations efficiently and securely at scale, saving precious time.

According to Ornell, “Getting the environment itself back online wouldn’t take that much longer without Rancher Prime. The real difference lies in the post-recovery work, such as providing proper access for developers, ensuring all infrastructure components are back in the cluster and so on. Rancher’s RBAC and Fleet make these steps significantly faster. For comparison, doing this manually would take days.”

Standardizing Cluster Environments with K3s and Traefik

ABAX uses K3s to standardize cluster environments between development and production. K3s is a lightweight Kubernetes distribution originally developed by Rancher Labs and is now a CNCF project. K3s uses Traefik as the default ingress controller to manage incoming traffic. “The tiny differences between developer machines and production environments leave room for error, especially when trying to figure out why a service is working on a developer machine but not on a production environment,” says Ornell.

“K3s, deployed through k3d, and Traefik enable us to deploy everything in a developer environment as if it were a production environment, making it easier to identify issues when they arise.”

Thomas Ornelll, Principal System Engineer (Level IV), ABAX

The team implemented k3d, a lightweight wrapper used for running K3s in Docker, to run small Kubernetes clusters on every developer machine. “K3s, deployed through k3d, enables us to deploy everything in a developer environment as if it were a production environment, making it easier to identify issues when they arise,” says Ornell.

Consequently, nothing has to change in how the Kubernetes and networking layers operate when applications move from development to production, speeding time to launch.

Resolving Issues with SUSE and Traefik Labs’ Enterprise Support

SUSE’s commitment to open source ethos means that Rancher technology is the same whether it comes from the community or enterprise version. For resource-constrained teams, paid support can be invaluable.

In working with SUSE Support, Ornell has found the team resolves issues before other vendors, working on the same issue, deliver a second reply. “SUSE has the competency to debug Kubernetes comprehensively, so we can figure out what is actually going wrong,” says Ornell. “We have very limited resources available. If we can offload debugging and figuring out what’s wrong to a support team instead of us having to do it, that’s golden on our end”

Support goes to another level when support teams from SUSE and Traefik Labs collaborate to resolve issues. Reflecting on a particular instance, Ornell states, “We couldn’t pinpoint where the issue was and we ended up being on a support call with support representatives on both sides collaborating, which to me is invaluable.”

Accelerating Solutions Via a Partner Ecosystem

SUSE’s partner network also provides additional value for those working with the company. This is how ABAX, for instance, discovered Traefik Labs when it sought to employ a Kubernetes-native ingress controller to manage incoming data from various client machinery, assets and automobiles. After browsing SUSE’s partner application catalog for Rancher Prime, ABAX selected Traefik and immediately integrated it into its infrastructure. Now, Traefik Enterprise handles every incoming request. Traefik Labs also played an essential role in ABAX’s cloud migration acceleration. Moreover, all customer-facing portals run on Traefik via a GKE cluster in Google Cloud, provisioned by Rancher Prime.

What’s Next?

Looking ahead, ABAX, now a major player in the European telematics market, plans to leverage the data it already processes to provide additional customer value. By embracing data sciences and artificial intelligence, ABAX customers will soon gain new insights that will inevitably lead to greater innovations. Backed by open source solutions and expertise, ABAX has a bright future ahead.

Originally posted on suse.com

]]>

Announcing Traefik Proxy 2.10

Nicolas Mengin — Wed, 26 Apr 2023 13:47:21 GMT

With improved native Kubernetes Service load balancing, new Prometheus metrics, new API group, and more.

There is always a lot of excitement around a new major version, but making the best possible Traefik Proxy 3.0 takes time and people. This past year, as we geared up to decide what Traefik proxy 3.0 should look like, we have also been working towards more open governance.

Our goal is to ensure the community can drive the direction of Traefik Proxy by influencing our roadmap and having significantly more input into feature design, implementation, and direct access via activities traditionally done by official maintainers, such as reviews. Check out these requests for design and input, as well as this hot topic on handling the migration to the new Kubernetes CRD definitions.

As part of our efforts to be more inclusive to our community, we have collected a series of small but mighty improvements that bring no breaking changes but enhance the user experience for many of our community members. Rather than holding these features until 3.0 is ready for release, we decided to ship these improvements early so that everyone can benefit from them right away.

Traefik Proxy 2.10 improves your ability to use service mesh with Traefik Proxy, enhances Prometheus metrics, and simplifies your Nomad configuration.

Let's jump into the new release and explore the nitty-gritty of your new abilities and how to use them.

Improved native Kubernetes Service load balancing

Up until now, Traefik Proxy only forwarded incoming traffic to pods. This made it difficult to address specific use cases that require the native Kubernetes load balancing with Traefik because it required the use of workarounds, for example, the creation of external services. These workarounds were unsatisfactory, and this missing feature was a blocker, especially for adopting Traefik for service mesh users.

Now, users have a new option for the providers Kubernetes Ingress and Kubernetes IngressRoute to decide whether the children of any given load balancer are directly in Pod IPs or if a Kubernetes Service is designated as the single child. In this case, the Kubernetes Service itself balances the load to the pods through a list of all endpoints used by the ingress controller in the upstream configuration.

This is especially important to users who employ a third-party service mesh, such as Cilium. Another benefit is that this configuration reinforces Traefik Proxy’s native support for zero-downtime deployments by removing any chance that traffic will be redirected to a non-existent client or pod (should a pod go down).

How it works

Enabling this feature is quite simple; all you need to do is add the `nativeLB` option to the service.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami
namespace: traefik
spec:
entryPoints:
- web
routes:
- kind: Rule
match: PathPrefix(`/who`)
services:
- kind: Service
name: whoami
namespace: app
port: 80
nativeLB: true # Enable the option

Note: In the specific case of a deleted pod, when a request comes in before Traefik has updated its routing configuration, you might generate a 502 bad gateway response because the Traefik configuration does not reflect the actual infrastructure. Using the `maxIdleConnectionPerHost` option mitigates the 502 error by creating a new connection with the backend service (pod), avoiding connection reuse to a pod that has suddenly gone down.

New Prometheus metrics

Prometheus is by far the most used third-party tool for metrics. To enhance the user experience when using Prometheus with Traefik Proxy, you can now split observations of the total request metric based on the value of one or multiple header values. This option allows you to gather more detailed information about your clients based on header information.

Headers are inherently flexible, so we can think of many ways you would use this feature, including creating a custom header to disclose app versions.

How it works

Traefik Proxy will now allow you to define extra labels for the `requests_total` metrics and the request header containing the value assigned to each label.

metrics:
prometheus:
buckets:
- 0.1
- 0.3
- 1.2
- 5.0
headerlabels:
useragent: "User-Agent"

Note: This feature is not enabled by default, therefore, there is no impact on performance by default. During testing, we did not observe any relevant performance hit. However, this does add an additional operation in the critical path of the request’s handling. When you enable the feature, if the header is not present in the request, it will be added automatically with an empty value. The label needs to be a valid label name for Prometheus.

Multiple namespaces in Nomad

Nomad allows you to use multiple namespaces within any given cluster. However, our original integration allowed you to use only a single namespace and required you to spin up an instance of Traefik Proxy per namespace within your cluster. We deprecated the singular namespace in the first Traefik Proxy 3.0 Beta, and this release brings the ability to use a single instance of Traefik to cover all namespaces within a given cluster.

providers:
nomad:
namespaces:
- "ns1"
- "ns2"

Introducing traefik.io API group

We are introducing our new CRD API group, `traefik.io`, with an eye to deprecating the `traefik.containo.us` API group in Traefik Proxy 3.0. Traefik Proxy v2.10 will bring no additional changes to the CRD scheme other than updating the API group. It will also include a warning log about the future depreciation of `traefik.containo.us` resources which will be removed in v3.0. You can read more about the community conversation in pull request 9600 and the Kubernetes CRD improvements in issue 9125. We would love to hear your feedback!

To use the new API group, install/update the CRDs and RBAC in your cluster, and you'll be able to define middlewares, IngressRoutes, etc., as either `traefik.containo.us` or `traefik.io`.

Wrapping up

We love that these features add flexibility to Traefik Proxy and hope you find them useful as well. We want to thank all the community members who have contributed to 2.10 through conversations on GitHub in issues and pull requests, donating code, testing, and sharing feedback on the 2.10 release candidate. This time, the special shout-out goes to Philipp and Sebastien, the top contributors of v2.10.

If you are interested in contributing to Traefik Proxy, join us on GitHub, where you can request features/enhancements, help us design the next version, or even get your hands dirty by creating or reviewing pull requests. For more information, check out the Release Notes and updated Docs. If you have questions, pop over to our Community Forum, where you can work with other community members to find your answers. And as always, we look forward to your feedback.

]]>

Announcing Traefik Enterprise 2.10

Nicolas Mengin — Thu, 20 Apr 2023 10:47:09 GMT

With support for claim data in OIDC access tokens, headers for rate-limited responses, and Kubernetes secrets for storing Authentication Sources credentials.

Delivering on our promise to relieve organizations from connectivity chaos, today, we are announcing the release of the latest Traefik Enterprise version.

The 2.10 release is short and sweet but has three new features to help current and future customers better secure and manage their APIs and services.

Let’s take a closer look.

OIDC middleware now inspects claims in access tokens

OIDC is being widely used for authentication and authorization purposes. To use OIDC as an authorization method, users need to implement claims. Claims contain user information and meta information on the OIDC service in the form of a name/value pair.

Until now, the OIDC middleware inspected claims in ID tokens, validated if the claim group, grp, has the value admin, and restricted the workload, allowing access only to the ID token that contains the claim grp and value admin.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-oidcAuth
spec:
  plugin:
    oidcAuth:
      source: oidcSource
      redirectUrl: example.com/callback
      session:
        secret: mysupersecret123
      claims: Equals(`grp`, `admin`) # Check the claim in the ID Token

While ID tokens are obtained only with the explicit consent of a human, access tokens are obtained through an automated process between systems and are considered a more secure method of adding an authorization layer to OIDC.

With the 2.10 release, we allow Traefik Enterprise users to add claims in Access tokens as well, enabling them to secure their APIs and services better.

Here is an example of how to use claims in Access tokens with the OIDC middleware.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-oidcAuth
spec:
  plugin:
    oidcAuth:
      source: oidcSource
      redirectUrl: example.com/callback
      session:
        secret: mysupersecret123
      claims: Equals(`access_token.grp`, `admin`) # Check the claim in the Access Token

Note: This feature works with JWT access tokens only.

Load sensitive data from Kubernetes Secrets

With this release, we are further improving how customers secure their APIs in Traefik Enterprise, by allowing for sensitive options values to be referenced in Traefik Enterprise using the Secrets object in Kubernetes.

With this improvement, credentials and other sensitive information are hidden in the static configuration.

To reference a Kubernetes Secret in Traefik Enterprise, use the form of a URN:

urn:k8s:secret:[namespace]:[name]:[valueKey]

Here’s an example of this configuration with JWT:

#...

authSources:
  jwtSource:
    jwt:
      signingSecret: urn:k8s:secret:traefikee:jwt-secret:secretValue

Note: The Traefik Enterprise Controllers need to be running within a Kubernetes cluster and users need to load the Kubernetes Secrets within the Traefik Enterprise deployment namespace.

Headers for rate-limited responses

Rate limiting is essential to API security. Limiting how often APIs can be called and throttling connections helps protect against traffic spikes and DDoS attacks.

This feature is an excellent addition for users who want to limit API consumers to a predefined number of requests per minute and inform consumers on the fly about their remaining amount of requests.

Using the ResponseHeaders option, users can configure Traefik Enterprise to inject the X-Rate-Limit-Remaining header in the response. This indicates how many tokens an API consumer has left in the token bucket before they get a 429 response.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-ratelimit
spec:
  plugin:
    rateLimit:
      responseHeaders: true

Summing up

Traefik Enterprise 2.10 is by no means a major release, but we are confident that it offers substantial benefits for customers looking to streamline their API security. If you are currently looking to level up your API game and solve the challenges that come with APIs, join our upcoming webinar and learn:

How an API gateway unlocks APIs in a cloud native world
How API gateways deliver great time-to-value and reduce infrastructure complexity

And in case you haven’t heard, we are at KubeCon Europe 2023! Feel free to drop by Booth S24 for a chat and a live demo of our new industry’s-first Kubernetes-native API management solution, Traefik Hub.

]]>

Production-Ready Kubernetes Deployments with the Traefik API Gateway and Supabase

Colin Wilson — Wed, 05 Apr 2023 16:11:40 GMT

API gateways have rapidly evolved to become a crucial component of most modern cloud infrastructure — particularly within microservices architectures. More so than other cloud components, API gateways need to be secure, robust, resilient, and easy to manage, and the Traefik API gateway is definitely up to this challenge.

In this guide, I want to give you a tour of the Traefik API gateway and explore how to configure it to enable a production-ready, self-hosted deployment of the Supabase Platform on Kubernetes.

Supabase describes itself as “an open source Firebase alternative”. The platform provides a collection of application backend features such as Authentication, Database (Postgres) and Storage (S3) and is quickly growing in popularity amongst many application development communities.

Why choose Traefik Proxy for this guide? Well, because the current Supabase Helm Chart utilizes a rather bare-bones deployment of Kong and Nginx. And although functional, both these configurations fall short of a production-ready environment, namely, a lack of authentication for the GUI and rate limiting of the API.

Substituting Kong and Nginx for Traefik Proxy makes up for the above shortcomings and provides a host of additional benefits:

Built-in Let's Encrypt integration
Simpler to configure
Several Middlewares available
Management of a single API Gateway instead of two
An upgrade path to Traefik Enterprise

Prerequisites

A Kubernetes Cluster (with cert-manager installed)
Knowledge of Kubernetes and Helm

Note: Many of the Traefik configurations and concepts used here also apply to other application platforms with similar API gateway requirements.

Architecture

The above architecture represents a high-level overview of the end goal. A production-ready deployment of Supabase on Kubernetes with Traefik API gateway securing the Supabase API and GUI (Studio).

Deploy Supabase on Kubernetes

Note: If you already have a Supabase deployment on Kubernetes, you can skip to the next section to install and configure Traefik Proxy. Since Traefik will be configured to expose the Supabase endpoints it’s important to disable any related Supabase endpoint ingress configurations in your existing deployment.

Use the Supabase Helm chart provided by the supabase-community repo to deploy Supabase on a Kubernetes cluster. The chart’s values.yaml options should be configured to meet the requirements of a production environment (and with Traefik Proxy in mind). So, configure:
- An externally managed or cluster replicated PostgreSQL database
- An external transactional mail service, e.g. SendGrid, Mailgun or Sparkpost
- An external (or internal HA) S3 compatible storage provider, e.g. AWS or Cloudflare
The chart’s default values.yaml enables an nginx ingress for both the Supabase Studio and API services (studio.ingress.enabled and kong.ingress.enabled). Since we’ll be configuring Traefik ingress routes for these services later on, both should be set to false.
Generate the JWT keys and secret values required to create the JWT secret inside the cluster.
Create a database secret containing the username and password of your PostgreSQL database and another secret with your SMTP provider credentials.
Deploy the Supabase helm chart inside your cluster.
helm -n default install supabase -f my-custom-values.yaml.

Use the kubectl port-forward command to check the Supabase API endpoints and GUI have successfully deployed in your cluster.

To check Supabase Studio, run the following command:

> kubectl port-forward service/supabase-supabase-studio -n default 3000:3000

Navigate to http://localhost:3000/projects and you should be presented with the Supabase Studio frontend:

Once you’ve confirmed Supabase Studio is working, go ahead and test the API endpoint next.

Navigate to http://localhost:3000/project/default/api?page=auth. This API Docs page features API commands with your previously configured access credentials substituted in. Click to copy the generated curl command used to authenticate with the /rest/v1/ endpoint.

Use the kubectl port-forward command once again, this time to proxy the Supabase API:

> kubectl port-forward service/supabase-supabase-kong -n default 8000:8000

Next, execute the curl command you copied from the API Docs in Studio:

curl 'https://api.supabase.local/rest/v1/' \
-H "apikey: SUPABASE_SERVICE_KEY" \
-H "Authorization: Bearer SUPABASE_SERVICE_KEY" | jq

The result should start similar to the partial snippet below:

  {
  "swagger": "2.0",
  "info": {
    "description": "",
    "title": "standard public schema",
    "version": "10.2.0.20230209 (pre-release)"
  },
……

Supabase is now successfully deployed and primed to be made production-ready with the help of the Traefik API gateway 🎉.

Deploy and configure the Traefik API gateway

As mentioned earlier, the default deployment of the supabase-kubernetes helm chart requires some important additional steps before it’s production-ready.

Install Traefik (Helm)

Before installing Traefik, make a minor addition to the values.yaml file so Traefik redirects all HTTP requests to HTTPS:

ports:
  web:
    redirectTo: websecure

Next, install Traefik via the traefik-helm-chart:

helm install -f myvalues.yaml traefik traefik/traefik

Configure the Traefik API gateway for Supabase’s APIs & GUI

Now it’s time to create the necessary Traefik configurations.

If you followed the Supabase chart deployment instructions above, you’ll recall the ingress configurations for both the Studio and API services were disabled (set to false).

Both ingress routes require a valid SSL certificate for their respective domains. e.g.

api.supabase.example.com for the Supabase API ingress route
studio.example.com for the Supabase Studio ingress route

Ensure these certificate secrets exist before creating the ingress routes that reference them.

Now, create the ingress routes that will publicly expose both the API endpoints and Studio services, this will also enable SSL/TLS termination for both routes:

api.supabase.example.com.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: api.supabase.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`api.supabase.example.com`)
      services:        
        - name: supabase-supabase-kong
          port: 8000
  tls:
    secretName: api.supabase.example.com

studio.example.com.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: studio.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`studio.example.com`)
      services:        
        - name: supabase-supabase-studio
          port: 3000
  tls:
    secretName: studio.example.com

Create two corresponding DNS records for your domain with your DNS provider, api.supabase.example.com and studio.example.com, both pointing to the IP address of the load balancer associated with your Traefik instance.

Protect the Supabase Studio dashboard with Traefik middleware

Navigate to https://studio.example.com, and you should (again) be presented with the Supabase Studio web interface. You’ll notice the Studio dashboard is currently unprotected — i.e., there’s no authentication step before access.

The Supabase dashboard can be secured using a Traefik *Auth middleware such as BasicAuth. Traefik's ForwardAuth middleware with an external authentication server like Authelia is another option.

However, I highly recommend taking access control to another level with Traefik Enterprise. Its built-in middleware OpenID Connect Authentication integrates with existing authentication deployments. See 'Going further with Traefik Enterprise' below for more details.

Rate limiting the Supabase API

At this point, Traefik is configured so both the Supabase Studio and API endpoints are secured against unauthorized or unauthenticated access.

So, can this deployment be considered production-ready? Not quite yet. The Supabase API has no protection against bursts of incoming traffic, be it from unexpected high traffic or malicious abuse, such as Denial-of-Service (DoS) attacks, it's a concern that cannot be left unaddressed.

Traefik has this covered with yet again another middleware option, the RateLimit middleware.

Define a rate limit to suit your needs using a middleware object:

# Here, an average of 200 requests per second is allowed.
# In addition, a burst of 100 requests is allowed.
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: ratelimit-supabase-api
  namespace: default
spec:
  rateLimit:
    average: 200
    burst: 100

This will set an overall rate limit across all Supabase API endpoints. This is a good starting point, plus the RateLimit middleware is capable of more granular rate limiting via options such as sourceCriterion.ipStrategy and sourceCriterion.requestHeaderName which sets rate limits based on incoming IP and request header groups respectively.

Apply the new middleware to the api.supabase.example.com ingress route:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: api.supabase.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`api.supabase.example.com`)
      services:        
        - name: supabase-supabase-kong
          port: 8000
      middlewares:
            - name: ratelimit-supabase-api
  tls:
    secretName: api.supabase.example.com

Once applied, your Supabase API route now has a safeguard against high volumes of traffic. You can also set a rate limit on the Studio service if required.

Production ready?

The Traefik API gateway is now handling:

TLS/Termination for the Supabase Studio and API routes
Secure authentication and authorization to Supabase Studio
Rate limiting of the Supabase API route

Your Supabase deployment is now production-ready!

Going further with Traefik Enterprise

Although this guide demonstrates a production-ready deployment of Supabase using the open source version of Traefik, it’s not without its limitations. Upgrading to Traefik Enterprise offers a path to overcoming these limitations. Let’s take a look at some ways in which Traefik Enterprise achieves this.

Authentication and authorization

As mentioned, Traefik Enterprise offers much more robust authentication solutions over middleware like BasicAuth and ForwardAuth. For example, the built-in OpenID Connect Authentication middleware means Traefik Enterprise can easily accommodate your existing authentication and authorization infrastructure to provide access control to Supabase.

The above is a simplified illustration of how the OIDC Authentication middleware functions. For a more detailed look at how to configure the OIDC middleware, check out Matt Elgin's article, 3 OIDC Configurations with Traefik Enterprise, from Basic to Advanced.

High availability

The very start of this guide mentions the importance of API gateways in modern microservices architectures. In contrast to Traefik’s open source edition, Traefik Enterprise is deployed by default as a cluster of proxy nodes. Substituting the default Supabase API gateway (a potential single point of failure) for Traefik Enterprise enables the Supabase endpoints to be served in a highly available and scalable fashion by a cluster of proxies. Not to mention the reduced complexity from now only managing a single API gateway installation, Traefik Enterprise.

Migrating to Traefik Enterprise also introduces a wealth of authentication protocols (beyond JWT) for authenticating with the Supabase API (including LDAP, OAuth2, OpenID, and HMAC). This opens the door for many other applications to interact with your API.

Distributed rate limiting

The rate limit middleware configurations applied to Supabase endpoints in this guide limit requests per an individual Traefik proxy. Traefik Enterprise’s distributed rate limiting ensures that requests are limited over time throughout your cluster and not just on an individual proxy.

Configuring this in Traefik Enterprise is as simple as adding a single line to your existing open source RateLimit middleware config:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: ratelimit-supabase-api
  namespace: default
spec:
  plugin: # <- just add this line to enable distributed rate limiting
	  rateLimit:
		average: 200
		burst: 100

Support

Last but not least — support. Traefik Enterprise comes with built-in support. A team of engineers is at your disposal should you encounter any issues or need any assistance with the installation, setup, or management of Traefik Enterprise. Quality support (in my opinion) is an important part of what makes a deployment “production-ready.”

Conclusion

This Supabase deployment has demonstrated how the Traefik API gateway can help build a production-ready application through the implementation of some of its key features.

Bear in mind authentication, rate limiting, TLS termination, and high availability are just the tip of the iceberg when it comes to the capabilities of the Traefik API gateway. If your application architecture requires a more complicated feature set, such as service mesh, security compliance, or disaster recovery, Traefik Enterprise can accommodate these needs.

To learn more about how the Traefik and Traefik Enterprise API gateways can enable production-ready applications and platforms, check out Traefik’s excellent glossary and the documentation for both Traefik Proxy and Traefik Enterprise. If you’d like to try out Traefik Enterprise for yourself, you can sign up for a 30-day free trial.

]]>

Axione Selects Traefik’s API Gateway to Simplify and Secure Deployments of Critical Applications

Kate Mikula — Wed, 29 Mar 2023 11:13:28 GMT

About Axione

Axione is a French telecommunications company that builds and operates network infrastructures across different cities and regions locally and abroad. It helps local authorities in their Public Initiative Network projects with a range of infrastructures and services to support the development and adoption of digital technology. It operates over 6 million Fiber to the Home (FITH) in 6,500 cities and has a workforce of 2,900 employees in 51 agencies.

Neil Orley, Head of SRE, and Alexandre Etmezguine, DevOps engineer, are part of the DevOps services team in the IT department. They are in charge of providing all IT services required to build network infrastructure, following the deployment of each project, and also equipping the sales department with the right tools to market their services.

Overview

Axione operates an infrastructure that consists of three large independent clusters, each of which is in a dedicated data center. The DevOps team manages applications and services within each cluster, handling everything from replacing the disks, and applying network upgrades to upgrading virtual machines. Around 90% of the infrastructure is legacy, consisting of VMs running monolithic, Java-based applications inside Docker containers. The team is currently using HashiCorp Consul to maintain a service registry for configuring VMs.

For the DevOps team, maintaining the security of infrastructure and applications and deploying the right tools to facilitate their management are paramount to maintaining smooth operations for the business. Recently Axione also started modernizing its architecture with new microservices projects deployed in the public cloud and are experiencing new challenges in managing a hybrid architecture between clusters running on-premises and in the cloud.

Choosing Traefik as an integrated API gateway and reverse proxy

Prior to migrating to Traefik, Axione relied on Nginx as a reverse proxy. The team managed Nginx manually, requiring them to go to each server individually to test and deploy configurations. The team was spending countless hours configuring Nginx and started facing an exponential number of issues when different people needed to edit multiple configurations, directly impacting their production environments.

The Axione team could not deploy new configurations without first waiting to finish editing and updating the existing ones. They first tried to solve this issue by creating a DevOps pipeline to automate the deployment of any configuration changes. But later, when requirements to add SSO logins emerged, they faced additional difficulties with the integration and maintenance of their authentication solutions. As the team started struggling more and more, they began looking for alternative solutions to Nginx.

“We were looking for a solution that could automate the configuration of service deployments to simplify and reduce maintenance costs.”

Alexandre Etmezguine, SRE DevOps, Axione

The team at Axione needed an API gateway that could be extended with reverse proxy functionality. They benchmarked numerous solutions and selected Traefik as the best and most flexible solution to answer their needs.

“When we benchmarked solutions, we found that Traefik's API gateway was the most adapted to our needs because of its flexibility, capabilities, and competitive price.”

Alexandre Etmezguine, SRE DevOps, Axione

Traefik simplifies routing, configuration, service deployment, and maintenance

The first requirement of the DevOps team was to find a solution that could easily automate the configuration and deployment of new services. Traefik's API gateway offers automated service discovery and routing, and also fully integrates with Axione’s CI/CD Git pipeline. Any changes are automatically and consistently pushed to all 3 clusters without any manual intervention. Today, it takes less than fifteen minutes to upgrade all their clusters with zero downtime, saving the team countless hours every month.

For the deployment of new services, the team uses Git’s pipeline capabilities as well as Traefik’s integration with HashiCorp Consul for service discovery. The service discovery integration with Traefik automatically generates a new configuration, making any deployment fully automated. As a result, the team now creates and populates environments for developers very quickly and is able to maintain DevOps practices across the development team.

“With Traefik's API gateway, we update our 3 clusters in less than 15 minutes with no downtime. We are saving so much time.”

Alexandre Etmezguine, SRE DevOps, Axione

Traefik strengthens application security

Given the growing array of cyber attacks in today’s world, security is a top priority for Axione. The DevOps team was looking for an integrated solution that could provide both reverse proxy and authentication capabilities. With Traefik, the team strengthened the security posture of their APIs and applications by using the OIDC middleware to integrate SSO logins. The team also integrates security headers to applications to forbid certain paths and routes to protect dangerous paths.

And thanks to Traefik’s library of plugins that include custom features, the team also integrates security features to rewrite header responses and add web application firewall capabilities. Traefik’s catalog of plugins consists of more than 100 plugins to this day.

“We are leveraging Traefik’s middleware and plugins. It is so easy to use. With only 2 lines of code, we have a plugin working”

Alexandre Etmezguine, SRE DevOps, Axione

Traefik is a highly available and performant solution

Traefik's API gateway equips Axione with a highly available platform to ensure they never lose any application requests. In the event the control plane is inaccessible, the data plane continues to work, serving application requests uninterrupted. Because of this high degree of availability, Axione today uses Traefik to route 100% of their traffic.

Traefik’s architecture has also increased the performance of Axione’s applications. Immediately after deploying some applications behind Traefik, Axione saw significant performance and latency improvements (about 20%) compared to running the same applications behind Nginx (visualized below).

“Traefik's API gateway instantly improved our application performance, even for our legacy applications, and it is quite impressive.”

“Traefik is highly distributed and available, which is important for us because we can update our clusters without any downtime.”

Alexandre Etmezguine, SRE DevOps, Axione

Traefik's API Gateway is infrastructure-agnostic.

The team at Axione is beginning to deploy services in the cloud to see how they can manage hybrid architectures that will stretch their clusters across on-prem and cloud environments. Because Traefik is infrastructure-agnostic and works in both legacy environments and cloud native environments, it can be used seamlessly both on-premises and in the cloud, with or without an orchestrator.

The team began exploring different orchestrators to use in the future. They are planning on switching to Kubernetes but are also looking at alternatives like HashiCorp Nomad. Traefik's API gateway is also orchestrator-agnostic and will remain their solution of choice regardless of the container orchestrator they choose.

“We secure applications and infrastructure while deploying tools that improve project management with the Agile methodology. We have begun to modernize our architecture with a microservice strategy and have started deploying new services in the cloud. The nice thing about Traefik is that it works with both legacy and cloud environments so we don’t have to learn and adopt a new tool as we adopt new cloud native architectures.”

Alexandre Etmezguine, SRE DevOps, Axione

Traefik Labs offers 24/7 enterprise support.

Given that uptime is a core requirement for Axione, the DevOps team required enterprise-grade support to ensure their system remains always operational. Traefik Labs offers unparalleled, 24x7 support to maintain the stability of the infrastructure. Axione has had a very positive experience with the support team.

“We have a terrific relationship with the support team at Traefik. David has helped us understand our architecture and what we wanted to achieve with Traefik's API gateway. We are very happy with the support, which is very efficient.”

Alexandre Etmezguine, SRE DevOps, Axione

Results

Axione today uses Traefik’s API gateway for both VMs and microservices. Axione has migrated all staging and production environments to Traefik and all public and private traffic of their applications enters through Traefik's API gateway. Axione currently has 250 applications and more than 1000 application URLs running behind Traefik.

Bottom line

Axione is in the process of modernizing its infrastructure, deploying new applications in microservices in the cloud. All applications are routed through Traefik's API gateway, which will remain the solution being used as they modernize their infrastructure. Traefik is a secure, performant, automated, and infrastructure-agnostic solution. It is part of a thriving ecosystem that allows the team to tap into a vast array of capabilities and integrations with cloud native solutions. As Axione continues scaling and modernizing its infrastructure, it will continue relying on Traefik.

“We started our journey with Traefik a year ago and are now renewing the contract for three years. We would definitely recommend Traefik's API gateway.”

Neil Orley, Head of SRE, Axione

]]>

Reducing Your Infrastructure Costs by Consolidating Networking Tools

Christian Turnipseed — Fri, 24 Mar 2023 11:11:12 GMT

Migration to the cloud and containers has left a scattered ingress landscape. As a result, modern infrastructure architectures rely on several different networking technologies and tools. Most infrastructures utilize four major networking tool categories: load balancers, reverse proxies, ingress controllers, and API gateways. However, many seemingly separate ingress tools have overlapping functionality. Identifying these overlaps brings an opportunity for significant cost savings.

These tools aim to simplify your overall routing strategy, primarily concerned with processing ingress traffic. They also balance traffic across multiple back ends, whether those are multiple applications or just multiple copies of our servers. All four tools also offer health checks to ensure they only route to valid healthy instances on the backend. What’s more, if we zoom in on reverse proxies, ingress controllers, and API gateways (excluding the load balancer), all of those use some combination of routing rules, providing a single entry point for multiple applications or APIs on the backend.

So, how do all these pieces play in as part of a networking stack, and how does a consolidated approach work?

The state of networking

In the complex world of cloud networking, it’s easy to end up with separate ingress controllers, proxies, load balancers, and other tools with overlapping functionalities. The diagram below shows a representative example of the infrastructure used by many organizations running a networking stack, for instance, in a public cloud provider.

The example architecture above includes a Kubernetes cluster with a number of pods that are running, user applications, as well as API services. These apps and services live behind an ingress controller in the cluster and a cloud load balancer that fronts the ingress controller itself. This infrastructure also includes a collection of API services running outside of Kubernetes — those could be running on virtual machine (VM) instances or containers outside of a Kubernetes orchestration platform. And last but not least, a set of virtual machine instances is used to run applications.

In this scenario, you would implement a reverse proxy in front of those VM applications, a load balancer across the copies of our API server, and an API gateway in front of the API instances and API pods in your cluster.

As you can see, this is getting complicated quickly, having to configure and manage different tools deployed into your stack that, in many cases, handle relatively similar or overlapping functionalities.

The multitude of tools used in an infrastructure like this one often translates to a multitude of different vendors, which in turn increases costs, configuration, and management complexity. Managing that level of complexity often requires a significant amount of internal IT resources in the form of dedicated representation for each tool or asset. Communication between different tools and vendors is also tricky and may require a lot of internal resources to implement workarounds that guarantee each tool works the way you need it to.

Key considerations for consolidation

Consolidating tools is a process that takes time to happen. You and your team need to identify tools that can take over multiple roles within your network (effectively replacing the various tools you currently use), fill gaps where needed, and formulate a migration plan. But how do you reach the decision to consolidate? How and when is it the right option for you? And where to begin? Here are five key questions that will help you kick off your consolidation efforts.

Where do tool functionalities overlap in my network architecture?
Which tools are flexible enough to fill multiple ingress network roles?
What constraints exist that inform where tools can be consolidated? (networking rules, compliance, performance)
Where should my consolidated network stack run primarily?
Do I need multiple layers of load balancing?

Once you have answered these questions with the help of your IT team, move on to the four main steps of consolidation.

Diagram network flows. Visualize your infrastructure by mapping out your network flow.
Identify duplicate components within the diagram of your architecture.
Pick a tool to use as a consolidated ingress.
Plan migration approach from current routing to consolidated routing.

The last step of consolidation (i.e., migration) is the most critical. This process is not something that you can plug and play overnight and magically solve all your issues. Teams have multiple projects to handle, sprint schedules to adhere to, deadlines to meet, etc. Migrations are never easy but sometimes necessary, the timing of a consolidation effort should be planned so that minimal downtime is achieved.

Consolidating networking tools saves you time and money

As it is universally accepted, time is money. The complexities of managing different tools and vendors weigh in on the time management of your IT staff. That is why consolidating your networking tools into a unified platform or tool that handles all of these major categories (load balancers, reverse proxies, ingress controllers, and API gateways) can be a game changer.

Let’s revisit that architecture diagram we went through earlier and take a look at how that

example architecture can be condensed down to a more streamlined, consolidated approach.

In the consolidated version of the example architecture, you can see that the separate tools for load balancer, reverse proxy, API gateway, and ingress controller are consolidated down to a single component.

While that component is within the Kubernetes cluster, labeled in the diagram as an ingress controller, it’s actually handling all of those different functionalities, doing load balancing to your instances on the backend, handling routing based on the reverse proxy-like rules either to your pods within the Kubernetes cluster, or the resources outside of it, and handling the routing for your non-Kubernetes workloads. It also handles the API gateway type functionality, so it handles your security measures — like authorization and authentication — or the rate limiting and more sophisticated traffic handling functionalities.

Note: In the example diagram, not every functionality has been consolidated down into a single ingress tool — the load balancer still lives in front of the consolidated ingress controller. The reason for this is that, in many cases, multiple layers of load balancing may need to happen. So, that means having a consolidated ingress tool handling most of your routing and load balancing for your existing APIs and applications but still needing some load balancing for the ingress controller components.

At this point, it is important to note that consolidating all the separate tools into a single component requires a tool that is flexible enough to route to all those different instances — whether that means plugging in different service discovery mechanisms, being able to route statically to the different instances, handling higher level functions like security, authorization, and authentication, and any other functionality that’s going to be critical in selecting exactly what we’re going to consolidate on.

Consolidation with Traefik Enterprise

Consolidating your networking tools may sound like a lot of work, but its benefits come in many different forms. In my day-to-day talks with customers, these are the main benefits that I most encounter that resonate with them as their key reasons for moving to a consolidated networking solution like Traffic Enterprise:

Cost-saving in the form of reduced cloud spend or on-prem hardware reallocation
Time-saving on the management of separate networking tools
Fewer network hops and reduced latency improves overall performance
Easier enforcement of security standards

Traefik Enterprise is a unified enterprise-grade solution for ingress and API gateway functionality with a distributed architecture for High availability and enables you to scale with your business needs. With these features in mind, let’s revisit that initial architecture diagram once more and look at how exactly you would use a tool like Traefik Enterprise to be the consolidated ingress and API gateway solution in this type of consolidated network architecture.

Traefik Enterprise leverages the multiple providers that can be used for service discovery — i.e., the ingress definition for your routing rules or an ingress route custom resource. Outside of Kubernetes, Traefik Enterprise leverages a file provider or a KV store to point to your non-Kubernetes workloads while still being able to route all of those different routing rules from a single consolidated ingress rule. It also allows you to handle those API gateway-type functionalities — handling various methods of authentication and authorization through middlewares, handling, automated TLS, termination, and certificate management, as well as rate limiting, and a bunch of other features you would expect from an API gateway.

If you want to learn more about how Traefik Enterprise can help your team reduce infrastructure costs, don’t hesitate to book a demo with us or start your free 30-day trial today!

]]>

Building Your Networking Strategy Using Multi-Cloud, Hybrid Cloud or Multi-Orchestrator Architectures

Kate Mikula — Tue, 07 Mar 2023 15:23:21 GMT

As businesses and their applications grow, so do the demands placed on their networks. Networking strategies are required to manage so many services spread across such a broad surface. But with so many kinds of networks out there, it isn’t always obvious which kind your organization is managing.

The terms hybrid cloud, multi-cloud, and multi-orchestrator are often used interchangeably, but they are three distinctly different architectures used in networking strategies, each with its own advantages and disadvantages. In this article, we will explore each in detail and highlight the differences between their basic and unified forms.

Hybrid cloud architectures

Hybrid cloud architectures run workloads in both data centers controlled by the organization as well as in public or private cloud infrastructures. There are many reasons a company would pursue this architecture.

A hybrid cloud setup may be a temporary phase of a wider cloud migration project. For example, a company may run legacy services in VMs or bare servers in an internal data center while also introducing microservices to a container orchestrator in a public cloud provider. A hybrid cloud setup may also be required in industries, such as healthcare and financial services, that handle sensitive information and are highly regulated. Sensitive information can be stored in internal data centers, while less sensitive information in the cloud.

Let’s examine what basic and unified hybrid cloud architectures look like.

What is a basic hybrid cloud architecture?

Hybrid cloud architectures combine public and private clouds. An example of an architecture might have some legacy services running in VMs in a data center with a reverse proxy routing traffic to individual services and also have a Kubernetes cluster running applications within pods in the cluster. An ingress controller handles a similar role to the reverse proxy on-prem, handling incoming requests and routing them to the correct pod in the backend. DNS records point traffic to the cloud and data center. This architecture is usually the first step in a cloud migration process.

Thee are several challenges to be aware of when adopting this architecture, such as how to handle the initial routing by environment. The DNS will route traffic based on the environment within which an application is living. The routing infrastructure is redundant, as the ingress controller and reverse proxy handle similar functions but in different environments.

What is a unified hybrid cloud architecture?

A unified hybrid cloud architecture is a variation of a basic hybrid cloud architecture that consolidates the routing approach. A centralized ingress controller handles everything, routing traffic from the internet to internal pods within a cluster and also to legacy services within a data center. This approach is simpler to manage, as there are fewer routing tools and infrastructure components to manage. This reduces maintenance and room for error.

Pick a unified ingress tool that is compatible with different workload types. It should be able to handle Kubernetes service discovery as well as the routing of traffic to static IPs or host names in VMs in the data center. While less maintenance is required, more planning is involved.

Ask yourself a few key questions when adopting a unified ingress tool. Is your application particularly sensitive to latency? Do you need to think about networking and firewall rules, so ports and protocols open effectively to communicate across different environments? Is everything that shouldn’t be open for communication secure? Your chosen tool should effectively and securely handle separate workload types across multiple environments.

Multi-cloud architectures

Rather than running a set of workloads in a single cloud provider, companies leveraging multi-cloud architectures use several. They might have one Kubernetes deployment running in GKE and GCP, some containers running as ECS services in AWS, and maybe a few VMs running in Azure. Obviously, these workload types are interchangeable across different cloud providers. Hyperscale cloud providers have long competed with each other for market share, and multi-cloud architectures allow companies to take advantage of several.

There are many benefits to this approach. Multi-cloud architectures increase resilience and fault tolerance. While you hopefully don’t need to plan for a major public cloud outage, it’s still a good idea to have a risk mitigation strategy that spreads your deployments across vendors. A multi-cloud architecture could be part of an overall strategy for cost optimization, as different cloud providers might have comparable services at varying prices. You may also want to take advantage of unique services offered by different providers. Multi-cloud architectures also help you avoid the vendor lock-in that comes from putting all your eggs in one basket (just be careful of avoiding multi-cloud lock-in).

What is a basic multi-cloud architecture?

Let’s walk through an example of a standard multi-cloud architecture, which takes a DNS-based approach.

In the diagram below, a few ECS services live in an ECS cluster, a few Kubernetes pods in a GKE cluster, and a few VMs in Azure. In front of each set of workloads is a reverse proxy or ingress controller. The two terms can be used interchangeably here as they both transmit incoming requests to the correct back-end service. The biggest challenge to overcome with this approach is that it is prescriptive, meaning the routing options are somewhat inflexible.

What is a unified multi-cloud architecture?

A more evolved multi-cloud architecture adds a reverse proxy in the front that can be thought of as an edge reverse proxy or a layer one reverse proxy. The reverse proxy handles the initial routing to individual clusters and unifies routing.

A unified multi-cloud architecture allows for far more sophisticated load balancing schemes. For example, you can expose and route unified API host names across multiple clouds by leveraging path-based routing. You can abstract the location of the actual services away from client requests.

Unified multi-cloud architectures can increase latency, as they add an additional stop to the routing path. They also require manual configuration to keep the list of locations in the first layer up to date — unless you have a tool that supports the dynamic syncing of service lists. Either way, make sure you get up to speed on best practices for networking and security.

Multi-orchestrator architectures

Multi-orchestrator architectures consist of applications running as containers within more than one container orchestrator. This could take the form of different orchestrators running within the same cloud or environment, and it could also overlap with a hybrid or multi-cloud architecture.

For example, you could have a greenfield application running in a cloud-based Kubernetes cluster as well as existing services in Docker Swarm. A multi-orchestrator architecture could be temporary if you’re migrating between orchestrators. You might be consolidating on Kubernetes from Docker Swarm. This architecture might be less temporary if you have some applications with specific requirements that prevent full standardization on a particular orchestrator type.

Multi-orchestrator architectures must be unified, as it’s very difficult to network traffic without a consolidated solution. In a unified multi-orchestrator architecture, a reverse proxy sits in layer one to distribute traffic to reverse proxies and ingress controllers associated with different orchestrators. In the diagram below, we have Docker, Kubernetes, and Nomad.

The benefits and challenges of multi-orchestrator architectures are similar to those found in a multi-cloud setup. This approach allows for flexibility. Networking updates are more readily changeable. If you choose to adopt a multi-orchestrator architecture, it’s important to select tools that can handle service discovery across different orchestrators, and that can hook into various orchestrator APIs. Automation is important too, as you will have to think about manual configurations if you don’t have a tool that automates the process for you. It will allow you to centrally configure security across all clusters.

How to evolve your advanced networking architecture with a multi-layered setup

Organizations managing advanced architectures will gain much from unifying their network. In hybrid cloud, multi-cloud, or multi-orchestrator architectures, a consolidated solution will increase efficiency and enable automation at scale. By adding a second reverse proxy or ingress controller to a second layer in front of your applications, you can unify your architecture.

Traefik Enterprise is a unified cloud native networking solution that brings ingress control, API management, and service mesh together in one simple control plane. It eases microservices networking complexity for developers and operators across an organization. The Traefik Provider allows organizations to build and operate a multi-layer architecture, and many companies have found success in doing so. AmeriSave uses Traefik Enterprise to build a two-layer architecture within which they can seamlessly transfer traffic between Kubernetes clusters in Microsoft Azure and Docker clusters in on-prem environments.

In many organizations that have reached a certain scale, more than one architecture will be in play. Some of the approaches we’ve looked at are a temporary product of a migration effort. Others are set up intentionally for the long term.

Think about your overall routing strategy when architecting. Make sure security is in place throughout. Learn whether it’s important to consolidate your networking solution with a multi-layered setup. Whether you embrace a hybrid-cloud, multi-cloud, and/or multi-orchestrator architecture, choose tools that are compatible with your setup and that simplify its management and configuration.

]]>

Unlock the Potential of Data APIs with Strong Authentication and Traefik Enterprise

Matt Elgin — Tue, 28 Feb 2023 14:00:00 GMT

When it comes to enterprise IT infrastructure, security is of paramount importance. Between the need for data protection and privacy, regulatory requirements, and the constant threat of bad actors on the network, there is little room for error when designing and maintaining enterprise systems.

Because of this, strong authentication is a critical component of any IT modernization project. One of the top goals for enterprises today is to open up the data held within legacy systems and expose it through APIs, microservices, and other modern means. And yet, while this data represents untapped business value, it’s essential to only expose it in controlled ways by using authentication to ensure each request’s validity.

Traefik can help. As a modern, cloud native edge router, Traefik Proxy directs valid requests from the external network to applications and services, while minimizing the risk posed by malformed, malicious, or fraudulent requests. One way it can do this is by acting as an intermediary to ensure that transactions are authorized. What’s more, Traefik Enterprise bundles additional, exclusive features to provide enterprise-grade authentication — including support for OpenID Connect (OIDC).

Who goes there?

One of Traefik’s key concepts is its use of middlewares, which are pluggable components that provide conditional controls over network traffic. These controls can take various forms, including enabling security features such as rate limiting, restricting requests by IP address, and authentication.

Traefik Enterprise's authentication middlewares work by referencing external authentication sources. Traefik Enterprise can act as a gatekeeper at the edge of the internal network by intercepting incoming requests and authenticating them against the external source before forwarding them to the appropriate applications.

This model can be particularly critical for legacy modernization projects because it allows authentication to occur externally to the application. You can add modern authentication methods to legacy applications to satisfy the latest security requirements, without making any direct modifications to legacy code.

But this model is not only beneficial for modernizing legacy applications. The benefits of standardizing authentication and authorization at the API gateway level apply equally to cloud native and legacy projects. Those benefits include reducing/eliminating duplication of effort, promoting compliance with security standards, and freeing up developers to work directly on the end applications instead of security features.

Enterprise options

Traefik Enterprise offers several middlewares for enterprise authentication, and the collection continues to grow. Among the methods that Traefik Enterprise supports are:

JSON web tokens (JWT)

JWT is a popular tool used to authenticate API calls and single sign-on (SSO) applications. It’s a method of digitally signing information as a JSON object. The JWT includes a set of claims, which typically describe the things that an authenticated user is allowed to do. The Traefik Enterprise JWT middleware can be added to routers in the dynamic configuration and verifies that a token is provided in the Authorization header. In case the token can't be passed as an Authorization header, you can also add it as form data or as a query parameter.

OpenID Connect (OIDC)

Traefik Enterprise also includes support for OIDC, an authentication layer built on top of the OAuth 2.0 protocol. OpenID Connect allows an application to obtain user login information by exchanging cryptographic tokens with an identity provider and is often used to implement federated SSO between multiple applications. With the OIDC Authentication middleware, you can secure your applications by delegating the authentication process to an external provider (e.g. Google Accounts, LinkedIn, GitHub, etc.) and obtaining the end user's session claims and scopes for authorization purposes.

Lightweight Directory Access Protocol (LDAP)

To verify user credentials (i.e. usernames and passwords) LDAP connects with a directory service that uses the LDAP protocol. The Traefik Enterprise LDAP middleware connects to an LDAP server to verify said credentials and was designed to avoid having sensitive information — such as LDAP credentials specified as labels (or in CRDs) by applications — and to allow multiple middlewares to reuse the same authentication method.

oAuth2

OAuth2 is an open standard dealing with resource access control and is the latest version of the authorization protocol OAuth. An OAuth client provides web, desktop, and mobile application authorization flows. Traefik Enterprise comes with two oAuth2 middlewares:

OAuth 2.0 Token Introspection Authentication Middleware: Retrieving metadata about an access token from an oAuth2 server which can be used to restrict access to applications.
OAuth 2.0 Client Credentials Authentication Middleware: Securing routes using the OAuth2 Client Credentials flow described in RFC 6749.

Open Policy Agent (OPA)

OPA is an open source policy engine. By providing a high-level declarative language, OPA helps unify the processes of policy enforcement across the networking stack. OPA allows you to specify policy-as-code and APIs you can use to offload policy decision-making and, among others, it works in microservices, Kubernetes, CI/CD pipelines, and API gateways. With the Traefik Enterprise Open Policy Agent middleware, you can restrict access to your services and enrich request headers with data extracted from policies.

Hash-based Message Authentication Codes (HMAC)

HMAC is a method of using cryptographic hash functions with a shared secret (also known as a symmetric key) to ensure the content delivered in an HTTP request is valid and genuine. Like digital signatures, HMAC can verify a message sender’s identity and that the message’s content is unaltered from the moment of the HMAC’s creation. The technique can be used to secure file transfers, API calls, and other machine-to-machine interactions. This HMAC middleware uses the content of an HTTP request and a shared secret to validate a digital signature computed. The HTTP request and the shared secret are sent to the proxy using the Authorization or Proxy-Authorization header, ensuring the identity of the sender and the integrity of the request.

Authentication the easy way

The best thing about implementing enterprise authentication using Traefik Enterprise, however, is how easy it is to do. Enabling any of the authentication middleware mentioned here is generally as simple as adding a few lines to your Traefik configuration to supply the necessary connection details, creating a middleware that points to your authentication source, and attaching that middleware to desired routers.

The authentication options available in Traefik Enterprise today offer a powerful range of options for exposing enterprise applications and data securely, without requiring extensive and risky legacy code changes. You can expect other such features to be included over time, as we continue our commitment to ensure Traefik Enterprise is a premier tool for enterprise application networking. To learn more about how Traefik and Traefik Enterprise can help you lock down enterprise data with secure authentication, watch our recent webinar, “Enterprise best practices to expose and secure microservices and APIs”. In this webinar, we discuss deploying OAuth and OpenID Connect with Okta to secure user logins, and we walk you through enabling mutual TLS (mTLS) for secure machine-to-machine communications.

]]>

Achieve Zero Downtime Access Control for Your Applications

Julien Salleyron — Tue, 21 Feb 2023 10:27:26 GMT

Access control secures your public-facing services and it is critical for all modern web applications. And it goes without saying that security is priority no.1 for the Traeifk Labs team and our users, that's why we provide access control as a built-in feature for Traefik Hub.

If you don’t already know our motto, Traefik Labs is on a mission to Make Networking Boring! Just providing built-in access control features wouldn’t cut it — it had to be simple and easy for the user to implement. So, we made sure Traefik Hub users could implement access control for their public-facing applications in just a few clicks, with zero downtime, and no changes to the original application.

But wait! What do you mean by “access control” and how does it work, I can hear someone asking.

Let’s take it from the beginning, shall we?

Defining access control

Access control is the mechanism you implement between your application and the outside world to control and secure who can access your application. Needless to say, access control is critical. Without it, anyone can potentially gain access to your application and try to hack your data or break your system.

Implementing access control for a single application is (almost) a piece of cake. Now, setting up access control for applications within a microservice architecture? That is a whole different monster. To properly protect your microservice applications, you need to implement access control policies for every single application in your system. Leave one application unprotected and you open a hole in your system, easily exploitable by malicious attackers.

Not to mention, choosing a way to secure your applications — Google authentication, OpenID Connect (OIDC), etc. — comes with its own configuration hustles.

That’s where Traefik Hub comes in to simplify the way you set up and implement access control.

Complicated access control mechanisms are a thing of the past

The Access Control Policy (ACP) feature in Traefik Hub is based on the forward authentication mechanism. This means that your reverse proxy, or ingress controller in Kubernetes, delegates the authentication process to the Traefik Hub agent to allow or deny access to your application.

This way you can add advanced authentication mechanisms, like OIDC or JWT, on any application — yes, even legacy apps!

When you enable an ACP on one of your services, Hub automatically configures the ForwardAuth feature on the reverse proxy you used to expose your application. ForwardAuth calls a URL on the Traefik Hub agent to validate and process the authentication workflow. Once this process is complete, and if the user is authorized, they get redirected to your application.

Bottom line, the only thing you had to do was to configure your Access Control Policy within the Traefik Hub UI, and enjoy the fact that your services are now protected in just a few clicks — with zero configuration changes to your applications or your infrastructure!

But that’s enough talk — let’s get down to business and see the ACP feature in action. I have two examples prepared for you: a homelab setup and a production use case.

Let’s jump in.

Access control for your homelab project

Let’s begin with a simple example of how to implement access control with Traefik Hub. In this example, I will show you how to publish a basic homelab project. For this example, I am using a simple motion-detecting application. To keep it simple, in my example I will only use this app to expose my camera. You can also configure the app to detect motion on your camera but we will not use this feature here.

Before we get started, make sure you have the following installed on your machine:

Docker
docker-compose

Create your Traefik Hub cluster with docker-compose

The first thing you need to do is to create your Traefk Hub account and install your first Traefik Hub Agent.

Once the agent is installed, you can deploy the Traefik Hub Agent in your cluster. In this example, I am using the docker-compose mode. Copy and paste the proposed docker-compose command to deploy all this directly in your cluster.

Now let’s create the configuration file for our motion application to expose the first camera. Create a motion.conf file with this content:

stream_port 8080
stream_localhost off

Then, we add our application in the ./docker-compose-hub-agent.yaml file created by this command.

# start the motion project (with privileged: true to simplify the camera configuration)
  motion:
	image: motionproject/motion:latest
	volumes:
	- ./motion.conf:/usr/local/etc/motion/motion.conf
	privileged: true

Now let’s reload the docker-compose project:

docker-compose -f ./docker-compose-hub-agent.yaml up -d

Note I: Here, for the sake of simplicity, I use the same docker-compose for both my Traefik Hub Agent, Traefik Proxy, and MyApp. If you want to use a different docker-compose for each, don't forget to add the motion network on the Traefik Proxy container to allow Traefik Proxy to contact MyApp.

Note II: If you already have a Traefik Proxy instance installed, you can add the Traefik Hub configuration and reuse it instead of deploying a new one.

Publish your app

To publish your application, click on the Service name that represents your application, then click the Publish the service button.

Finally, type the port we configured for our motion application, 8080, in the Service Port field, and click Save and Publish to publish your application.

Go to the URL Traefik Hub published for you, and you should see your camera.

If you take a look at the URL, you may notice that it uses HTTPS with a valid certificate for your domain. However, this URL is not protected at all. To keep malicious individuals from watching your camera, you need to set an Access Control Policy.

Set up access control with OIDC to protect your service

For this application, let’s add an ACP with OIDC and Google.

First, go to https://console.cloud.google.com/apis/credentials and create an OAuth client ID in the Create credentials section.

Note: You need to configure the redirect URL with the domain of your published service followed by /callback.

Now head back to the Traefik Hub and create your new ACP using the OIDC method. In the Provider field, select OIDC Google. Fill out the Client ID and Client Secret with the info you got from Google Credentials, and configure the redirect URL with /callback (the path we configured in Google Credentials).

Add the users you want to authorize to access your application and save the ACP.

All is done! Your service is published and secured thanks to your OIDC Access Control Policy.

Publish and secure your production API

Implementing access control for a simple homelab project was insanely simple, right? Now, let’s take it up a notch! Taking our example from your homelab to your production environment, let’s say you have a Kubernetes cluster with a service you want to publish and secure.

Before we get started, make sure you have your Kubernetes cluster up and running.

Create your Traefik Hub cluster with Kubernetes

Create your Hub cluster, just as we did earlier, choose the Kubernetes platform, and apply all the relevant snippets. This will deploy the Traefik Hub Agent and Traefik Proxy as the ingress in your cluster.

Here’s an example of a service:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: whami
  name: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - image: traefik/whoami
        name: whoami
        imagePullPolicy: Always
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  ports:
  - name: web
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: whoami

The only thing you have to do now is to find your service in the Traefik Hub UI and publish it.

You can verify that it works with a curl call.

Excellent! Now that your service is publicly available, let’s secure it with an ACP.

Set up access control with JWT to protect your service

In this example, we will secure our service with the JWT Token Mechanism. Click on the Add Access Control and select Create New ACP. Here, we’ll create a new JWT policy with a signing secret.

You can now add some forward headers, based on the claim you will add in the JWT token (see the step below).

Now let’s try to call this service without a JWT Token.

Indeed, we are not authorized to call this URL. Let’s add the JWT token. You can generate the JWT Token on jwt.io with a claim for name and the right signing secret. Then call the URL and add the Authorization Header.

Bonus round I

So, we just published a service and secured using ACP and we did so through Traefik Hub’s tunneling mechanism.

But ACPs in Traefik Hub don’t necessarily have to use this tunneling mechanism. If you already have an ingress controller and an ingress set up to publish your service, you can add ACPs to your already existing ingress directly in the Traefik Hub UI.

As you can see, when I tried to call the URL defined in my ingress, the JWT token and headers from the claim were automatically forwarded.

Bonus round II

If you don’t want to use the Traefik Hub UI, but still want to take advantage of Traefik Hub’s ACP feature, you can use the AccessControlPolicy CRD directly in your cluster and apply it with the kubectl command.

apiVersion: hub.traefik.io/v1alpha1
kind: AccessControlPolicy
metadata:
  name: jwt
spec:
  jwt:
    forwardHeaders:
      name: name
    signingSecret: my signing secret

Then you can add an annotation on your ingress to apply this ACP on your ingress
hub.traefik.io/access-control-policy:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-whoami
  annotations:
      hub.traefik.io/access-control-policy: jwt

spec:
  rules:
    - host: whoami.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80

Conclusion

And that’s how you do it!

Simple and fast, Traefik Hub helps you secure your publicly available applications in, literally, a few clicks. If you want to learn more about the power of Traefik Hub, we have a bunch of resources that I recommend you check out:

Documentation — Welcome to Traefik Hub
Blog Post — Traefik Hub in a World of GitOps

If you want to jump right in and start trying it out yourself, create your free account today!

]]>

The History and Evolution of APIs

Kate Mikula — Tue, 07 Feb 2023 12:39:28 GMT

APIs have become an important part of the tech landscape. They’ve been around for decades and have evolved a fair amount since their inception. API stands for Application Programming Interface, which is a number of definitions for building application software. APIs use standardized protocols to allow services to communicate with each other without having to know how others are implemented.

In this blog post, we will explore their history in order to understand how and why they have become such an integral part of modern software development.

1950s: The idea of an API was born

Back in the 1950s, an API was understood as a potential method to facilitate communication between two computers. The term was first mentioned in a 1951 book written by Maurice Wilkes and David Wheeler called ‘The Preparation of Programs for an Electronic Digital Computer.’ It outlined several key computing terms, including an early version of an API. At this stage, an API was starting to exist, but they were limited to simple, command-line interfaces that enabled programmers to interact with computers. These early APIs set the stage for variations of APIs to crop up in the future.

1960s and 70s: The concept of an API evolved

Throughout the 1960s, computers started growing in popularity, and organizations were beginning to experiment with their uses. The term API was understood at this time as the interaction of a single application with the rest of a computer system. By introducing a consistent application interface (usually Fortran subroutine calls), programmers could free themselves from the idiosyncrasies of the graphics display device. APIs were used to enable communication between mainframes and other systems, such as terminals and printers. They could benefit from hardware independence if a computer or display needed to be replaced. In 1974 the term API was introduced in a paper called ‘The Relational and Network Approaches: Comparison of the Application Programming Interface.’ It introduced APIs to the world of databases. It became evident that APIs could combine different interfaces to support all types of programming.

1980s: APIs set the stage for the internet

In the 1980s, computer networks started becoming commonplace and programmers needed to access libraries located both in their local computers and also in computers elsewhere. APIs enabled Remote Procedure Calls (RPCs) that were generally supported by Java. APIs played a critical role in enabling cross-platform compatibility, allowing developers to create applications that could run on multiple platforms.

1990s: APIs started growing in popularity

The 1990s saw the birth of the internet, and APIs were a way for applications to exchange data with the use of a standard set of protocols. Instead of only enabling an application to share messages with the rest of its computer system, APIs allowed applications to share messages with applications in other computer systems across the internet. Carl Malamud described APIs as “a set of services available to a programmer for performing certain tasks.” APIs were still in their early stages, but they were to pave the way for the emergence of web services in the next decade.

2000s: API adoption allowed new business models to take off

In the early 2000s, the internet was taking off, and developers were looking for ways to simplify web development and programming. In 2000, a dissertation by Roy Fielding called ‘Architectural Styles and the Design of Network-based Software Architectures’ defined REST as the protocol of choice, allowing for standardized communication between devices across the internet.

As online applications grew in popularity, organizations started moving everything to the cloud. Salesforce, eBay, and Amazon pioneered the delivery of services, using HTTP to provide access to machine-readable data in a JSON or XML format through web APIs. Very soon, both innovative startups and large-scale enterprises were implementing as-a-Service offerings that leveraged the cloud and its API-first model. By the time Apple launched the iPhone in 2007, APIs had already revolutionized the way companies deployed infrastructure.

Not only were APIs infiltrating SaaS-based applications, but they were also being built on top of platforms that leveraged APIs. Amazon was one of the first cloud providers to pioneer an API-focused determination, mandating all shared digital resources to have an API. Others followed their lead, inspired by Amazon Simple Storage (S3) and Amazon Elastic Compute (EC2).

2010s: APIs continued evolving

By 2010, social media was overtaking the population, and its applications were setting the stage for a new generation of APIs. As the demand for efficient, cost-effective, and highly scalable applications became urgent, APIs made it easy for organizations to integrate their information systems with third-party services (such as payment processors and CRMs) as well as API-driven cloud platforms.

When Kubernetes entered the scene in the mid-2010s, it encouraged a move towards distributed systems consisting of loosely-coupled, ephemeral microservices, each implementing its own API. APIs empowered organizations to extend the reach of their web applications to a global user base. As APIs were based on standardized protocols, they enabled developers to create applications in multiple environments and with multiple services.

2020s: Modernizing APIs

By the time COVID-19 heightened our dependence on web services, APIs continued growing in sheer usage as systems continued being broken down into more microservices. Today, they are becoming vital for Internet of Things (IoT) devices and are building an interconnected yet distributed world of data. Moreover, they are increasingly important for the construction of AI.

Organizations are running applications across different cloud providers in distributed systems, and effective APIs are key for making sure all services can communicate with one another. They rely on well-designed APIs to reap the benefits of cloud native applications, such as scalability, performance, and security. Today, rather than starting with the application itself, developers adopt an API-first approach, designing APIs first and then building applications around them.

The future evolution of APIs

The evolution of APIs has so far been a long and winding road. APIs have been around for many years and have become the standard for businesses to remain digitally relevant. As API adoption continues growing, APIs themselves are evolving in two ways to address some of their inherent challenges.

Firstly, security is becoming more important each year, itself being a byproduct of the proliferation of APIs. As systems become increasingly distributed, hackers have a much wider surface area to attack meaning far more vulnerabilities to exploit. This makes the case for API security imperative.

Zero-trust security is a hot trend at the moment as it addresses these concerns and will likely continue growing in adoption as a result. Zero-trust is essentially the principle that you should never trust, by default, any entities in your infrastructure or network. Even if a request entering your application comes from a client inside your network, you must verify the client is both authenticated and authorized. Your application needs to know who they are and that they have the right permissions. You must also make sure all network calls are encrypted.
Rate limiting middleware is used to keep APIs secure, as it minimizes the risk of a server crashing from a Denial of Service (DoS) attack. In these attacks, the attacker either floods your server with more requests than it can handle or transmits requests that prevent your user from accessing your application.
Observability is becoming even more critical for the evolution of APIs. Without effective methods for observability, you have no way to identify and troubleshoot breaches or incidents as they occur.

Secondly, API governance will continue to be a key focus in the future. API governance is a discipline that involves setting policies, standards, and guidelines for the development, deployment, and maintenance of APIs within an organization. The primary motivation for API Governance is to ensure that APIs are designed and implemented in a consistent, secure, and scalable manner that aligns with the overall goals and objectives of the organization.

Acing API management in your organization

Distributed systems are innately complex, and API management is needed more than ever. Effective API management is the action of publishing, securing, managing, and observing APIs as they interact with a myriad of applications and services. It involves integrating an API gateway, which is a technical layer that sits between the API consumers and the API provider, providing features such as security, traffic management, and request/response transformation.

Traefik Enterprise is an all-in-one ingress controller, service mesh, and API gateway created to ease microservice complexity. It helps you ace API management by focusing on the operational aspects of APIs, such as monitoring, performance, and analytics.

APIs have allowed developers to create more powerful and efficient applications quicker than ever before with ever-increasing abstraction levels. APIs truly have become the core of modern software development. What started out as a way for two computers to exchange data has become the foundation for modern applications and services, especially in distributed systems.

]]>

Reverse Proxy vs. Ingress Controller vs. API Gateway: Understanding the Differences and When to Use Them

Eirini Eleni Papadopoulou — Tue, 31 Jan 2023 14:26:44 GMT

If you are new to the networking ecosystem and you are trying to figure out what networking pieces you need and when, I’m sure you’ve come across these three terms: reverse proxy, API gateway, and ingress controller. And just like all of us when we first started in this industry, you must feel confused about what each of them means, what are the differences between them, and when you need to use which.

But fear not, my friends. Sit tight and allow me to help you through these confusing times by diving into the fundamentals of the networking stack.

Let’s start by breaking down every term, what it is, and its common use cases.

Getting to know the reverse proxy

You can think of the reverse proxy as that old-school phone operator, you know, back when there used to be call centers and phone operators. Back then, when someone was picking up the phone, they were connected to a call center, the caller stated the name and the address of the person they wanted to call, and the phone operator connected them. A reverse proxy does a similar job by receiving user requests and then forwarding said requests to the appropriate server, as you can see in the diagram below.

Reverse proxies have a very simple function — they are used in front of an application or group of applications and act as the middleman between the client and the application.

As I mentioned earlier, reverse proxies route user requests to the appropriate server, assuming that you are utilizing multiple servers. So, naturally, those of you using a single server are probably wondering whether or not it makes sense for you to even implement a reverse proxy. In fact, reverse proxies are useful even in single-server scenarios where you can take advantage of features like rate limiting, IP filtering and access control, authentication, request validation, and caching.

If you want to dig deeper into reverse proxies and how they differ from forward proxies or load balancers, check out our reverse proxy 101 article.

Getting to know the ingress controller

In a nutshell, an ingress controller is a reverse proxy for the Kubernetes universe. It acts as a reverse proxy, routing traffic from the outside world to the correct service within a Kubernetes cluster, and allows you to configure an HTTP or HTTPS load balancer for the said cluster.

To better understand this, let’s take a step back first and look at the Ingress itself. A Kubernetes Ingress is an API object that determines how incoming traffic from the internet should reach the internal cluster Services, which then in turn send requests to groups of Pods. The Ingress itself has no power over the system — it is actually a configuration request for the ingress controller.

The ingress controller accepts traffic from outside the Kubernetes platform and load balances it to Pods running inside the platform, this way adding a layer of abstraction to traffic routing. Ingress controllers convert configurations from Ingress resources into routing rules recognized and implemented by reverse proxies.

Ingress controllers are used to expose multiple services from within your Kubernetes cluster to the outside world, using a single endpoint — for example, a DNS name or IP address — to access them. Specifically, ingress controllers are used to:

Expose multiple services under a single DNS name
Implement path-based routing, where different URLs map to different services
Implement host-based routing, where different hostnames map to different services
Implement basic authentication or other access control methods for your applications
Implement rate limiting for your applications
Offload SSL/TLS termination from your applications to the ingress controller

Ingress controllers are undeniably one of the most complicated pieces of a networking stack. You can find more details about how Ingress and ingress controllers work in our Kubernetes ingress controller 101 article.

Getting to know the API gateway

Deployed at the edge of your infrastructure, an API gateway acts as a single entry point that routes client API requests to your backend microservices. Essentially, an API gateway is a reverse proxy handling incoming user requests and, although it includes many of the functionalities commonly found in reverse proxies, there is a key difference between the two.

Contrary to reverse proxies, API gateways have the ability to address cross-cutting, or system-wide, concerns. Concerns refer to the parts of your system's architecture that have been branched based on its functionality. Cross-cutting concerns are concerns that are shared among a number of different system components or APIs and include, among others, configuration management, security, auditing, exception management, and logging.

API gateways are commonly used in architectures where you need to expose multiple microservices or serverless functions to the outside world through a set of APIs, and they handle a number of tasks. On top of the functions we already saw as part of a typical reverse proxy, API gateways can handle:

Load balancing: Distributing incoming traffic across multiple servers to improve performance and availability.
Rate limiting: Matching the flow of traffic to your infrastructure’s capacity.
Access control: Adding an extra layer of security by authenticating incoming connections before they reach the web servers, and by hiding the internal IP addresses and network structure of the web servers from external clients.
SSL/TLS termination: Offloading the task of handling SSL/TLS connections from the web servers to the reverse proxy, allowing the web servers to focus on handling requests.
Caching: Improving performance by caching frequently-requested content closer to the client.
Request/response transformation: Modifying incoming requests or outgoing responses to conform to specific requirements, such as adding or removing headers, compressing/decompressing, and encrypting/decrypting content.
Logging and monitoring: Collecting API usage and performance data.

API Gateways also come with a few extra handy functionalities, namely service discovery, circuit breaker, and request aggregation.

If you want to learn more about API gateways and the kinds of features you should expect from a modern API gateway, see this page.

Reverse proxy vs. ingress controller vs. API gateway

Now, to bring everything together, how do you differentiate those three? In essence, an ingress controller does the same job as a reverse proxy or an API gateway when it comes to handling incoming traffic and routing it to the appropriate server/Service. However, the ingress controller operates at a different level of the network stack.

What I mean here, of course, is that the ingress controller operates in a Kubernetes environment. In that sense, the ingress controller is a specific type of reverse proxy designed to operate within Kubernetes clusters. Just as the API gateway sits at the edge of your infrastructure, the ingress controller sits at the edge of the cluster, listening for incoming traffic, and then routes it to the appropriate Kubernetes Service within the cluster, based on the rules defined in the Ingress resource, as we saw earlier.

Bonus round: What about service mesh?

I know that we’ve covered a lot of ground already and I don’t want to scare you off but there is one more piece of the overall networking stack that I’d like to bring to your attention and that is the service mesh.

A service mesh is a piece of software that sits within the network and provides additional functionality for managing communication between microservices. It operates at the level of individual service-to-service communication and adds security, observability, and reliability to a microservices application.

There are also different types of service mesh architectures — sidecar proxy and host/node proxy — which I recommend you check out in our service mesh 101 article.

Can I use reverse proxies, API gateways, ingress controllers, and service mesh all at once?

The simple answer to this question is yes, you can! Long story short, ingress controllers are the Kubernetes-native way to configure your reverse proxy, the API gateway is a specific type of reverse proxy — a reverse proxy on "steroids,” if you will — while the service mesh is a network proxy tailored for microservices. Since all of those pieces operate on different layers of the application, you can use a combination of all four in your infrastructure. For example:

You can set up a reverse proxy in front of your ingress controller and API gateway to handle SSL/TLS termination, caching, load balancing, and request/response transformation.
You can set up an ingress controller to handle the routing of incoming traffic from your reverse proxy to the appropriate Kubernetes Service within your cluster.
You can set up an API gateway to handle authentication, rate limiting, and request/response transformation for your microservices within the cluster.
You can set up a service mesh to handle internal communication (i.e. load balancing, traffic shaping, and service discovery) between your microservices.

But doesn’t that add a ridiculous amount of complexity to my infrastructure, I hear you asking. Well, not necessarily. Although adding multiple layers of proxies and controllers can increase the complexity of your system, you won’t have to worry about it if you are using the right tools.

Traefik Enterprise is an all-in-one cloud native networking solution that brings the ingress controller, API gateway, and service mesh all together in one simple control plane. Built on top of open source Traefik Proxy and Traefik Mesh, Traefik Enterprise eases microservices networking complexity for developers and operations teams across your organization. Care to give it a try? Sign up for a 30-day free trial today!

]]>

Exploring the Tailscale-Traefik Proxy Integration

Mathieu Lonjaret — Tue, 24 Jan 2023 12:57:19 GMT

Last month, we announced the release of the first beta for Traefik Proxy 3.0, and with it came the exciting new integration with Tailscale, a VPN service that allows you to create your own private networks from your home, using whatever device you want.

But Tailscale goes beyond providing a service to create a private network. It also offers TLS certificate management, where Tailscale provides you with a valid certificate for your internal Tailscale services. Behind the scenes, Tailscale gets the certificate from Let’s Encrypt. The biggest benefit here is that Tailscale manages the certificate lifecycle for you, so there is no need to worry about renewing or exposing an endpoint to resolve TLS challenges between Let’s Encrypt and your proxy instance.

In this article, I want to show you the two main ways Traefik Proxy makes use of Tailscale — one based on the utilization of the TLS management feature and one bonus story for nerds!

You can also check out the announcement of the Tailscale-Traefik integration on the Tailscale Blog.

Tailscale as a TLS certificates provider

Tailscale is, first and foremost, a VPN, which means all traffic between the nodes of your tailnet is already encrypted by WireGuard. If you're running, for example, a webserver on one of your nodes (i.e. your server), and you want to reach it from another of your nodes (i.e. your laptop), there is no need for HTTPS, in terms of security, and you could do it over HTTP.

However, software at the application level (e.g. your browser) is unaware that traffic is already encrypted, and it might, rightfully so, "complain" about it — your browser will display the Not secure warning near the URL bar. Other tools might even be stricter about it.

For this reason, Tailscale also offers a (beta) feature for HTTPS certificates, which provides you with a Let's Encrypt TLS certificate for the nodes in your tailnet.
Once this feature is enabled, instead of your laptop reaching your server with http://your-server-tailscale-IP, you can reach it withhttps://your-server-tailnet-name — assuming your server can do HTTPS as well — making your browser happy, as it sees you are using TLS, and your life easier.

If you are interested in trying this feature without Traefik Proxy, you need to follow the steps below:

Set up the Tailscale bits
Set up your webserver (or reverse proxy) to handle TLS
Call tailscale cert on your server to ask Tailscale to provide you with the TLS certificate
Adjust your webserver configuration to take that certificate into account
Handle certificate renewal later on

Automating TLS certificates with the Traefik-Tailscale integration

Now, if you want to try this Tailscale feature with Traefik Proxy, you have a way of automating this process. Traefik comes with an ACME provider, which can be configured to automatically ask Let's Encrypt for certificates, for the relevant routes described on its configured routers.

In that respect, Tailscale's role as a certificate provider is very similar to Let's Encrypt, so it made sense for us to capitalize on the experience we already had with the ACME provider, and adapt the work to add the same feature for Tailscale.

Let's showcase the feature with an example of a setup from A to Z.

Start with the Tailscale part: In your tailnet’s DNS settings, enable MagicDNS, make a note of your tailnet name for later (in the example below, you'll have to replace yak-bebop.ts.net with your own), and enable HTTPS Certificates
Configure Traefik Proxy: We'll use the file provider for simplicity, but there are examples for the other providers on our documentation page that you can easily adapt for this example.

Static configuration:

[entryPoints]
	[entryPoints.websecure]
		address = ":443"

[providers]
	[providers.file]
		filename = "/path/to/your/dynamic.toml"

[certificatesResolvers.myresolver.tailscale]

[api]
	debug = true

[log]
	level = "DEBUG"

Dynamic configuration:

[http]
	[http.routers]
		[http.routers.towhoami]
			service = "whoami"
			rule = "Host(`myserver.yak-bebop.ts.net`)"
			[http.routers.towhoami.tls]
				certResolver = "myresolver"

	[http.services]
		[http.services.whoami]
			[http.services.whoami.loadBalancer]
				[[http.services.whoami.loadBalancer.servers]]
					# docker run -d -p 6060:80 traefik/whoami
					url = "<http://localhost:6060>"

Note: In the Host rule, we're using our full Tailscale hostname for the server — the concatenation of the server's machine name, myserver, (that you can find in your Tailscale admin console, or with the tailscale status command), and the tailnet name, yak-bebop.ts.net, that is provided with MagicDNS.

Start Traefik Proxy: On startup, Traefik should automatically try to get certificates for TLS routes with a Host rule, which, in our example, means that, if everything goes well, you should see log lines (if your log level is DEBUG) such as:

2023-01-09T11:02:51+01:00 DBG ../../pkg/server/router/tcp/manager.go:235 > Adding route for myserver.yak-bebop.ts.net with TLS options default entryPointName=websecure
2023-01-09T11:03:36+01:00 DBG ../../pkg/provider/tailscale/provider.go:253 > Fetched certificate for domain "myserver.yak-bebop.ts.net" providerName=myresolver.tailscale
2023-01-09T11:03:36+01:00 DBG ../../pkg/server/configurationwatcher.go:226 > Configuration received config={"http":{},"tcp":{},"tls":{},"udp":{}} providerName=myresolver.tailscale
2023-01-09T11:03:36+01:00 DBG ../../pkg/tls/certificate.go:158 > Adding certificate for domain(s) myserver.yak-bebop.ts.net

Enjoy your TLS route! You can now access your Tailscale hostname (https://myserver.yak-bebop.ts.net in the example) over HTTPS in your browser.

Tailscale as a tunnel between a Mac host and containers

The (convoluted) background

The Traefik project, just like most large software projects, has integration test suites that we run both on our development machines (mostly laptops), and automatically on our Continuous Integration (CI) platform when submitting a pull request, notably to detect regressions.

In our case, that usually means we use at least three components for a test: Traefik itself, a third-party component, like a backend (e.g. the traefik/whoami webserver), and the test itself, which can be mainly viewed as an (HTTP, or not) client that makes requests in Go.

For historic reasons, at some point, we ended up in a situation where all of these components would, by default, run in Docker. The rationale is the usual: you want reproducibility so that the setup is the same everywhere, and the test will run on the CI, as well as your laptop. And on your dev laptop, it also allows you to avoid the need to install and configure various third parties — think databases, for example.

However, on Mac machines, there are two (sort of interlinked) major drawbacks to that situation: slow run time, and inconvenient workflow.

When we are debugging, or working on a new feature, it is pretty important to be able to make a change, rerun one test in particular, and get some feedback in a decent amount of time. Otherwise, aside from being tedious, you're stuck in this in-between where there's not enough time to context-switch to something else, and it's not fast enough to stay in the ideal flow where you can keep on iterating.
Having Traefik Proxy in Docker is an obstacle for two reasons. First, the Docker image has to be rebuilt for every little iteration you want to try, which is automated but is still somewhat slow. Second, it means the whole test setup and run is also way slower than it should be, especially on Mac, mostly because there's a Linux VM in between.

As the vast majority of the Traefik team is using Mac now, this has become an annoying enough problem that we wanted to take care of it. And that is when one of us nerd-sniped another into using Tailscale. Why, you ask? Because of the aforementioned Linux VM.

See, the best of both worlds would be to keep the third-party backends in Docker (for convenience), but take Traefik and the client code out of Docker. This means the clients, and Traefik, have to be able to reach the backends in their containers (and sometimes vice-versa). On Linux, it is somewhat doable, but on Mac, it gets considerably harder, given that there is a Linux VM in between the Mac host and the containers. So, we wanted to see if Tailscale allowed us to achieve that with minimal work.

The solution

The basic idea that is key to the solution is another nifty Tailscale feature, the subnet router. If a Tailscale node sits in a container that is in the same Docker network as all our other Docker containers, then it can reach all these containers. And since it is also part of our tailnet, it can also reach our Mac host (assuming that this host is also part of the tailnet, of course), acting as a gateway between both networks. For the more technically inclined, most of the changes related to that idea are in this commit.

The gist of it is that now we have a flag (IN_DOCKER environment variable) which conveys the intent whether to build Traefik Proxy and to run the tests in Docker or directly on the host.

If not true, we look for a tailscale.secret file, which should contain a Tailscale auth key (ephemeral, but reusable). We then start, with the docker compose API, a tailscale/tailscale container in the same Docker network as the other containers, in which we run Tailscale with the auth key, and --advertise-routes=172.31.42.0/24, in order to make it a subnet router for all the Docker containers.

Finally, for the gritty details on the Tailscale side, I want to mention two things:

As seen above, you need to generate an ephemeral, reusable auth key, which can be done on your Keys page at https://login.tailscale.com/admin/settings/keys
You need an autoApprovers section in the ACLs, in order to automatically approve the routes to the subnet relay. For our purposes, it looks like this:

"autoApprovers": {
		// Allow myself to automatically advertize routes for docker networks
		"routes": {
			"172.0.0.0/8": ["your_tailscale_identity"],
		},
	},

And that's pretty much it!

So even if the idea of using a VPN to communicate between your host and some containers in Docker seems like overkill, it actually works! And it does make our life simpler as it considerably improves the feedback loop for us when iterating on tests.

Is there a better solution? Probably.

Would it require considerably more changes to our tests setup? Maybe.

Was it fun to do? Definitely 😉

Don’t forget to check out the announcement of the Tailscale-Traefik Proxy integration on the Tailscale Blog and their official documentation.

]]>

Distributed Tracing with Traefik and Jaeger on Kubernetes

Michel Loiseleur — Wed, 18 Jan 2023 16:43:53 GMT

Originally published: March 2021
Updated: December 2022

Hello, and welcome back to this blog series on Site Reliability Engineering and how Traefik Proxy can help supply the monitoring and visibility that are necessary to maintain application health.

In the first article, we discussed log analysis while the second covered Traefik metrics with Prometheus. In this article, we will explore another open source project, Jaeger, and how to perform distributed tracing for applications on Kubernetes.

What is distributed tracing?

Debugging anomalies, bottlenecks, and performance issues is a challenge in distributed architectures, such as microservices. Each user request typically involves the collaboration of many services to deliver the intended outcome. Because traditional monitoring methods like application logs and metrics tend to target monolithic applications, they can fail to capture the full performance trail for every request.

Distributed Tracing, therefore, is an important profiling technique that complements log monitoring and metrics. It captures the transaction flow distributed across various application components and services involved in processing a user request. The captured data can then be visualized to show which component malfunctioned and caused an issue, such as an error or bottleneck.

This post demonstrates how to integrate Traefik Proxy with Jaeger, an open source end-to-end distributed tracing application that is also a Cloud Native Computing Foundation (CNCF) project. The integration captures traces for user requests across the various components of a hypothetical application running on a Kubernetes cluster.

Prerequisites

This post will walk you through the process of integrating Traefik Proxy and Jaeger, but you'll need to have a few things setup first:

A Kubernetes cluster running at localhost. The Traefik Labs team often uses k3d for this purpose, which creates a local cluster in Docker containers. However, k3d comes bundled with the latest version of k3s, and k3s comes packaged with Traefik ver 1.7, which you'll want to disable so you can use the latest version. The following command creates the cluster and exposes it on port 8081:
k3d cluster create dev -p "8081:80@loadbalancer" --k3s-arg "--no-deploy=traefik@server:*"
The kubectl command-line tool, configured to point to your cluster. (If you created your cluster using K3d and the instructions above, this will already be done for you.)
A recent version of the Helm package manager for Kubernetes.
The set of configuration files that accompany this article, which is available on GitHub:
git clone https://github.com/traefik-tech-blog/traefik-sre-tracing/

You do not need to have Traefik 2.x preinstalled, as you'll do that along the way.

Note: To keep this tutorial simple, everything is deployed on the default namespace and without any kind of protection on the Traefik dashboard. On production, you should use custom namespaces and implement access control for the dashboard.

Set up distributed tracing

First, you'll need to install and configure Jaeger on your Kubernetes cluster. The simplest way is to use the official Helm chart. As a first step, add the jaegertracing repository to your Helm repo list and update its contents:

helm repo add jaegertracing https://jaegertracing.github.io/helm-charts
helm repo update

The Jaeger repository provides two charts: jaeger and jaeger-operator. For the purpose of this tutorial, we deploy the jaeger-operator chart, which makes it easy to configure a minimal installation. To learn more about the Jaeger Operator for Kubernetes, consult the official documentation.

As it’s explained in the documentation, you’ll need to install cert-manager before installing this operator:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml

And after, we can install jaeger-operator:

helm install jaeger-op --set rbac.clusterRole=true jaegertracing/jaeger-operator

Minimal deployment

Deploying Jaeger in all its details is a topic well beyond the scope of this article. Here, we deploy Jaeger with all-in-one topology using the below configuration, which will be sufficient to demonstrate the integration:

# jaeger.yaml
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: jaeger

The above configuration creates an instance named jaeger. It also creates a query-ui, an agent, and a collector. All these related services are prefixed with jaeger. It does not deploy a database like Cassandra or Elastic; instead, it relies on in-memory data processing.

kubectl apply -f jaeger.yaml

You can confirm Jaeger is running by doing a lookup on this CRD and on deployed
services:

$ kubectl get jaegers.jaegertracing.io
NAME 	STATUS	VERSION   STRATEGY   STORAGE   AGE
jaeger   Running   1.39.0	allinone   memory	5m52s

$ kubectl get services
NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                  AGE
kubernetes                          ClusterIP   10.43.0.1       <none>        443/TCP                                  76m
jaeger-op-jaeger-operator-metrics   ClusterIP   10.43.86.167    <none>        8383/TCP,8686/TCP                        82s
jaeger-collector-headless           ClusterIP   None            <none>        9411/TCP,14250/TCP,14267/TCP,14268/TCP   47s
jaeger-collector                    ClusterIP   10.43.163.147   <none>        9411/TCP,14250/TCP,14267/TCP,14268/TCP   47s
jaeger-query                        ClusterIP   10.43.27.251    <none>        16686/TCP                                47s
jaeger-agent                        ClusterIP   None            <none>        5775/UDP,5778/TCP,6831/UDP,6832/UDP      47s

Install and configure Traefik Proxy

Now it's time to deploy Traefik Proxy, which you'll do using the official Helm chart. If you haven't already, add Traefik Labs to your Helm repository list using the below commands:

helm repo add traefik https://traefik.github.io/charts
helm repo update

Next, deploy the latest version of Traefik in the kube-system namespace. For this demo, however, the standard configuration of the Helm chart won't be enough. As part of the deployment, you need to ensure that Jaeger integration is enabled in Traefik. You do this by passing additionalArguments configuration flags in the traefik-values.yaml file:

tracing:
  jaeger:
    samplingServerURL: http://jaeger-agent.default.svc:5778/sampling
    localAgentHostPort: jaeger-agent.default.svc:6831

As shown in the above configuration, you need to provide an address for the Jaeger agent. By default, this is localhost, and if you deploy jaeger-agent as a sidecar, this works as expected. In this deployment, however, you need to provide an explicit address for jaeger-agent, which corresponds to the jaeger-agent.default.svc hostname that was configured by the Helm chart.

helm install traefik traefik/traefik -f ./traefik-values.yaml

Once the pods are created, you can verify the Jaeger integration by using port forwarding to expose the Traefik dashboard:

kubectl port-forward $(kubectl -n kube-system get pods --selector "app.kubernetes.io/name=traefik" --output=name) 9000:9000

If you access the Traefik dashboard at http://localhost:9000/dashboard/, you will see that Jaeger distributed tracing is enabled under the Features section:

Now is also a good time to expose the Jaeger UI, which is served on port 16686:

kubectl  port-forward service/jaeger-query 16686:16686

When you access the Jaeger dashboard at http://localhost:16686/, you will see traefik in the Service pull-down, and the Traefik endpoints will be listed in the Operations pull-down:

Deploy Hot R.O.D.

Now that your integration is working, you need an application to trace. For this purpose, let’s deploy Hot R.O.D. - Rides On Demand, which is an example application created by the Jaeger team. It is a demo ride-booking service that consists of three microservices: driver-service, customer-service, and route-service. Each service also has accompanying storage, such as a MySQL database or Redis cache.

The application includes four pre-built "customer personas" who can book a ride using the application UI. When a car is booked, the application will find a driver and dispatch the car.

Throughout the process, Jaeger will capture the user request as it flows through the various services (driver-service, customer-service, route-service). Individual service handling will be shown as a span, and all related spans are visualized in a graph known as the trace.

Deploy the Service along with the IngressRoute using this following configuration file:

$ kubectl apply -f hotrod.yaml
deployment.apps/hotrod created
service/hotrod created
ingressroute.traefik.containo.us/hotrod created

The hotrod route will match the hostname hotrod.localhost, which allows you to open the application UI on http://hotrod.localhost:8081/.

In the above UI, you can see the four prebuilt customer personas. This UI is not required for this distributed tracing demo, however, as you can use command-line tools.

Application traces

To see Jaeger in action, send a few user requests to the application using a sample customer persona. For example, try the following curl commands:

curl -I "http://localhost:8081/dispatch?customer=392" -H "host:hotrod.localhost"
curl -I "http://localhost:8081/dispatch?customer=123" -H "host:hotrod.localhost"

Each command triggers a sequence of requests to produce the expected result. You can see the generated traces in the Jaeger UI when you select traefik as the Service and hotrod.localhost as the Operation and click Find Traces:

Select either of the traces to explore the detailed request flow.

The display above shows the top two spans expanded to show the information forwarded by Traefik Proxy. Each span shows the request duration, along with non-mandatory sections for Tags, Process, and Logs. The Tags section contains key-value pairs that can be associated with request handling.

The Tags field of the topmost traefik span shows information related to HTTP handling, such as the status code, URL, host, and so on. The next span shows the routing information for the request, including the router and service names.

Jaeger can also deduce an overall architecture by analyzing the request traces. This diagram is available under the System Architecture > DAG tab:

The graph shows that you made two requests, which were routed to the frontend service. The frontend service then fanned out requests to the customer, driver, and route services.

Returning to the Search tab of the Jaeger UI, you can see that in the current cluster, you have traces generated for the following three entrypoints :

traefik-dashboard, which you used for lookup
ping api, used by Kubernetes for health checks
hotrod.localhost, used by the Hot R.O.D. application

As you deploy more applications to your cluster, you will see more entries in the Operations drop-down, based on the entrypoint match.

Wrap up

This post has presented a very simple demonstration of integrating Traefik Proxy with Jaeger. There is much more to explore with Jaeger, and similar integrations can be done with other distributed tracing systems, such as NewRelic or Datadog. Whichever one you choose, Traefik makes it easy to follow the progress of each request and gain insights into the application flow.

We hope you've enjoyed this series of articles on how Traefik's capabilities can enable app monitoring and health analysis for SRE. If you missed the earlier installments on log aggregation and metrics, respectively, be sure to take a look:

All three articles demonstrate how readily available open source software, including Traefik Proxy, can empower practices that both increase application uptime and contribute to improving the design of distributed systems.

If you'd like to explore new features of Traefik on monitoring and visibility, check out Traefik Proxy v3 Beta 1, with native OpenTelemetry support.

]]>

Log Aggregation in Kubernetes with Traefik Proxy

Michel Loiseleur — Wed, 18 Jan 2023 16:38:55 GMT

Originally published: February 2021
Updated: December 2022

When deployed as a Kubernetes ingress controller, Traefik Proxy can process and route many thousands of requests without a complaint! And yet, for the operations team, visibility into what's happening behind the scenes is essential. Is the application healthy? Is it working as intended? Monitoring distributed systems is one of the core principles of the set of practices known as Site Reliability Engineering (SRE).

This article is the first in a series of posts on Traefik Proxy and SRE techniques that explores how Traefik's built-in logging features can help to provide visibility. When combined with a set of open source observability projects like Open Telemetry, Elastic Stack, Grafana Stack, etc., Traefik becomes part of a rich set of tools for network log analysis and visualization.

Prerequisites

If you'd like to follow along with this tutorial on your own machine, you'll need a few things first:

A Kubernetes cluster running at localhost. One way to achieve this is to create a local cluster running in Docker containers using kind
Helm v3.9+ installed
The kubectl command-line tool is installed and configured to access your cluster.

kind requires some config in order to use an IngressController on localhost:

$ cat kind.config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30000
    hostPort: 80
    protocol: TCP
  - containerPort: 30001
    hostPort: 443
    protocol: TCP

All config files are in the public GitHub repository that accompanies this article, so you just have to clone it and create our local cluster with this network configuration:

$ git clone https://github.com/traefik-tech-blog/traefik-sre-logging/ 
$ cd traefik-sre-logging 
$ kind create cluster --config=kind.config 
$ kubectl cluster-info
$ kubectl get nodes

Set up Traefik Proxy

First, let’s start by deploying Traefik Proxy using the Helm chart and enabling logs. We can configure Helm chart deployment with a values file. We’ll start by just enabling access.logs. By default, they will be on stdout of the Traefik pod.

$ cat traefik-values-01.yaml 
ports:
  web:
    nodePort: 30000
  websecure:
    nodePort: 30001
logs:
  access:
    enabled: true
ingressRoute:
  dashboard:
    enabled: true
    # You should protect your dashboard if it’s on the internet matchRule:
    matchRule: Host(`dashboard.docker.localhost`)
    entryPoints: ["web"]      
additionalArguments:
- "--ping"

The ports with nodePort is only needed with our kind setup. Let's deploy it:

$ helm repo add traefik https://traefik.github.io/charts
$ helm install traefik -f traefik-values traefik/traefik

You should be able to access Traefik dashboard on the configured Host:

And to see logs on stdout of Traefik pods:

kubectl logs -f -l app.kubernetes.io/name=traefik

Deploy a sample application

For the purpose of this tutorial, I am using our sample app, whoami:

$ kubectl apply -f apps/

It can then be accessed with a browser or with curl:

$ curl http://whoami.docker.localhost

Configure logging for a collector

If you want to collect logs, you need to put those logs into a file, and often you'll also need to convert it from raw to JSON. For this, we use a local volume (emptyDir). You can change the format and set a path for access logs with those values:

--- traefik-values-01.yaml	2022-12-02 16:14:07.973877899 +0100
+++ traefik-values-02.yaml	2022-12-05 17:18:34.839234310 +0100
@@ -6,6 +6,14 @@
 logs:
   access:
     enabled: true
+    format: json
+    filePath: "/var/log/traefik/access.log"
+deployment:
+  additionalVolumes:
+  - name: logs
+additionalVolumeMounts:
+- name: logs
+  mountPath: /var/log/traefik
 ingressRoute:
   dashboard:
     enabled: true

To see the logs, we'll use a sidecar container to tail this file:

--- traefik-values-02.yaml	2022-12-05 17:18:34.839234310 +0100
+++ traefik-values-03.yaml	2022-12-05 17:21:47.387319123 +0100
@@ -9,6 +9,13 @@
     format: json
     filePath: "/var/log/traefik/access.log"
 deployment:
+  additionalContainers:
+  - name: tail-accesslogs
+    image: busybox
+    args: [ "/bin/sh", "-c", "tail -n+1 -F /var/log/traefik/access.log" ]
+    volumeMounts:
+    - name: logs
+      mountPath: /var/log/traefik
   additionalVolumes:
   - name: logs
 additionalVolumeMounts:

Instead of tail, you can, of course, use a log collector. After applying it, we can see logs in JSON format on the tail-accesslogs sidecar:

$ helm upgrade -i -f traefik-values-03.yaml traefik traefik/traefik
$ kubectl logs -f -l app.kubernetes.io/name=traefik -c tail-accesslogs

You should see all current fields of ping requests:

{
  "ClientAddr": "10.244.0.1:41388",
  "ClientHost": "10.244.0.1",
  "ClientPort": "41388",
  "ClientUsername": "-",
  "DownstreamContentSize": 2,
  "DownstreamStatus": 200,
  "Duration": 34633,
  "OriginContentSize": 2,
  "OriginDuration": 7325,
  "OriginStatus": 200,
  "Overhead": 27308,
  "RequestAddr": "10.244.0.13:9000",
  "RequestContentSize": 0,
  "RequestCount": 64,
  "RequestHost": "10.244.0.13",
  "RequestMethod": "GET",
  "RequestPath": "/ping",
  "RequestPort": "9000",
  "RequestProtocol": "HTTP/1.1",
  "RequestScheme": "http",
  "RetryAttempts": 0,
  "RouterName": "ping@internal",
  "StartLocal": "2022-12-05T16:25:29.38659907Z",
  "StartUTC": "2022-12-05T16:25:29.38659907Z",
  "entryPointName": "traefik",
  "level": "info",
  "msg": "",
  "time": "2022-12-05T16:25:29Z"
}

Add headers to access.logs

As you can see and, aligned with the documentation, there is no header by default. Let's try to add an HTTP header to those access.logs:

--- traefik-values-03.yaml	2022-12-05 17:21:47.387319123 +0100
+++ traefik-values-04.yaml	2022-12-05 17:42:44.140244962 +0100
@@ -8,6 +8,9 @@
     enabled: true
     format: json
     filePath: "/var/log/traefik/access.log"
+    fields:
+      headers:
+        defaultmode: keep
 deployment:
   additionalContainers:
   - name: tail-accesslogs

Apply the change:

$ helm upgrade -i -f traefik-values-04.yaml traefik traefik/traefik
$ kubectl logs -f -l app.kubernetes.io/name=traefik -c tail-accesslogs

And see headerfields on the sidecar container, prefixed with request:

{
  [...],
  "request_Accept": "*/*",
  "request_Connection": "close",
  "request_User-Agent": "kube-probe/1.25",
  "request_X-Forwarded-Host": "10.244.0.16:9000",
  "request_X-Forwarded-Port": "9000",
  "request_X-Forwarded-Proto": "http",
  "request_X-Forwarded-Server": "traefik-b6b999947-wb5lw",
  "request_X-Real-Ip": "10.244.0.1"
}

Protect sensitive header

Now let's say that we want to:

Check that Authorization header with a token is here, but without writing this sensitive value.
Drop Accept and Connection fields

--- traefik-values-04.yaml	2022-12-05 17:42:44.140244962 +0100
+++ traefik-values-05.yaml	2022-12-05 17:56:34.397503223 +0100
@@ -11,6 +11,10 @@
     fields:
       headers:
         defaultmode: keep
+        names:
+          Accept: drop
+          Connection: drop
+          Authorization: redact
 deployment:
   additionalContainers:
   - name: tail-accesslogs

After applying it:

$ helm upgrade -i -f traefik-values-05.yaml traefik traefik/traefik
$ kubectl logs -f -l app.kubernetes.io/name=traefik -c tail-accesslogs

And after sending a request with the Authorization header:

curl -X GET "http://whoami.docker.localhost" -H "Authorization: Bearer Thiswontbeintheaccesslogs"

We can see the new format, with sensitive the Authorization header redacted:

{
  [...],
  "origin_Accept-Ranges": "bytes",
  "origin_Content-Length": "3124",
  "origin_Content-Security-Policy": "frame-src self https://traefik.io https://*.traefik.io;",
  "origin_Content-Type": "text/html; charset=utf-8",
  "request_Authorization": "REDACTED",
  "request_User-Agent": "curl/7.74.0",
  "request_X-Forwarded-Host": "dashboard.docker.localhost",
  "request_X-Forwarded-Port": "80",
  "request_X-Forwarded-Proto": "http",
  "request_X-Forwarded-Server": "traefik-74d455649f-ndhcj",
  "request_X-Real-Ip": "10.244.0.1"
}

And there are also new origin headers sent by Curl with our test.

Timezone and buffering

Now, let's try to change the timezone and use the buffering feature to increase performance.

--- traefik-values-05.yaml	2022-12-05 17:56:34.397503223 +0100
+++ traefik-values-06.yaml	2022-12-06 09:27:12.536753905 +0100
@@ -8,6 +8,7 @@
     enabled: true
     format: json
     filePath: "/var/log/traefik/access.log"
+    bufferingSize: 1000
     fields:
       headers:
         defaultmode: keep
@@ -15,6 +16,9 @@
           Accept: drop
           Connection: drop
           Authorization: redact
+env:
+- name: TZ
+  value: "Brazil/East"
 deployment:
   additionalContainers:
   - name: tail-accesslogs

Apply it like the other changes:

$ helm upgrade -i -f traefik-values-06.yaml traefik traefik/traefik
$ kubectl logs -f -l app.kubernetes.io/name=traefik -c tail-accesslogs

And see the difference in timezones:

{
  [...],
  "StartLocal": "2022-12-06T05:29:15.353148769-03:00",
  "StartUTC": "2022-12-06T08:29:15.353148769Z",
  "time": "2022-12-06T05:29:15-03:00"
}

StartLocal and time are using the Brazil/East timezone, which is UTC-3.
For the buffering, we can see on a simple load test that Traefik Proxy can handle +16% more traffic:

$ # Before bufferingSize
$ ab -n 10000 -C 100 http://whoami.docker.localhost/
[...]
Requests per second:    1448.70 [#/sec] (mean)
$ # After applying bufferingSize to 1000
Requests per second:    1687.53 [#/sec] (mean)

Summary

This simple tutorial serves to demonstrate Traefik's comprehensive logging capabilities. You can really tailor Traefik Proxy logs before collecting them. It's a powerful tool for understanding the health and performance of services running on Kubernetes clusters.

In the next two articles of this series, I will cover the other two legs of observability: metrics and tracing, used with OpenTelemetry, Prometheus, and many others.

As usual, if you love Traefik Proxy and there are features you'd like to see in future releases, open a feature request or get in touch with other community members on our Forum. And if you'd like to dig deeper into how your Traefik Proxy instances operate, check out Traefik Hub, our Cloud Native networking platform.

]]>

Secure Web Applications with Traefik Proxy, cert-manager, and Let’s Encrypt

Richard Hillmann — Tue, 17 Jan 2023 13:01:42 GMT

Managing TLS certificates has never been easier. Not that long ago, running secure websites was a tedious job for engineers as they had to deal with complex business processes and chores. Who does not remember the times when you had to make a purchase requisition, get in touch with your vendor with your Certificate Signing Request (CSR), watch out for an email to validate your domain, and eventually announce a maintenance window that you will have fun with deploying the certificate in production?

Phew — I’m certainly glad those days are gone!

Shaken by the revolutionary non-profit Certificate Authority, Let’s Encrypt, and its ACME protocol, the market gradually moved into fully-automated solutions that enabled developers to deliver secure websites at no costs with the least effort.

Since day one, Traefik Proxy provides a native Let’s Encrypt integration to automate the full lifecycle of certificates. Without the need to handle any third-party tooling, Traefik Proxy is the natural choice for automated certificate management.

While using a single instance of Traefik Proxy with Let's Encrypt works like a charm, however, running multiple instances can raise some issues. If your production environment requires you to use Let's Encrypt with high availability (HA) in Kubernetes, you always have the option of Traefik Enterprise, which includes distributed Let's Encrypt as a supported feature.

But if you want to stick with Traefik Proxy, you have nothing to fear!

With Kubernetes we got a powerful and extensible platform to solve a lot of complex scenarios. cert-manager is a powerful solution that helps us automate and manage almost everything around TLS certificates. It provides a set of Custom Resource Definitions (CRD) for various scenarios and integrates well with native Ingress or Gateway resources.

cert-manager stores and caches certificates and private keys in Kubernetes secrets, making them highly available for further consumption by ingress controllers (like Traefik Proxy) or applications.

Note: By default, cert-manager does not clean up secrets automatically, allowing it to re-attach to already issued certificates and avoid issuing new certificates. This becomes very handy in scenarios when you need to create and delete lots of resources and would not like to be rate limited.

cert-manager can interact with a variety of sources to issue certificates including Let’s Encrypt, HashiCorp Vault as well as private PKI. For unsupported cases like AWS Private Certificate Authority, Google Cloud Certificate Authority Service or Cloudflare Origin CA the External Issuer allows you to extend cert-manager capabilities..

But enough talk! Time to get down to business and dig into how you can use cert-manager to extend Traefik Proxy’s capabilities.

Prerequisites

To follow this tutorial, you’ll need the following:

A Kubernetes cluster >= v1.20
A public hosted DNS domain for Let’s Encrypt — for the purpose of this article I will use Cloudflare
A Kubernetes native ingress controller: Traefik Proxy 2.9, you can install the helm chart with this command:
```
helm install traefik traefik/traefik
```

cert-manager 1.10 which you can install with this command:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml

A service providing a web port. In this tutorial, I’ll be using whoami as an example:

apiVersion: v1
kind: Namespace
metadata:
 name: whoami
---
apiVersion: v1
kind: Service
metadata:
 name: whoami
 namespace: whoami
spec:
 ports:
   - name: web
     port: 80
     targetPort: web
 selector:
   app: whoami
---
apiVersion: apps/v1
kind: Deployment
metadata:
 name: whoami
 namespace: whoami
spec:
 selector:
   matchLabels:
     app: whoami
 template:
   metadata:
     labels:
       app: whoami
   spec:
     containers:
       - name: whoami
         image: traefik/whoami
         ports:
           - name: web
             containerPort: 80

Traefik Proxy with cert-manager and Let’s Encrypt

Let’s explore how we can secure a web application in combination with a Kubernetes ingress controller like Traefik Proxy and cert-manager. Let’s Encrypt provides multiple challenge types to validate control of a domain name. Depending on your requirements you may choose HTTP-01 when your service is public reachable or DNS-01 for private endpoints.

Please be aware of rate limits when using lets encrypt. To avoid unpleasant surprises it is recommended to use the Let’s Encrypt staging environment:

staging: https://acme-staging-v02.api.letsencrypt.org/directory`
production: https://acme-v02.api.letsencrypt.org/directory

HTTP challenge

For most common scenarios the HTTP-01 challenge is a convenient start to solve an ACME based validation. To make this scenario work, Traefik Proxy needs to be reachable from the internet on HTTP port 80, and the used DNS domain has to be configured to point to it.

When a new certificate needs to be issued (or renewed), cert-manager will create a temporary Ingress resource to route requests made by the ACME server to the specific matched host and ./well-known/acme-challenge/xxx path, so it can answer with the desired response.

Implementing the challenge

First you need to define a new cert-manager Issuer to represent a certificate issuing authority. This example uses the ACME-based Certificate Authority in conjunction with Let’s Encrypt.

Note: You need to change the server to production to retrieve a certificate that will be accepted by your browser.

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
 name: le-example-http
 namespace: whoami
spec:
 acme:
   email: user@example.com
   # We use the staging server here for testing to avoid hitting
   server: https://acme-staging-v02.api.letsencrypt.org/directory
   privateKeySecretRef:
     # if not existing, it will register a new account and stores it
     name: example-issuer-account-key
   solvers:
     - http01:
         # The ingressClass used to create the necessary ingress routes
         ingress:
           class: traefik

Next, you’ll need a Kubernetes Ingress resource to define the domain for TLS we want to attach.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: whoami
 namespace: whoami
 annotations:
   cert-manager.io/issuer: "le-example-http"
spec:
 tls:
   - hosts:
       - whoami.example.com
     secretName: tls-whoami-ingress-http
 rules:
   - host: whoami.example.com
     http:
       paths:
         - path: /
           pathType: Prefix
           backend:
             service:
               name: whoami
               port:
                 name: web

cert-manager automatically creates a new Certificate resource for the specified domain with the given secretName, provisions a CertificateRequest to request a signed certificate from one of the configured issuers, and stores the certificate and private key with the same name as the secret. The annotation cert-manager.io/issuer requires the name of the previously created Issuer and enables the resource to be managed by cert-manager.

Once the secret has been created, Traefik Proxy will fetch the certificate and private key and will serve it when the requested domain is called. Alternatively, you can also deploy a ClusterIssuer resource, which is accessible across all namespaces and referenced by the annotation cert-manager.io/cluster-issuer.

Note: cert-manager will not clean up certificates on its own, so they can be easily re-attached even if someone makes changes to the given Ingress object. If there is already an existing and valid certificate in place, it will be re-used.

DNS challenge

In some cases, you are not able to use the HTTP challenge (usually when your service is only internally available) and have to fall back to a DNS challenge. All you need to have in place is a registered DNS domain that can be resolved from the internet.

Unfortunately, cert-manager only supports a small range of DNS providers natively or dynamic DNS via RFC2136. Luckily there is the option to extend this with custom webhook solvers, so make sure to check out existing projects before implementing your own.

Implementing the challenge

The process looks almost the same as with the HTTP challenge. Instead of specifying the HTTP challenge, you need to set up the Issuer for using the DNS challenge. cert-manager will take care of creating the necessary validation records in the respected DNS zone.

apiVersion: v1
kind: Secret
metadata:
 name: cloudflare-api-token-secret
type: Opaque
stringData:
 api-token: <API Token>
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
 name: le-example-dns
 namespace: whoami
spec:
 acme:
   email: user@example.com
   # We use the staging server here for testing to avoid hitting
   server: https://acme-staging-v02.api.letsencrypt.org/directory
   privateKeySecretRef:
     # if not existing, it will register a new account and stores it
     name: example-issuer-account-key
   solvers:
     - dns01:
         cloudflare:
           apiTokenSecretRef:
             name: cloudflare-api-token-secret
             key: api-token

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: whoami
 namespace: whoami
 annotations:
   cert-manager.io/issuer: "le-example-dns"
spec:
 tls:
   - hosts:
       - whoami.example.com
     secretName: tls-whoami-ingress-dns
 rules:
   - host: whoami.example.com
     http:
       paths:
         - path: /
           pathType: Prefix
           backend:
             service:
               name: whoami
               port:
                 name: web

Troubleshooting

All cert--manager resources provide handy status and event information. It helps you understand problems and verify everything is working as expected.

$ kubectl -n whoami get issuer -o wide
NAME              READY   STATUS
le-example-http   True    The ACME account was registered with the ACME server

$ kubectl -n whoami get certificateRequest -o wide
NAME                            APPROVED   DENIED   READY   ISSUER            STATUS
tls-whoami-ingress-http-fdw2x   True                True    le-example-http   Certificate fetched from issuer successfully

$ kubectl -n whoami get certificates
NAME                      READY   SECRET                    ISSUER            STATUS
tls-whoami-ingress-http   True    tls-whoami-ingress-http   le-example-http   Certificate is up to date and has not expired

$ kubectl -n whoami describe secret tls-whoami-ingress-http
Annotations:  cert-manager.io/alt-names: whoami.example.com
             cert-manager.io/certificate-name: tls-whoami-ingress-http
             cert-manager.io/common-name: whoami.example.com
             cert-manager.io/issuer-name: le-example-http

Type:  kubernetes.io/tls

Data
====
tls.crt:  2449 bytes
tls.key:  1679 bytes

Summary

This blog post just scratched the surface on the possibilities of cert-manager in conjunction with Let’s Encrypt. It already helps users to automate enrolling our application with publicly valid certificates for HTTPS while keeping it up to date.

Today, cert-manager is the almost perfect solution in Kubernetes for dealing with any kind of work with certificates. It is even possible to create your own simple private PKI without the need to deal with any CLI tools for automation.

I’d also recommend you explore more advanced features and use cases, like securing your pod-to-pod communication by leveraging the CSI driver for mTLS or the CSI SPIFFE driver.

Did you know that Traefik Proxy 3.0 Beta 1 added native support for SPIFFE? Check out the latest beta version of Traefik Proxy, play around with the new features and capabilities, and don’t forget to share your feedback!

]]>

Mocktail: The Mock Generator for Strongly-Typed Mocks You’ve Been Looking For

Ludovic Fernandez — Tue, 10 Jan 2023 10:22:30 GMT

In software development, mocks are important testing pieces used to produce isolation for a targeted code. When you need to test a piece of code, in some cases, you want to isolate that tested piece — for example, you want to test a service that uses a database, but you don't want to call the real database.

There are several approaches to answering this problem, but in this article, I’ll only focus on the use of mocks. Mocks are simulated objects that mimic a behavior. The use of mocks is common and useful, but mocks have certain usage and maintainability constraints. Mocks can be simple or complex depending on the size of the tested piece and the complexity of the expected behavior.

Creating mocks manually or with a framework

Mocks can be used to serve several needs that depend on how the piece that you are trying to replace is used inside the tested code. Maybe the object is just needed to compile, maybe the object needs to return always the same thing, or maybe the object needs to be "smart" enough to be able to return the right element based on some parameters.

The first approach when you need a mock is to create it manually.

package foo

import "fmt"

type API interface {
   Get(userID string) *User
   Save(user *User) error
}

type User struct {
   ID   	string
   username string
   Domain   string
}

type Service struct {
   api API
}

func (s Service) GetDomain(userID string) (string, error) {
   user := s.api.Get(userID)
   if user == nil {
  	return "", fmt.Errorf("user %q not found", userID)
   }

   if user.Domain != "" {
  	return user.Domain, nil
   }

   user.Domain = user.username + ".example.com"

   err := s.api.Save(user)
   if err != nil {
  	return "", err
   }

   return user.Domain, nil
}

package foo

import (
   "testing"

   "github.com/stretchr/testify/assert"
   "github.com/stretchr/testify/require"
)

type apiMock struct {
   getResponse  *User
   getCallCount int

   saveResponse  error
   saveCallCount int
}

func (m *apiMock) Get(_ string) *User {
   m.getCallCount++

   return m.getResponse
}

func (m *apiMock) Save(_ *User) error {
   m.saveCallCount++
   return m.saveResponse
}

func TestSimple(t *testing.T) {
   user := &User{
  	ID:   	"8411fd4127",
  	username: "foo",
  	Domain:   "",
   }

   mck := &apiMock{
  	getResponse:  user,
  	saveResponse: nil,
   }

   service := Service{api: mck}

   data, err := service.GetDomain("8411fd4127")
   require.NoError(t, err)

   assert.Equal(t, "foo.example.com", data)
   assert.Equal(t, 1, mck.getCallCount)
   assert.Equal(t, 1, mck.saveCallCount)
}

The handwritten mocks can be enough in a lot of use cases, but sometimes you can have large objects or complex expected behavior. In those cases, the approach will be to use a framework that helps with that level of complexity.

At Traefik Labs, we use Testify for test assertions. Testify is the most popular assertions library in Go, it helps to improve the test readability and reduce the size of the test plumbing, and it also contains a mock system.

package foo

import (
   "testing"

   "github.com/stretchr/testify/assert"
   "github.com/stretchr/testify/mock"
   "github.com/stretchr/testify/require"
)

type apiMock struct{ mock.Mock }

func newAPIMock(tb testing.TB) *apiMock {
   tb.Helper()

   m := &apiMock{}
   m.Mock.Test(tb)

   tb.Cleanup(func() { m.AssertExpectations(tb) })

   return m
}

func (s *apiMock) Get(userID string) *User {
   ret := s.Called(userID)
   return ret.Get(0).(*User)
}

func (s *apiMock) Save(user *User) error {
   ret := s.Called(user)
   return ret.Error(0)
}

func TestTestify(t *testing.T) {
   mck := newAPIMock(t)

   service := Service{api: mck}

   user := &User{
  	ID:   	"8411fd4127",
  	username: "foo",
  	Domain:   "",
   }
   mck.On("Get", "8411fd4127").Return(user).Once()

   mck.On("Save", user).Return(nil).Once()

   data, err := service.GetDomain("8411fd4127")
   require.NoError(t, err)

   assert.Equal(t, "foo.example.com", data)
}

The mock system of Testify is great, but you still have to write a lot of things by hand. It is at this moment that you realize you need a generator.

The problem with generic generators

When our team started working on our latest product, Traefik Hub, we were faced with the need to use mocks for a number of tests. At first, we started with simple handwritten mocks. However, we quickly came across a few issues. The first problem that we faced with handwritten mocks was the fact that writing that kind of code is extremely repetitive and boring to maintain. After some discussion, the team decided to use a tool to generate them. There are some existing mock generators in Go, and they are great, but none provided exactly what we wanted. We expected to have fluent syntax and strongly typed mocks, and the generated mocks from these generators were weak against changes.

The generated mocks used a string to call the method, and the arguments were just variadic of interfaces. When you are using strongly-typed languages like Go, you always prefer to stay in the world of strongly-typed things!

The main problem with weakly-typed elements is maintainability: when you need to change the signature of a method (adding a new parameter, changing a type, etc.), the changes will not be propagated to the mocks, and the compiler will not be able to see if something is broken.

At this point, the team thought that it was easier to come back to our precious handwritten mocks. However, I had a different idea. Why not create our own mock generator?

And that’s how Mocktail was born!

Have a Mocktail, I say!

Mocktail generates strongly typed mocks and provides a simple, fluent syntax. The methods of the mocks have the same signature as the real method signature. The number of parameters and the types of those parameters are the same as the real methods.

Using Mocktail is extremely simple. Create a file called mock_test.go and add directives for the mocks you need (for example, // mocktail:MyObject).

package foo

// mocktail:Foo
// mocktail:Bar
// ...

Then run Mocktail at the root of your project.

$ mocktail

You are now able to use your mocks easily!

package foo

import (
   "testing"

   "github.com/stretchr/testify/assert"
   "github.com/stretchr/testify/mock"
   "github.com/stretchr/testify/require"
)

func TestMocktail(t *testing.T) {
   mck := newAPIMock(t)

   service := Service{api: mck}

   user := &User{
  	ID:   	"8411fd4127",
  	username: "foo",
  	Domain:   "",
   }
   mck.OnGet("8411fd4127").TypedReturns(user).Once()

   mck.OnSave(user).TypedReturns(nil).Once()

   data, err := service.GetDomain("8411fd4127")
   require.NoError(t, err)

   assert.Equal(t, "foo.example.com", data)
}

Summing up

The good thing about mocktails is that you can go wild with them without any severe side effects — except maybe for a bit of a sugar rush!

We created Mocktail to serve the needs of our team and tackle the specific issues that we were facing in our process. But my sincere hope is that this nifty little tool will be of use to many of you and save you hours of frustration manually creating mocks.

If you’re eager to try out Mocktail, you’ll find it on GitHub.

Thanks for reading!

]]>

Run APIs Easily. Anywhere. | Traefik Labs

One Platform. One Gateway. Every Request Governed.

The Architecture at a Glance

Two Estates, One Management Plane

The Gateway Layer: Traefik Hub Across Both Estates

API Gateway

AI Gateway

MCP Gateway

The VMware Exit: Traefik as the Application Delivery Constant

Migrate (DAY 0)

Modernize

Transform

Why Regulated Industries Should Pay Attention

The Upgrade Path is the Commercial Insight

SUSECON 2026 and What Comes Next

Summary: The Three Waves are Converging

Further Reading

About SUSE

About Traefik Labs

One Gateway. Three Estates. Zero Compromises.

The Real Challenges Behind the Buzzwords

Unified Application Intelligence: The Nutanix and Traefik Labs Joint Solution

Under the Hood: How the Integration Works

Auto-Discovery Across AHV Virtual Machines

Native Kubernetes Integration on NKP

Complementing Flow Networking at Layer 7

AI Workload Routing and Governance

What This Looks Like: A Real Application Flow

The Time to Act Is Now

Breaking the Monolith: How an API Gateway Turns a Risky Migration into a Controlled Journey

The Gateway as the Starting Point

Observability: See Before You Cut

Centralizing Auth: Your First Win

Bridging the Infrastructure Gap: No Re-Platforming Required

The Strangler Fig: Carving Out Your First Service

Testing in Production Safely

Mirroring: Zero-Risk Shadow Testing

A/B Testing: Comparing Old and New

Failover: The Monolith as Your Safety Net

Circuit Breaker: Automatic Damage Control

Canary Deployments: Shipping New Features Safely

Identity-Aware Routing: Migrating by Who, Not Just What

The Triple Gate: API, AI, and MCP

The Migration Is a Journey

Implementing Forrester's AEGIS Framework at the Infrastructure Layer

Key Takeaways

Why Infrastructure-Layer Enforcement Matters for AEGIS

AEGIS Core Principles: Least Agency, Continuous Assurance, and Securing Intent

Principle 1: Least Agency

Principle 2: Continuous Risk Management

Principle 3: Securing Intent

Mapping the Six AEGIS Domains

Domain 1: Governance, Risk, and Compliance

Domain 2: Identity and Access Management

Domain 3: Data Security and Privacy

Domain 4: Application Security and DevSecOps

Domain 5: Threat Management and Security Operations

Domain 6: Zero Trust Architecture

Putting It Together: One Request, Five Domains

Coverage Summary

What's Next

Frequently Asked Questions

What is Forrester's AEGIS framework?

What is "least agency" in the AEGIS framework?

How do you implement AEGIS controls at the infrastructure layer?

How many AEGIS domains can be enforced at the infrastructure layer?

What is the difference between AEGIS and the OWASP Top 10 for Agentic Applications?

The Ingress NGINX Retirement Created Three Risks. Most Teams Are Only Managing One.

Migration Risk

Security Risk

Stagnation Risk

A Further Dimension for European Organizations

All Three Risks Are Addressable. Now Is the Time.

OWASP Agentic Top 10: Where Should These Controls Be Enforced?

Key Takeaways

The Enforcement Boundary Problem

ASI-01: Agent Goal Hijack

ASI-02: Tool Misuse & Exploitation

ASI-03: Identity & Privilege Abuse

ASI-04: Supply Chain Vulnerabilities