Unified Gateway for VMs, Containers, and AI | Traefik Labs

Ten Years In, Traefik is Having Its Biggest Year Ever

Emile Vauge — Thu, 09 Jul 2026 14:30:58 GMT

This week, Traefik crossed two milestones at once: 1,000 contributors on GitHub and 3.5 billion pulls of the official Docker image. And it happened as the project enters its second decade.

I would love to tell you this is a story about longevity. It isn't. 2026 is on pace to be the most active year in Traefik's history, with more pull requests merged than in any year before. Ten years in, the project is not aging gracefully. It is accelerating. And the reasons why say a lot about where cloud native infrastructure is heading.

From Cloud Native to Claude Native

I wrote Traefik's first lines of code in 2015, the year the cloud native movement got its name. Docker was two years old. Kubernetes had just hit 1.0. The CNCF was founded that summer. Everyone was suddenly running dozens of containers that appeared and disappeared in seconds, and the reverse proxies of the day, built for a world of static config files and graceful reloads, simply could not keep up.

Traefik was built on one idea: infrastructure should configure itself. Point Traefik at your orchestrator, and it discovers your services, builds its routing table, and updates it in real time as containers come and go. No templates, no config generation, no reload. In 2015 that was a radical idea. Today it is table stakes, and I am proud that Traefik helped make it so.

That first year, fourteen people committed code to the repository.

The graveyard of infrastructure software is full of projects that solved one era's problem perfectly and then stopped. What has kept Traefik alive is that the founding idea, dynamic, self-configuring traffic management, turned out to apply to every wave that followed.

When the industry consolidated on Kubernetes, Traefik became one of the most widely deployed ingress controllers in the world, and the platforms voted with their defaults. Rancher (SUSE) made Traefik the out-of-the-box ingress of K3s from the beginning, Nutanix standardized its Kubernetes Platform on it years ago, and Canonical MicroK8s now ships it as the default ingress controller on every cluster. The wave is still growing: RKE2 adopts Traefik as its default starting with v1.36 currently shipping, and since the retirement of Ingress NGINX, IBM Cloud, OVHcloud, TIBCO and others have each independently selected Traefik as their strategic ingress and Gateway API solution. When platform vendors who compete with each other all land on the same proxy, that is not a trend; it is a standard.

When organizations realized their ingress point was actually their API entry point, we built Traefik Hub: full API management on top of the proxy, with governance, versioning, access control, and a developer portal, running on the same runtime that already handled the traffic.

When LLMs reached production infrastructure, with their fragmented SDKs, scattered credentials, and zero observability, we applied the same playbook to AI traffic. The Traefik AI Gateway gives teams one secure, observable entry point to every model provider. And as agentic AI took off, we extended it with an MCP Gateway, bringing the same governance to the traffic between agents and tools.

Today, this converges into a single product vision: a unified gateway for VMs, containers, and AI. One runtime, one control plane, wherever your workloads live and whatever they talk to. And moving across that spectrum is seamless by design: start with open source Traefik Proxy, turn on Hub's API management, add the AI Gateway, and each step is a configuration change on the runtime you already operate, never a replatform. Your routing rules, middlewares, and operational knowledge carry over intact. Container router, Kubernetes ingress, API management, AI and MCP gateway: four eras, one architecture, one community.

3.5 Billion Pulls, in Perspective

Ten years of crossing eras compounds into a number: 3.5 billion pulls of the official Docker image. To put it in context, I went through the pull counts of Docker's official images, the curated library that includes the most foundational software in our industry. The handful of images that sit above Traefik are the bedrock of modern computing: operating system bases like Alpine and Ubuntu, language runtimes like Python and Node, and the major open source databases.

Traefik is one of the fifteen most downloaded official images on Docker Hub, standing shoulder to shoulder with the operating systems, languages, and databases the entire industry runs on.

Every one of those pulls is a decision to put Traefik on the path of production traffic. That is not a vanity metric. That is trust, compounded daily for ten years.

The Most Active Year in Traefik's History

Adoption tells you about a project's past. Development activity tells you about its future.

In 2025, the community merged nearly 500 pull requests. In the first six months of 2026 alone, it merged 470, a pace of roughly 940 for the year: almost double last year, and the highest sustained activity in the project's history.

A ten-year-old project setting all-time activity records is not normal. It is happening because of two deliberate bets.

The first bet: making the Ingress NGINX migration a non-event. When the Kubernetes SIG Network announced the retirement of Ingress NGINX, millions of clusters were suddenly on a countdown to running unmaintained software. Rather than telling users to rewrite everything, we built a native NGINX Ingress provider into Traefik: it reads existing Ingress resources and their nginx.ingress.kubernetes.io annotations directly, more than 80 of them, with no conversion and no YAML rewriting. That engineering effort, and the wave of adopters it brought, is one of the engines behind the record numbers above. New users file issues, contribute fixes, and push the project forward.

The second bet: going all-in on Gateway API. We didn't just implement the spec; we invested in shaping it, contributing to the specification work and its extension into AI and LLM routing. Gateway API is the future of Kubernetes networking, and Traefik is one of the few projects where you can run legacy Ingress, NGINX annotations, and Gateway API side by side, on the same instance, and migrate at your own pace. Being at the frontier of the spec means being where the ecosystem's energy is, and that energy shows up directly in our repository.

Activity follows relevance. The numbers are at an all-time high because, ten years in, Traefik is solving the two most pressing traffic problems in Kubernetes today.

1,000 Contributors, and Two New Maintainers

The milestone that means the most to me is not the pulls.

Fourteen people wrote code for Traefik in 2015. This month, the one thousandth contributor landed a change in traefik/traefik. Docker pulls measure adoption; contributors measure ownership. A thousand people own a piece of this project.

To every one of the 1,000, thank you. You built one of the most downloaded pieces of infrastructure software on the planet.

This week, I had the honor of welcoming two new maintainers: Kangmin Kim (@amazon7737), from Busan, South Korea, and Nándor Kollár (@nandorKollar), from Munich, Germany. Both earned it through high-quality contributions.

1,000 contributors, 3.5 billion pulls, and, a record nobody asked me to announce, our highest year ever for CVE reports. You do not get the first two without the third: security researchers only audit software the world actually runs. So, thank you Claude, I suppose. Jokes aside, vulnerabilities being found, and more importantly quickly patched, are the sign of a healthy project.

The Next Decade Starts Now

There has never been a better moment to jump in. The project is moving faster than at any point in its ten years, and the best time to join a community is when it is accelerating. We have already surpassed the 1,000 contributor mark since I began writing this blog post, so if you've ever thought about contributing to open source, contributor number 1,009 is a good first issue away.

Ten years ago, Traefik was a bet that infrastructure should configure itself. A thousand contributors and 3.5 billion pulls later, the bet keeps paying off, and the pace keeps rising.

I said this is not a story about longevity. It is a story about momentum. And the most active year in Traefik's history is the one happening right now.

Resources

]]>

What Happens If The Connection Goes Down? Lessons Learned from Anthropic's Fable 5 and Mythos 5 Going Dark

Immánuel Fodor — Mon, 29 Jun 2026 23:54:24 GMT

A government switched off a frontier model overnight. Here is why that reframes the sovereignty argument, and why the open-weights ecosystem finally makes the alternative real.

On Friday, June 12, at 5:21 pm Eastern, Anthropic received a letter. By the time most of the US East Coast was thinking about dinner, two of the most capable AI models in commercial use, Fable 5 and Mythos 5, were gone. Disabled for every customer, worldwide, within the span of an afternoon.

The trigger was an export control directive from the US government, citing national security authorities, ordering Anthropic to suspend access for any foreign national, whether inside or outside the United States, including the company's own foreign-national employees. Unable to cleanly segment access by nationality, Anthropic disabled the models for everyone. The company said it disagreed with the order and believed it was a misunderstanding. The models stayed dark anyway.

The shape of this matters more than the headline.

An outage would have ended. A price change or a deprecation notice would have come with a warning and a migration window, the kind of disruption every vendor relationship is built to absorb. This was different in kind. Someone outside your company, and outside your vendor's full control, decided that a model your product depended on would stop answering. Available to hundreds of millions of people that morning. Gone by that evening.

One detail makes this concrete for me in a way it might not be for a reader in San Francisco. I am a foreign national, working from Budapest, and the directive, read literally, was about people like me. The access I had at the start of my workday, under the strict terms of the order, was gone by the close of business. The reason came down to the passport I hold.

We Wrote the Question in April. In June We Got Our Answer.

Two months ago, we published The Five Levels of AI Sovereignty, a maturity model for figuring out how much control an organization actually has over its AI infrastructure. One of the first questions we tell people to ask is blunt: what happens if your internet connection goes down?

We meant it literally. Outages, air-gaps, and the network cable. We were thinking about availability zones and egress firewalls.

June 12 added a way to lose a model we hadn't explored. The connection was fine. The data center was fine. The vendor was willing. The model still went away, because the legal right to use it was revoked by someone with no stake in your uptime. Our framework has a dimension that catches this after the fact, the one we labeled vendor lock-in and escape velocity. We framed it around contracts and pricing renewals. A Friday-afternoon letter from the Commerce Department belongs in the same column, and it never occurred to us to put it there.

The lesson holds, sharpened. If the most capable thing in your stack is a model you do not control, your continuity is a function of decisions made in rooms you will never sit in.

The Reason This Argument Was Easy to Ignore Until Now

For most of the last two years, the honest counterargument to all of this was capability. You could run a model locally, and it would be meaningfully worse than the hosted frontier. For many teams, the math was uneven enough that they accepted the dependency and moved on. The hosted model was simply better, and the gap was wide.

That calculation changed this spring, and it changed fast.

The strongest open-weight models in the world right now are Chinese. Moonshot's Kimi K2.6 leads the open field on the Artificial Analysis Intelligence Index at around 54, ahead of every open competitor and inside the top tier overall. DeepSeek shipped V4 Pro and V4 Flash in April. Alibaba's Qwen 3.5 and 3.6 lines landed under permissive Apache licensing. Z.ai shipped GLM-5.2 on June 13, the day after the Fable 5 order, posting the highest open-weight SWE-bench Pro coding score to date and, of note, an Anthropic-compatible API.

Then, on June 4, Nvidia released Nemotron 3 Ultra, announced days earlier at Computex. It is a 550-billion-parameter mixture-of-experts model with roughly 55 billion parameters active per token, the most capable open-weight model a US lab has shipped, scoring about 48 on the same index. Nvidia published the weights, the post-trained checkpoints, the datasets, and the training recipes. It serves over 300 tokens per second on early endpoints, several times faster than comparably sized peers, and it was built deliberately for agentic work: tool calls, structured output, long-horizon planning that holds up over many turns.

The gaps are real. Nemotron 3 Ultra is the best US open model, and it still trails Kimi K2.6 by several points on the index. A 550B model wants a real GPU; this is data-center hardware, not a spare workstation. Self-hosting swaps an API bill for a hardware and operations bill, and that cost is worth modeling carefully before anyone commits.

Here is what survives all those caveats. Each of these models can run with zero external dependency. You pull the weights once. After that, there is no API to call, no license server to phone home to, no provider in the request path, and nobody in a position to send a letter that turns the model off. The capability gap is now small enough that, for a large class of workloads, closing the dependency is worth the few benchmark points you give up.

The Part Where the Gateway Earns Its Place

A swap like this only survives contact with a real codebase if the application does not know which model answered.

If your services call a model provider's SDK directly, switching from a hosted model to a local one means editing and redeploying every service that talks to a model. Routed through a gateway, the model becomes a setting. The address your applications send their prompts to can point at a hosted frontier model today and a self-hosted Nemotron or Kimi deployment tomorrow, and nothing downstream has to change. The application keeps speaking the same protocol. The gateway decides where the request actually lands.

The safety layer matters here just as much, and it is the dimension that quietly drops teams a level. If your jailbreak detection, PII redaction, and content filtering are API calls to a hosted moderation model, then your "sovereign" deployment has a cloud dependency baked into every single request. Move those checks to local models running behind the same gateway, and the dependency disappears. That is the AI Safety Architecture dimension we flagged as the one teams most often miss. After June 12, it is the one I would check first.

This is the same place we have been pointing to for a while. Model routing, fallback, token-cost governance, and safety are cross-cutting concerns that belong in the infrastructure layer rather than scattered through application code. Put them at the gate, and you can change models, run them air-gapped, and keep one audit trail across the three trust boundaries where AI traffic actually crosses. None of this is new architecture invented for the occasion. The AI Gateway here is part of Traefik Hub, built on the same Traefik foundation that teams already run in front of their traffic. The air-gapped deployment path is the one defense and healthcare teams have been asking us about for a year. June 12 simply gave everyone else a concrete reason to care.

The New Question Is Who Can Switch You Off

The sovereignty conversation has mostly been about data: where it lives, which jurisdiction's rules apply, and whether a region's name on a contract satisfies an auditor. Those questions still matter.

The Fable 5 directive added a different one. Can someone outside your organization turn off the model your product runs on without notice and without recourse? For a hosted frontier model, the answer is now demonstrably yes, and that someone does not have to be your vendor. For a model whose weights sit on a disk you own, behind a gateway you operate, the answer is no. That is the whole distinction, and as of this spring, it is finally buildable without surrendering so much capability that no one would choose it.

We built the maturity model to help teams find out where they really stand. If you read it in April and filed sovereignty under "important, not urgent," this is the part where the second word changes.

Frequently Asked Questions

Does running an open-weight model mean giving up the frontier?

For now, partly. The strongest open model, Kimi K2.6, trails the best closed models, and the best US open model, Nemotron 3 Ultra, trails Kimi. The gap is real and far smaller than it was a year ago. For many production workloads, it is small enough that removing the external dependency is the better trade. For the few tasks that genuinely need the absolute top of the frontier, a gateway lets you route only those to a hosted model and keep everything else local.

Isn't this just a hardware problem in disguise?

Partly, yes. A 550B model needs a serious GPU, and self-hosting trades an API bill for a hardware and operations bill. That is a real cost and worth modeling carefully. What you buy with it is continuity that no external party can revoke. Whether it is worth the spend depends on how much your product would suffer if the model went dark on a Friday afternoon.

We already route through a gateway. Are we covered?

You are covered for the swap, which is the hard part. If your apps reach models through a gateway rather than calling providers directly, changing the model is a configuration change instead of a cross-service rewrite. The piece most setups still miss is that the model is only one of three boundaries an agent crosses. Traefik runs all three as one control plane from a single binary: the API Gateway in front of your services, the AI Gateway in front of the models, and the MCP Gateway in front of the tools agents call. Routing the model locally closes one dependency. Running the safety checks, tool authorization, and policy for all three at the same gate, on your own infrastructure, is what keeps the entire chain off the public internet.

]]>

Service Middlewares: A New Composition Layer In Traefik v3.7

Simon Delicata — Tue, 09 Jun 2026 21:46:49 GMT

For most of Traefik’s existence, every middleware has been attached to a router. That model works well for what it was built for: access control, rate limiting, CORS, redirects—things that describe the route’s policy. But some transformations have nothing to do with the route. They exist because a backend has its own contract, and every request reaching that backend needs to honor it. With one router pointing to one dedicated service, you can fudge that by hanging backend-contract middlewares off the router. The model breaks the moment a service is shared by several routers, or composed from several backends.

The composite case makes this structural. As soon as you reach for composite Traefik services (e.g., weighted splits for canary deployments, mirroring for shadow validation, and failover between heterogeneous upstreams) a single router middleware can no longer express what you need. If you want to add a header to the canary branch only, for example, a router middleware applies to both. Or maybe you want to strip credentials from mirrored traffic, but your router middleware would strip them from the primary too. What if you want each side of a failover to carry its own upstream conventions, but there's only one middleware chain and it runs before the service selection.

These aren’t edge cases you can work around with better routing rules. They’re things the router-middleware model simply cannot express.

Traefik v3.7 adds a second layer. Middlewares can now live on the service itself, executing after the routing decision and, crucially, after the service-selection decision in composite services. This post walks through the scenarios where that matters.

The Two-Layer Model

The historical pipeline was straightforward:

Request → Route match → Router middlewares → Service → Backend

Service middlewares add a second execution point:

Request → Route match → Router middlewares → Service → Service middlewares → Backend

The rule of thumb fits in two sentences: If the middleware concerns who can access or how the request arrives, it belongs on the router. If it concerns how the backend receives the request, it belongs on the service.

Authentication (user-facing), rate limiting meant to protect a route, and CORS are router middlewares. Backend-specific header adaptation, credential injection for an upstream API, and path rewriting that matches a backend’s expectations are service middlewares. The split is almost always obvious once you ask which layer owns the concern.

Three things follow from this model. First, the Middlewares field sits on the Service type itself, so it’s available uniformly whether the service is a plain LoadBalancer, a Weighted, a Mirroring, or a Failover. Second, in composite services (Weighted, Mirroring, Failover), service middlewares on a child execute only on the branch that actually receives the request, which is what makes every scenario below work. Third, the Kubernetes Gateway API expresses the exact same idea through backendRefs[].filters; what Traefik calls a service middleware, Gateway API calls a backendRef filter.

One practical note on where middlewares attach. In the Kubernetes CRD and the Gateway API, you can nest middlewares inline on a child of a composite (a weighted entry, a mirror, a failover branch). In the file provider, a weighted child or a mirror entry is just a reference (no inline middleware field) so per-branch behaviour is expressed by making the child a standalone service that carries its own middlewares. Same outcome, one level of indirection. Scenario 3 below uses this form.

Scenario 1: Canary with Per-Backend Transformation

You run a Weighted service that sends 90% of traffic to the stable version of your API and 10% to a canary. The canary was built against a newer internal contract and expects an extra header (let's say X-Feature-Flag: new-engine) that the stable backend doesn’t recognise and would silently ignore (or, worse, reject).

Attaching X-Feature-Flag to the router adds the header to both branches. Removing it from the router and writing application logic to forward it conditionally moves the problem somewhere else. Duplicating the router for each variant defeats the purpose of weighted traffic in the first place.

With service middlewares, the transformation lives on the canary child only:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: add-feature-flag
spec:
  headers:
    customRequestHeaders:
      X-Feature-Flag: new-engine
---
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: api
spec:
  weighted:
    services:
      - name: api-stable
        port: 80
        weight: 90
      - name: api-canary
        port: 80
        weight: 10
        middlewares:
          - name: add-feature-flag

The stable branch is untouched. The canary branch fires the middleware after the weighted decision has been made, so it only applies to the 10% that actually reach the canary backend. Roll the canary back by flipping the weights to 100 / 0 and there's no middleware to remove, no route to change.

The Gateway API expresses this with its own vocabulary. The same idea, “apply this header modification to this backend only,” becomes a filter on a backendRef:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api
spec:
  rules:
    - backendRefs:
        - name: api-stable
          port: 80
          weight: 90
        - name: api-canary
          port: 80
          weight: 10
          filters:
            - type: RequestHeaderModifier
              requestHeaderModifier:
                set:
                  - name: X-Feature-Flag
                    value: new-engine

Traefik v3.7 supports RequestHeaderModifier, ResponseHeaderModifier, RequestRedirect, and URLRewrite on backendRefs[].filters, plus ExtensionRef for Traefik-native middlewares. If you already standardise on Gateway API, this is the native spelling and you get per-backend transformations without leaving the spec. If you’re on the Traefik CRDs, the TraefikService version above gives you the same behaviour. Same idea, two spellings.

Scenario 2: Mirroring with Redaction

Traffic mirroring is how you validate a new implementation, a shadow analytics pipeline, or a benchmarking rig against real requests without risking production. Traefik’s Mirroring service duplicates incoming requests to one or more mirror backends while the primary response is what the client actually sees.

The problem is that mirrored requests are identical copies of the originals. Every Authorization header, every session cookie, every API key your primary backend receives also ends up on the mirror. If the mirror is run by a different team, lives in a sandbox environment, or is instrumented with verbose logging, you’ve just extended the trust boundary of your production secrets to places it probably shouldn’t reach.

Router middlewares can’t help: stripping Authorization from the router would also strip it from the primary backend and break production auth. Service middlewares, placed on the mirror branch only, solve it:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: scrub-secrets
spec:
  headers:
    customRequestHeaders:
      Authorization: ""
      Cookie: ""
      X-Api-Key: ""
      X-Shadow: "true"
---
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: orders-with-shadow
spec:
  mirroring:
    name: orders-primary
    port: 80
    mirrors:
      - name: orders-shadow
        port: 80
        percent: 100
        middlewares:
          - name: scrub-secrets

The primary backend keeps its full request. The mirror receives the same request with credentials removed and an X-Shadow: true marker so its logs are unambiguous. The scrub is a single piece of configuration that can be audited in one place, which matters when the middleware is a security control, not a convenience.

Scenario 3: Failover Between Heterogeneous Providers

A Failover service sends traffic to a primary backend and falls back to a secondary when the primary is unhealthy. When the two backends are identical (i.e., two replicas of the same service) no per-branch transformation is needed. But when they aren’t identical, each side has its own contract.

Take payment processing, for example. Your primary is Cashew, your fallback is Doshly. Cashew expects Accept: application/vnd.cashew.v2+json and accepts charges at /v1/charges, while Doshly wants Accept: application/vnd.doshly.V3+json and routes payments through /v3/payments. A router middleware can only set one Accept header and one path. Whichever backend the failover selects, the other’s contract is wrong.

Service middlewares let each branch carry its own adaptation:

http:
  services:
    payments:
      failover:
        service: cashew
        fallback: doshly
        errors:
          status:
            - "500-599"

    cashew:
      loadBalancer:
        servers:
          - url: "https://api.cashew.io"
      middlewares:
        - cashew-version
        - cashew-path

    doshly:
      loadBalancer:
        servers:
          - url: "https://checkout.doshly.io"
      middlewares:
        - doshly-version
        - doshly-path

  middlewares:
    cashew-version:
      headers:
        customRequestHeaders:
          Accept: "application/vnd.cashew.v2+json"

    cashew-path:
      replacePath:
        path: "/v1/charges"

    doshly-version:
      headers:
        customRequestHeaders:
          Accept: "application/vnd.doshly.v3+json"

    doshly-path:
      replacePath:
        path: "/v3/payments"

The failover decision picks the right backend and the right contract in a single configuration. Switch upstreams by swapping which one is primary. The middleware chain follows automatically.

Scenario 4: Per-Backend Rate Limiting for a Fragile Upstream

Your search endpoint fans out to two backends through a Weighted service: an internal catalog that handles thousands of requests per second without flinching, and a partner’s public API with a hard quota of 100 req/sec. Cross it and you get a barrage of 429s plus a 30-second cool-down that hurts everyone downstream.

A router-level rate limit forces you to size for the slowest backend. Throttle at the partner’s 100 req/sec and you choke the internal catalog for no reason. Keep the router’s limit high and you push the partner over the cliff on every surge. Either way, when your tier is renegotiated and the quota moves, you have to find every router that touches this service to update it.

A service middleware puts the limit with the backend that actually has the constraint:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: partner-quota
spec:
  rateLimit:
    average: 80
    burst: 40
    # bucket by request Host rather than by client IP — only the
    # public search host routes to this backend, so every bucket
    # collapses to one, giving us a single global counter that
    # protects the upstream instead of fairly sharing among clients
    sourceCriterion:
      requestHost: true
---
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: search
spec:
  weighted:
    services:
      - name: catalog-internal
        port: 80
        weight: 70
      - name: catalog-partner
        port: 80
        weight: 30
        middlewares:
          - name: partner-quota

The internal branch runs uncapped. The partner branch is protected from your own bursts, so you stop pushing it over the cliff and triggering 429-storms. Callers see a Traefik 429 you can absorb (cache, retry with backoff, degrade gracefully) instead of a partner-side rejection that takes half a minute to clear. When the quota changes, one object updates.

The same reasoning extends to circuitBreaker for unpredictable degradation: rate limit when you know the quota up front, circuit breaker when you have to react to symptoms. Both belong on the service that’s fragile, not on a router that doesn’t know which branch the request will end up on.

Scenario 5: External API with Credential Injection

Your platform proxies requests to an external geocoding API that authenticates with a custom header. Internally, three routes send traffic to this service: one from the public search page, one from the admin dashboard, one from a batch processing pipeline. Each route has its own access control, but the geocoding API always needs the same X-Api-Key injected.

Without service middlewares, you’d duplicate the credential-injection middleware on each router, and every new route to this service needs to remember to include it. Forget one, and requests arrive at the external API without authentication, producing silent 401s that are hard to trace.

With a service middleware, the injection lives on the service itself:

http:
  routers:
    geocoding-public:
      rule: "Host(`search.example.com`) && PathPrefix(`/geocode`)"
      middlewares:
        - public-rate-limit
      service: geocoding

    geocoding-admin:
      rule: "Host(`admin.internal`) && PathPrefix(`/geocode`)"
      middlewares:
        - admin-ipallowlist
      service: geocoding

    geocoding-batch:
      rule: "Host(`batch.internal`) && PathPrefix(`/geocode`)"
      middlewares:
        - batch-basicauth
      service: geocoding

  services:
    geocoding:
      loadBalancer:
        servers:
          - url: "https://api.geocoding-provider.com"
      middlewares:
        - inject-geo-key

  middlewares:
    inject-geo-key:
      headers:
        customRequestHeaders:
          X-Api-Key: "geo_prod_xxx"

Each route keeps its own authentication. The backend’s credential (i.e., how the upstream API wants to be spoken to) lives in one place and travels with the service. Add a fourth route without knowing that inject-geo-key exists; it works automatically.

One caveat: customRequestHeaders takes literal strings, not Secret references. Source the key through your secret-injection workflow (templated manifests, External Secrets Operator, sealed secrets) rather than committing the value as shown.

More Places This Fits

The same pattern shows up in dozens of situations. A few that don’t need a full walkthrough but are worth knowing about:

Circuit Breaker on a Fragile Backend

Some upstreams don’t fail at a predictable quota; they return 500s in bursts, intermittent failures that cascade if you keep sending traffic. A circuitBreaker as a service middleware trips when the error ratio crosses a threshold and gives the backend time to recover, while the other backends in the composite keep serving normally.

Plugin-Driven Body Transformations

Built-in middlewares don't touch request or response bodies; plugins do. As service middlewares, they slot into the same composition model. A plugin on a weighted child can rewrite JSON for the migrating branch only, leaving the legacy branch untouched until the migration finishes.

Header Translation During a Backend Migration

You’re shifting traffic from a legacy backend that expects Authorization: Bearer … to a new one with its own X-Api-Key (a backend-owned secret, independent of the client’s bearer). A Weighted service runs the split; the new backend carries a middleware that sets its X-Api-Key and strips the inbound Authorization. Roll forward by adjusting the weights. When the legacy service is decommissioned, the translation middleware goes with it.

Which Middleware Goes Where

Here's a short guide to keep handy. While there are no hard and fast rules, the guide below provides some default answers to the key question, "what is this middleware actually about?"

Middleware Concern	Layer
User authentication (who can access the route)	Router
Credential injection for the upstream API	Service
Rate limit to protect a route or a client	Router
Rate limit to protect a fragile backend	Service
CORS	Router
Strip prefix, path rewrite	Service
Request headers adapting to a backend’s contract	Service
Response headers shaping the client view	Router
User-facing redirects	Router
Host header rewriting for upstream routing	Service

When you’re unsure, ask whether the middleware would still make sense if you swapped the backend for an entirely different implementation. If the answer is “yes, the middleware is still needed” it belongs on the router. If the answer is “no, this is specific to how this backend wants to be spoken to” it belongs on the service.

Service middlewares are available from Traefik v3.7 onwards, in the file provider, the Kubernetes CRD (standalone IngressRoute services and all TraefikService variants), the Kubernetes Ingress provider (via the traefik.ingress.kubernetes.io/service.middlewares annotation), and the Gateway API (backendRefs[].filters). The full reference is in the Traefik documentation.

]]>

Ingress NGINX Migration Guide (3 of 3): Cut Over One Ingress at a Time with Zero Downtime

Nicolas Mengin — Wed, 20 May 2026 16:50:50 GMT

A real zero-downtime migration is not a moment; it is an invariant. At every step, including the one you are about to take, there must be a working configuration you can revert to in seconds. Not a backup. Not a recovery plan. A live, running, traffic-serving configuration, one kubectl edit away.

That is what this post is about. The combination of Traefik v3.7's native Ingress NGINX annotation support and its priority-based routing model lets you keep that invariant from the first packet to the last. You move Ingresses one at a time. You always have somewhere to fall back to. And when the last Ingress is migrated, you find out you have been in production on Traefik for weeks already.

This is the third and final post of the series, published alongside the Traefik v3.7 GA:

Audit: inventory your annotations and translate the NGINX ConfigMap.
Install: deploy Traefik with Helm next to your existing NGINX controller.
Progressive migration (this post): cut over route by route, with zero downtime.

The official Traefik migration guide documents a blue-green approach. This post takes a different path, one designed for teams that cannot freeze a cluster for an hour, let alone a weekend.

The Catch-All is the Invariant

Every progressive migration in this series rests on one mechanism: a Traefik catch-all router sitting in front of Ingress NGINX, with a priority lower than any real Ingress. Traefik forwards any request for which it has no matching rule to Ingress NGINX unchanged.

That router is the safety net. As long as it is in place, every request that does not match a migrated Ingress still flows through Ingress NGINX exactly as it did before. You can migrate one Ingress, ten, or zero, and the rest of your production traffic does not know the difference.

There is no big-bang. There is no maintenance window. There is no "migration day". There is a sequence of tiny, individually reversible changes, each protected by the same fallback.

The rest of this post walks through the two operational moves that make this work: putting Traefik in front of Ingress NGINX (Step 1), and migrating Ingresses one by one (Step 2). They both rely on the catch-all routes, as well as on the audits and installations you have already carried out.

Initial Situation

Coming out of the second post:

Ingress NGINX serves all external traffic via a LoadBalancer (or NodePort) service.
An IngressClass named nginx, with controller k8s.io/ingress-nginx, tells the cluster which Ingress resources Ingress NGINX should handle.
Traefik is deployed in the cluster but unreachable from outside. It is configured to accept both Ingress resources (with Ingress NGINX annotations) on the traefik-nginx IngressClass, and Traefik CRDs (IngressRoute, IngressRouteTCP).

Current traffic flow:

For the rest of this post, we will work against a representative production-style Ingress that uses the features most likely to trip up a naive migration: snippets, header manipulation, and SSL redirects.

# IngressClass
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-nginx-annotation
  annotations:
    # SNIPPETS
    nginx.ingress.kubernetes.io/server-snippet: |
      location = /exact {
            return 200 "exact-match";
          }
    nginx.ingress.kubernetes.io/configuration-snippet: |
      # Add Headers on the response
      add_header X-Method $request_method;
      set $my_var "hello";
      add_header X-My-Var $my_var;
      set $combined "$request_method-$request_uri";
      add_header X-Combined $combined;
      # Add/Set Headers on the request
      proxy_set_header X-Backend-Uri $request_uri;
      more_set_input_headers "X-Custom-Input:input-value";
    # SSL Redirect
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  rules:
  - host: whoami.docker.localhost
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: whoami
            port:
              number: 80
  tls:
  - hosts:
    - whoami.docker.localhost
    secretName: external-certs <= Set your TLS files secret name here

For Ingress NGINX to honor the snippets on this Ingress, the controller ConfigMap must allow them:

# NGINX Configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration
data:
  use-forwarded-headers: "true"
  allow-snippet-annotations: "true"
  annotations-risk-level: Critical

Step 1: Put Traefik in Front of Ingress NGINX

This first step is to put Traefik on the traffic path, while Ingress NGINX keeps handling every request unchanged. Once this step is complete, the invariant is established: a single one-line change to the LoadBalancer selector reverts the entire cluster to the pre-Traefik state.

1. Check Traefik Configuration

In the previous blog post, we configured Traefik with the kubernetesIngressNGINX provider pointed at the traefik-nginx IngressClass, the traefik-ingress-controller controller, and the kubernetesCRD provider enabled, so we can define IngressRoute and IngressRouteTCP resources in the next step.

Before we go further, double-check the Traefik deployment is wired the way we expect:

$kubectl describe deployment/traefik
…
Pod Template:
  Labels:           app.kubernetes.io/name=traefik
  Service Account:  traefik-ingress-controller
  Containers:
   traefik:
    Image:       traefik:v3.7.0
    Ports:       80/TCP, 8080/TCP, 443/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP
    Args:
      --api.insecure
      --entrypoints.traefik.address=:8080
      --entrypoints.web.address=:80
      --entrypoints.websecure.address=:443
      --entryPoints.websecure.http.tls=true
      --providers.kubernetesIngressNGINX.ingressClass=traefik-nginx
      --providers.kubernetesIngressNGINX.ingressClassByName=true
      --providers.kubernetesIngressNGINX.controllerClass="traefik.io/ingress-controller"
      --providers.kubernetesIngressNGINX.allowSnippetAnnotations=true
      --providers.kubernetesIngressNGINX.httpentrypoint=web
      --providers.kubernetesIngressNGINX.httpsentrypoint=websecure
      --providers.kubernetescrd

2. Expose Ingress NGINX as a ClusterIP service

For Traefik to forward to Ingress NGINX, Ingress NGINX needs a stable internal address. We create a ClusterIP Service that selects the existing Ingress NGINX pods. The external LoadBalancer stays untouched for now, Ingress NGINX is reachable from two places, the LoadBalancer (still serving production) and the new ClusterIP (about to be used by Traefik).

# Internal ClusterIP Service for ingress-nginx
apiVersion: v1
kind: Service
metadata:
  name: internal-nginx-svc
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 443

3. Add Catch-All Routes

We create two resources, one IngressRouteTCP and one IngressRoute, one for each entrypoint. These match every hostname and forward the request to Ingress NGINX. The TLS entry point uses passthrough mode, meaning ingress-nginx continues to terminate TLS exactly as before.

# Catch-all TCP route for TLS connections (TLS passthrough)
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: traefik-catchall-tcp-tls
spec:
  entryPoints:
  - websecure
  routes:
  - match: HostSNI(`*`)
    services:
    - name: internal-nginx-svc
      port: 443
  tls:
    passthrough: true
---
# Catch-all HTTP route for non-TLS connections
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-catchall-http
spec:
  entryPoints:
  - web
  routes:
  - match: HostRegexp(`.*`)
    services:
    - name: internal-nginx-svc
      port: 80

4. Swing the LoadBalancer to Traefik

To complete step 1, the final move is to update the external LoadBalancer selector from ingress-nginx to traefik. This is the only change visible to external clients, and it is instantaneous. Traefik starts receiving traffic and, because the only routes it knows about are the two catch-alls, forwards every request untouched to ingress-nginx.

apiVersion: v1
kind: Service
metadata:
  name: ingress-controller
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: traefik # <= was ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 443

New traffic flow:

There is no user impact: every request passes through Traefik and arrives at ingress-nginx exactly as before.

The Traefik dashboard confirms the state. Besides the routers that expose the dashboard itself, only two catch-all routers are active, one TCP and one HTTP, both at low priority.

Step 2: Migrate Ingress Resources One by One

With Traefik in place and a live fallback running, we can now move each Ingress resource to Traefik independently, at whatever pace suits the team.

The process is the same for each Ingress resource:

Edit the manifest and change ingressClassName from nginx to traefik-nginx.
Apply.
Traefik picks it up immediately and creates an HTTP router. Because the new router matches a specific host (or path), and the catch-all matches *, the more specific rule wins. From that instant on, the Ingress is served by Traefik directly.
Confirm in the Traefik dashboard that the new router is active and green.

Here is the updated Ingress. Note that the only change is the IngressClass name:

# Ingress, updated to use the traefik-nginx IngressClass
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-nginx-annotation
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |
      location = /exact {
            return 200 "exact-match";
          }
    nginx.ingress.kubernetes.io/configuration-snippet: |
      # Add Headers on the response
      add_header X-Method $request_method;
      set $my_var "hello";
      add_header X-My-Var $my_var;
      set $combined "$request_method-$request_uri";
      add_header X-Combined $combined;
      # Add/Set Headers on the request
      proxy_set_header X-Backend-Uri $request_uri;
      more_set_input_headers "X-Custom-Input:input-value";
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: traefik-nginx  # <= changed from nginx
  rules:
  - host: whoami.docker.localhost
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: whoami
            port:
              number: 80
  tls:
  - hosts:
    - whoami.docker.localhost
    secretName: external-certs <= Set your TLS files secret name here

The annotations are unchanged. Traefik v3.7 reads them natively and translates them into the equivalent routing rules and middlewares. The audit you did in the first post is what makes this trustworthy: you already know, before you apply, that this Ingress falls in the Supported bucket.

After applying, the dashboard shows the new HTTP router active alongside the remaining catch-all routes:

At this point, two flows coexist in the cluster: one for the migrated Ingresses, and still the previous one for the not-yet-migrated Ingresses.

Rolling back is symmetrical: flip the IngressClass back to nginx, and the Ingress returns to ingress-nginx; the catch-all takes care of the rest. Nothing else in the cluster needs to know.

Repeat for each Ingress, at whatever cadence the team can handle. One per sprint, ten per afternoon, fifty in a coordinated push. The catch-all does not care.

Once the last Ingress has been migrated, Ingress NGINX is no longer needed. Clean up in two steps:

Delete the catch-all IngressRoute and IngressRouteTCP resources (traefik-catchall-http and traefik-catchall-tcp-tls).
Remove the ingress-nginx resources you don’t need anymore, keeping the Load-Balancer Service:
a. The Deployment (ingress-nginx-controller)
b. The internal ClusterIP service (internal-nginx-svc)
c. The ConfigMap (nginx-configuration)
d. The RBAC (ingress-nginx ClusterRole and ClusterRoleBinding)
e. The ServiceAccount (ingress-nginx)
f. The original nginx IngressClass.

If you installed ingress-nginx via Helm, add the annotation helm.sh/resource-policy: keep to the ingress-nginx-controller Service prevents the uninstall from deleting it. That matters: that Service holds the LoadBalancer IP your DNS points at, you do not want to lose it.

Final traffic flow:

Et voilà. Traefik is now the sole Ingress Controller, handling every route with full native support for the existing annotation-based configuration, no manifest rewrites required.

Conclusion

The way most teams think about an Ingress controller migration is: we have to swap A for B, what is the safest moment to do it? The answer is always disappointing, because there is no safe moment to swap one production component for another.

The way to migrate is to stop framing it as a swap at all. Put Traefik on the traffic path with a catch-all, and from that point on, you are not migrating a controller; you are migrating Ingress resources, one at a time, each with its own atomic decision and its own one-line rollback. The catch-all is what makes the difference between "we need a maintenance window" and "we can do this Tuesday afternoon".

Traefik v3.7 makes the per-Ingress move trivial because it reads ingress-nginx annotations natively. The audit told you which Ingresses are in the safe bucket. The install put Traefik in the cluster without touching production. This post connects the two: with a single catch-all router, every Ingress migration becomes the smallest possible change.

Give it a try, and let the Traefik community know how it goes!

Useful Links

]]>

Ingress NGINX Migration Guide (2 of 3): Traefik Lands, Ingress NGINX Stays Safe.

Emile Vauge — Wed, 13 May 2026 15:09:11 GMT

The riskiest moment in an Ingress NGINX migration is not the cutover. It is the second between "we have to migrate" and "we have actually started". Teams either freeze (because the project feels enormous) or jump straight to the cutover (because they want it over with). Both produce outages.

There is a much better way: install Traefik in your cluster as a second controller, with its own IngressClass, and let it sit there. Your existing Ingress NGINX keeps serving everything. No traffic moves yet. Once Traefik is ready and validated, the third post walks through moving your Ingress resources to it one by one.

This is the second post of a 3-part series on migrating off Ingress NGINX, published alongside the Traefik v3.7 GA:

Audit: inventory your annotations and translate the NGINX ConfigMap (read our guide here).
Install (this post): deploy Traefik with Helm next to your existing NGINX controller.
Progressive migration: cut over route by route, with zero downtime.

This post focuses on the Traefik installation only. Moving your Ingress resources from Ingress NGINX to Traefik, one at a time, is the topic of the third post. Here, we get Traefik deployed, configured with the new IngressClass, and validated against a duplicated test Ingress, sitting next to Ingress NGINX and ready to take over when you decide.

If you have not run the audit yet, do that first. This post assumes you already know which of your Ingress resources fall in the Vanilla, Supported, Unsupported, and Invalid buckets, and that your global ConfigMap has been translated to Traefik's static configuration.

Why Install Side by Side, Instead of Replacing in Place

The instinct, especially under deadline pressure, is to uninstall Ingress NGINX and install Traefik. Same IngressClass, same Service, same LoadBalancer IP. Done by lunchtime.

Don't.

When the new controller takes over the same traffic path, every misconfigured annotation, every forgotten ConfigMap setting, every silently different default becomes a production incident. The blast radius is your entire cluster. The rollback is a re-installation of the old controller, which takes time you do not have when 80 services are returning 503.

The side-by-side install eliminates this class of failure. You deploy Traefik in its own namespace, on a new, dedicated IngressClass (traefik-nginx). Your existing Ingress resources keep their nginx IngressClass. They keep flowing through Ingress NGINX, untouched. Traefik is in the cluster, configured, and idle until you explicitly point an Ingress at it.

More importantly, this separation is what makes a progressive migration possible. Once both controllers are running, you can move your Ingress resources from Ingress NGINX to Traefik one at a time: edit a single Ingress, change its ingressClassName from nginx to traefik-nginx, validate, move on to the next. No big-bang. No frozen weekend. Each route is its own atomic decision, with its own rollback (just flip the IngressClass back). The Traefik migration guide documents this as a blue-green pattern, and it is exactly what the third post will walk through. This post focuses on getting Traefik installed and ready for it.

Install Traefik With the Official Helm Chart

The traefik/traefik-helm-chart is the canonical install path. Start by adding the repo:

helm repo add --force-update traefik <https://traefik.github.io/charts>

Create a dedicated namespace for the new controller. Keeping it separate from ingress-nginx is good hygiene: it makes RBAC scoping, monitoring, and uninstall trivial later.

kubectl create namespace traefik

Now the values file. The key piece is providers.kubernetesIngressNGINX.enabled: true, with the new IngressClass and controllerClass that Traefik will own:

# traefik-values.yaml

providers:
  # Read Ingress resources using the new traefik-nginx IngressClass.
  # Existing nginx-class Ingresses are NOT touched: they keep flowing
  # through ingress-nginx until you explicitly migrate them (the third post).
  kubernetesIngressNGINX:
    enabled: true
    # New IngressClass that Traefik owns, distinct from ingress-nginx's "nginx".
    ingressClass: "traefik-nginx"
    ingressClassByName: true
    # Traefik's own controller class for the new IngressClass.
    controllerClass: "traefik.io/ingress-controller"
    # Accept snippet annotations when migrated Ingresses use them.
    allowSnippetAnnotations: true

  # Standard Kubernetes Ingress provider disabled: we only consume the new class.
  kubernetesIngress:
    enabled: false

# Let the chart create the IngressClass that Traefik watches.
# This is the same IngressClass that the kubernetesIngressNGINX provider
# above is configured to consume. Not default-class: we do not want any
# Ingress without an explicit class to land on Traefik.
ingressClass:
  enabled: true
  isDefaultClass: false
  name: "traefik-nginx"

# No Service for Traefik yet. The third post will repoint your existing
# ingress-nginx Service (which already has a LoadBalancer IP) to Traefik's
# pod selector. No need to provision anything extra to validate, port-forward
# to the pod is enough.
service:
  enabled: false

# Dashboard for validation, reached via port-forward during install.
ingressRoute:
  dashboard:
    enabled: true

Install with:

helm install traefik traefik/traefik \\
  --namespace traefik \\
  --values traefik-values.yaml

That's the entire install. One Helm command, one values file. The chart deploys the controller and creates the traefik-nginx IngressClass in the same operation. The new controller is running, configured to watch a class that nothing in your cluster uses yet, with no LoadBalancer of its own. It is in the cluster, but invisible to production traffic.

Verify the pods are up:

kubectl -n traefik get pods

You should see one (or more) Traefik pod in Running state. No Service in the namespace. That is by design: the third post will reuse the existing Ingress NGINX Service by editing its selector.

Translate the ConfigMap Into Static Configuration

If your audit identified ConfigMap keys that need to come over (timeouts, body sizes, default backend, allow-cross-namespace, etc.), the corresponding Traefik options live under providers.kubernetesIngressNGINX in the same values file. The mapping is in the migration guide.

A typical example:

providers:
  kubernetesIngressNGINX:
    enabled: true
    ingressClass: "traefik-nginx"
    ingressClassByName: true
    controllerClass: "traefik.io/ingress-controller"
    allowSnippetAnnotations: true

    # Translated from ingress-nginx ConfigMap
    proxyBodySize: 16777216           # was proxy-body-size: "16m"
    proxyConnectTimeout: 30           # was proxy-connect-timeout: "30"
    proxyBuffering: true              # was proxy-buffering: "on"
    allowCrossNamespaceResources: true
    globalAllowedResponseHeaders:
      - X-Frame-Options
      - X-Custom-Header

Apply with helm upgrade traefik traefik/traefik -n traefik -f traefik-values.yaml.

Cross-Cutting Keys: TLS Options as a Concrete Example

Some ConfigMap keys do not belong under the provider at all. The clearest case is TLS policy: ssl-protocols, ssl-ciphers, ssl-prefer-server-ciphers. In Ingress NGINX these are single cluster-wide knobs in the ConfigMap. In Traefik, they live in a TLSOption CRD that can be cluster-wide (named default) or per-Ingress.

Say your Ingress NGINX ConfigMap has:

data:
  ssl-protocols: "TLSv1.2 TLSv1.3"
  ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384"
  ssl-prefer-server-ciphers: "true"

The Traefik equivalent is a single TLSOption named default, which Traefik applies cluster-wide to every route that does not specify its own:

# default-tls-options.yaml
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: default
  namespace: traefik
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  preferServerCipherSuites: true

kubectl apply -f default-tls-options.yaml

Two things to know when you do this translation:

Cipher names. NGINX uses OpenSSL names (ECDHE-ECDSA-AES128-GCM-SHA256); Traefik uses Go's crypto/tls names (TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256). The mapping is one-to-one but the strings are different. The TLS 1.3 cipher suites in Go are non-configurable, so they're omitted from the list.
TLS versions. ssl-protocols: "TLSv1.2 TLSv1.3" becomes minVersion: VersionTLS12. Traefik will also accept TLS 1.3 unless you set maxVersion. To explicitly cap at one version, set both minVersion and maxVersion to the same value.

The same pattern applies to the other cross-cutting keys: HSTS (hsts, hsts-max-age, hsts-include-subdomains) becomes a Headers middleware attached to an entryPoint, and PROXY protocol (use-proxy-protocol) becomes an entryPoint setting in the same traefik-values.yaml you used for install. The ConfigMap migration step of the guide lists the full mapping.

Validate With a Duplicated Ingress

Now the most important part: prove that Traefik would correctly serve your Ingresses before anyone reroutes traffic.

Because Traefik watches the new traefik-nginx IngressClass, it sees none of your existing production Ingresses. That is the safe default. To validate that Traefik handles your annotations correctly, pick one or two representative Ingresses from your audit and duplicate them into a test namespace with the new IngressClass. The original Ingresses stay on Ingress NGINX, untouched. The clones run on Traefik.

Pick something that exercises the annotations you care about, for example one with snippets, authentication, or rewrite rules. Then duplicate it like this:

# test/duplicate-whoami.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: app-prod-test
---
# Same Deployment + Service as in production, copied verbatim
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: app-prod-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: app-prod-test
spec:
  selector:
    app: whoami
  ports:
    - port: 80
      targetPort: 80
---
# Same Ingress as in production, with two differences:
# 1. ingressClassName: nginx -> traefik-nginx
# 2. host gets a -test suffix so it does not collide
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  namespace: app-prod-test
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      add_header X-Custom-Header "from-snippet";
spec:
  ingressClassName: traefik-nginx
  rules:
    - host: whoami-test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80

kubectl apply -f test/duplicate-whoami.yaml

Now hit it through Traefik via port-forward:

kubectl -n traefik port-forward deploy/traefik 18000:80 &

curl -i -H "Host: whoami-test.example.com" <http://localhost:18000/>

You should see a 200 OK, the whoami response body, and the headers your annotations declared (in this example, X-Custom-Header: from-snippet). If the annotations land correctly here, they will land correctly in production once you move the real Ingress in the third post.

Repeat for two or three of your higher-risk Ingresses. The audit told you which ones deserve scrutiny; this is the place to scrutinize them. Once you are confident, delete the test namespace (or keep it as a permanent staging surface for future migrations):

kubectl delete namespace app-prod-test

Open the dashboard to see what Traefik knows:

kubectl -n traefik port-forward deploy/traefik 9000:9000 &
xdg-open <http://localhost:9000/dashboard/>   # use `open` on macOS

In steady state, the dashboard will show only your test routes (since no production Ingress is in traefik-nginx yet). That is the whole point: the dashboard becomes a real-time witness of what you migrate, route by route, in the third post.

A Pre-Flight Checklist Before You Migrate

Before you move to the progressive migration (the third post), confirm:

Traefik pods are healthy and running in the traefik namespace
The traefik-nginx IngressClass exists, with controller traefik.io/ingress-controller
Your test Ingress (or two) running with ingressClassName: traefik-nginx responds correctly through Traefik, with the expected annotations applied
Your ConfigMap-derived settings (timeouts, body sizes, allowlists) are in your values file and visible in the Traefik container args
The Traefik dashboard is reachable via kubectl port-forward
No production Ingress has been switched to traefik-nginx yet. Everything that matters still flows through Ingress NGINX.

If every box is checked, you have done the safest thing possible: you have a fully functional Traefik controller, validated against duplicated copies of your real Ingresses, sitting in the cluster waiting to be used. Nothing has shipped to production yet.

What Comes Next

If you followed the steps above, everything is now in place for the migration. Traefik is installed, configured, and watching its own dedicated IngressClass. Your ConfigMap-derived settings are translated. Your TLS policy is in a TLSOption. You have curl-validated at least one duplicated Ingress and confirmed Traefik handles your annotations correctly. Production traffic still flows through Ingress NGINX, untouched.

That setup is not an accident. Every decision in this post (separate namespace, new IngressClass, no Service of its own, validation through a duplicate rather than the real Ingress, no DNS or LoadBalancer change yet) was made so that the actual migration in the third post is as boring and low-risk as possible. There is no big-bang to schedule. When you flip the IngressClass on your first real Ingress in the third post, you are doing something you have already proven works, with an instant rollback if anything is off.

The next post walks through that progressive migration: moving your Ingress resources from nginx to traefik-nginx one at a time, using a Traefik catch-all in front of Ingress NGINX to preserve zero-downtime guarantees, and a blue-green pattern that lets you roll back at any point with a single line change. With Traefik already installed and validated, the progressive migration stops being a project and becomes a sprint task.

Resources

Helm chart: github.com/traefik/traefik-helm-chart
NGINX Ingress provider reference: doc.traefik.io
Migration guide: doc.traefik.io/traefik/migrate/nginx-to-traefik (and the ConfigMap migration section)
Migration tool: github.com/traefik/ingress-nginx-migration

]]>

Harness Engineering vs Platform Engineering: A Mental Model for How Both Win

Sudeep Goswami, Immánuel Fodor — Thu, 07 May 2026 21:25:19 GMT

Two posts in early 2026 reframed how the industry thinks about agent quality. LangChain's Deep Agents team published harness engineering numbers that moved their coding agent from outside the Top 30 to rank 5 on Terminal-Bench 2.0, a 13.7-point gain achieved by holding GPT-5.2-Codex fixed and changing only system prompts, tools, and middleware. A few weeks later, Cursor published a long read on how its harness team manages context, classifies errors, runs continuous A/B tests, and tunes per model family. Then came the Cursor SDK, exposing the harness as a programmable surface to anyone. Anthropic's Opus 4.7 release notes called out self-verification as a built-in model capability. A new vernacular took hold: the model is one half, the harness is the other, and anything not in the model lives in the harness.

The conclusion most readers drew was that the harness is the product. That is a useful corrective. It pushes back on the model-of-the-month obsession and puts attention on the engineering work that actually compounds.

But the harness is only half the architecture. The other half is platform engineering, and the boundary between the two is the most consequential decision an enterprise architect makes when planning agentic systems at scale.

An earlier post argued that AI cross-cutting concerns belong at the platform layer for the same reasons HTTP cross-cutting concerns ended up at the API gateway over the past decade. This post takes that conclusion as given and asks the next question: where exactly is the line between platform and harness, what does each plane own, and why does the line keep getting blurred in the field?

What Harness Engineering Actually Owns

A harness is everything wrapping the model inside a single agent process. Anatomies vary, but a useful decomposition has emerged across the major harness publications:

The orchestration loop (the ReAct or Thought-Action-Observation cycle), system prompt assembly, tool definitions and tool-call execution, hooks and middleware around model and tool calls, sub-agent delegation, in-process memory and state, sandboxed code execution, context window compression and offloading, lifecycle management for sessions, structured output parsing, and self-verification logic.

Every one of these decisions is task-specific or model-specific. The harness is where domain knowledge about a particular agent's job lives. It is the right place to encode that a coding agent should re-read its own diff before declaring done, that a research agent should plan then execute, that a support agent should escalate after three failed tool calls. The harness optimizes for one agent's task performance, token budget, and latency profile.

Three things follow from this scope. First, harness engineering is fundamentally an application-layer discipline. Second, harness improvements compound through observation, with techniques like tracing, evaluation harnesses, and iterative tuning against benchmarks. Third, every harness optimizes for itself.

What Platform Engineering Owns

Production agents do not run alone. A single enterprise will run dozens, often hundreds. Different teams pick different harness frameworks. Some agents call frontier APIs, some call self-hosted models, some call sovereign deployments. Some agents call MCP servers operated by the same company; others call MCP servers run by partners or third parties. Some agents serve internal users; others serve external customers under different commercial and regulatory terms.

The concerns that span this fleet are platform engineering concerns. Identity and authentication of the calling agent. Authorization of which tools and data each agent can touch. Audit records that a regulator or incident responder can read. Rate limiting and concurrency control across tenants. Cost attribution and FinOps quotas. Secrets handling. Content filtering for prompt and response payloads. Regional routing for data residency. Failover and circuit breaking when a model provider degrades. Consistent observability across the full traffic graph.

These concerns share three properties. They must be enforced consistently across agents built by different teams using different frameworks. They must be auditable from a single control point. And they must outlive any individual harness or model choice, because the harness ecosystem is moving faster than any enterprise can refactor.

The Triple Gate Pattern (API Gateway for north-south application traffic, AI Gateway for model traffic, MCP Gateway for tool and data-source traffic) is one expression of this platform layer. The specific pattern matters less than the architectural principle: every call leaving the agent process traverses a governed control point.

The Reference Architecture Blueprint

A complete agentic system has three layers with clear seams between them.

Layer 1: The harness, in-process. Owns orchestration, prompt assembly, tool selection logic, sub-agent fan-out, context compression, in-process memory, and self-verification. Owned by the application team. Optimized per agent.

Layer 2: The platform, out-of-process. Owns identity, authorization, audit, rate limits, cost controls, secrets, content filtering, regional routing, failover, and traffic-graph observability. Owned by platform or infrastructure engineering. Enforced uniformly across all agents.

Layer 3: The targets. Models (frontier APIs, self-hosted endpoints, sovereign deployments), tools and MCP servers (internal, partner, third-party), and existing application APIs. Owned by various teams or providers, treated as governed dependencies.

The seam between Layer 1 and Layer 2 is the most important design decision. It is the point at which every model call, every tool invocation, every MCP request leaves the agent process. If that seam is well defined, the harness can evolve independently of the platform. The application team can rip out one harness framework and adopt another without renegotiating governance. The platform team can introduce a new policy without coordinating with every application team.

If the seam is poorly defined, the two layers smear into each other. Authorization decisions end up in harness middleware where the security team cannot audit them. Cost controls end up in platform code where application teams cannot tune them. Every change becomes a cross-team negotiation.

Cross-Cutting Concerns, Mapped

The contested zone is the set of concerns that could plausibly live in either layer. Each entry below passes the cross-cutting test from the previous post (uniformity across applications, no domain knowledge required, systemic blast radius when wrong) but the question this post addresses is sharper: when the call leaves the agent process, what stays inside the harness and what moves to the platform?

Concerns Owned by the Harness (In-Process, Task-Specific)

Concern	Reasoning
Tool selection for a given task	Task-specific; depends on prompt and intermediate state
Per-task model selection (capability fit)	Depends on sub-task semantics
Sub-agent delegation logic	Internal to one agent's planning
Context window compression	Intra-loop performance optimization
Self-verification of model output	Task correctness within a single agent
Eval and tracing of agent reasoning	Reasoning is internal; benefits from harness-native observability
Session persistence and checkpointing	Internal to agent lifecycle

Concerns Owned by the Platform (Out-of-Process, Fleet-Wide)

Concern	Reasoning
Identity of the calling agent	Must be consistent across harnesses; required for audit
Authorization to invoke a tool	Enforces TBAC across all agents; cannot be self-asserted
Secrets injection into tool calls	Secrets must not enter the harness process memory
Rate limits and concurrency caps	Multi-tenant fairness and provider quotas
Per-policy model routing (cost, region, compliance)	Enterprise-wide policy, not per-task
Failover when a model provider degrades	Must be transparent to the harness; consistent SLOs
Content filtering and DLP on prompts and responses	Regulatory and data-protection consistency
Audit trail of every model and tool call	Single source of truth for compliance
Traffic-graph observability and correlation	Cross-agent and cross-tenant visibility
Cost attribution by team or product	Requires enforcement at the call site, not self-report

A simple test for any concern not listed: if the answer must be consistent across multiple agent applications, or if a regulator might ask to see it, the concern belongs at the platform. If the answer is task-specific or model-specific within one agent, the concern belongs in the harness.

The Governance Gravitational Pull

Harness frameworks have a recurring pattern of absorbing concerns that belong at the platform layer. Recent harness releases have introduced declarative permission rules in the harness, model routing logic in harness profiles, secrets handled inside the harness process, and audit hooks bolted onto harness middleware. Each of these is technically possible. None of them scale across an enterprise.

The pull is structural, not malicious. Three forces drive it.

The first is commercial. Harness frameworks compete by being indispensable. The more concerns a harness owns, the more switching cost it accumulates. A harness that handles auth, routing, and audit is stickier than one that does only orchestration. From the vendor's perspective, this is rational behavior.

The second is technical. In-process is faster than out-of-process. A permission check that runs locally is microseconds; a check that traverses a network gateway is milliseconds. For a harness optimizing per-task latency, the in-process answer is always tempting.

The third is organizational. Application teams move faster than platform teams. When a coding agent needs a new tool yesterday, asking the platform team to onboard it next sprint is not a workable answer. So application teams build governance shortcuts inside the harness, intending to migrate them later. They rarely do.

The trap closes when the enterprise has five harnesses, each with its own auth model, each with its own audit format, each with its own rate-limit configuration. The CISO discovers there is no single place to ask which agents are allowed to read which data. The FinOps team discovers that cost attribution is inferred from logs scraped after the fact. The compliance team discovers that audit records are scattered across vendor-specific trace stores. By that point, retrofitting platform-layer governance requires touching every harness.

Anti-Patterns

A short list of anti-patterns observed in real deployments.

Auth in the Harness

Authorization decisions are encoded in harness middleware or skill definitions.

Symptom: The security team cannot answer "which agents can call this MCP server" without reading source code from multiple application repositories.

Remedy: Move authorization to a gateway that enforces a uniform policy language, keyed on agent identity and task context.

Per-Product Harness with No Platform

Each product team picks its own harness, runs its own model contracts, and ships independently.

Symptom: Many model contracts, several audit formats, no consolidated cost view.

Remedy: Standardize the platform layer (gateways, identity, audit) before standardizing the harness layer. The platform should be harness-agnostic; the harness should be platform-aware.

Harness Vendor as Governance Vendor

Teams treat a harness framework's enterprise edition as the governance solution.

Symptom: Governance scope is limited to whatever the harness vendor implements; introducing a second harness requires re-implementing controls.

Remedy: Keep governance in the platform layer where it is harness-independent.

Model List in Harness Configuration

Each harness ships with a hard-coded list of supported models and routing rules.

Symptom: Introducing a new model means coordinating updates across every harness; deprecating one is harder still.

Remedy: Harnesses should request a model class (planning, coding, summarization); the AI Gateway resolves the class to a specific model based on policy, region, and cost.

MCP Servers as Direct Dependencies

Harnesses connect directly to MCP servers over the network without an intermediate gateway.

Symptom: Every MCP server reinvents auth, rate limiting, and audit, often badly. The OWASP MCP Top 10 documents the resulting attack surface in detail.

Remedy: MCP traffic flows through an MCP Gateway that handles identity, TBAC, secrets, and audit on behalf of every server behind it.

A Look Forward

Two trends bear watching.

The harness gets thinner over time. Self-verification, planning, and tool use are migrating into the model itself. Opus 4.7's release notes call out attention to instructions and self-verification as model capabilities, which means harness components built specifically to compensate for those gaps will be absorbed. LangChain has acknowledged this trajectory explicitly, framing harness components as bets on what the model cannot yet do.

The platform gets thicker over time. Every regulatory regime that touches AI (the EU AI Act, sector-specific frameworks for finance and healthcare, sovereign data rules) lands at the platform layer because that is where enforcement is possible. Every cost-control reflex inside a CFO organization lands at the platform layer because that is where attribution can be enforced rather than reported. Every breach analysis lands at the platform layer because that is where the audit record lives.

The architectural takeaway is straightforward. Bet that the harness will keep changing. Bet that the platform will keep accreting requirements. Build the seam between them deliberately, and resist the pull to collapse them into one layer.

The harness is not the product. The model is not the product. The platform is not the product. The product is the agent that ships safely, scales economically, and survives an audit. That requires that both the harness and the platform are owned by the right teams and separated by a clean seam.

Frequently Asked Questions

Our harness framework already has an auth plugin. Why isn't that good enough?

It works for one agent. The problem surfaces when the second and third agents ship, built by different teams on different frameworks, each with its own auth plugin configured differently. At that point there is no single place to answer which agents are allowed to call this data source; the answer is scattered across application repositories. Authorization needs to be enforced at a control point that is independent of any individual harness, keyed on agent identity, not on which framework happened to ship a plugin first.

What is the minimum viable platform layer for a team just starting with agents?

Three things: identity for the calling agent, an audit trail of every model and tool call, and a single egress point for model traffic. Everything else (e.g., cost attribution, content filtering, regional routing) can be layered on top. The mistake most teams make is skipping the egress point entirely and wiring agents directly to model APIs. That decision is cheap to make and expensive to undo once the fleet grows.

How do we handle the latency penalty of out-of-process governance?

The penalty is real but routinely overstated. A well-deployed gateway adds single-digit milliseconds to a call that already takes hundreds of milliseconds to complete a model round-trip. The more important question is where latency actually matters in an agentic workflow. And the answer is almost always inside the orchestration loop, which is harness territory. Platform-layer governance sits outside that loop. Optimize the harness for latency; build the platform for correctness and auditability.

We already have three different harness frameworks in production. Where do we start?

Start with the platform, not the harnesses. Standardizing harness frameworks is a multi-team, multi-quarter negotiation. Standardizing the egress layer is not; it is an infrastructure decision that can be made once and applied uniformly. Get identity, audit, and model routing consistent across all three harnesses first. Once the platform seam is clean, harness consolidation becomes optional rather than urgent.

As models absorb more harness capabilities, does the platform layer shrink too?

The opposite. As self-verification, planning, and tool use migrate into the model, the harness gets thinner and the calls leaving the agent process get more consequential, not less. A model that autonomously selects and invokes tools is making more decisions that require audit, authorization, and cost attribution, not fewer. Every capability that moves from the harness into the model is a capability that now needs to be governed at the platform layer rather than observed inside a single agent process.

]]>

Cross-Cutting Concerns Are Back, and the AI Stack Is Abusing the Gateway Pattern

Immánuel Fodor, Sudeep Goswami — Thu, 07 May 2026 18:34:27 GMT

Some architectural problems do not stay solved. They come back every time the system grows by an order of magnitude, with the same root conditions in a more demanding form. Cross-cutting concerns are one such problem.

They were named decades ago, in the era of aspect-oriented programming. They were resolved at the network layer over the 2010s, when API gateways absorbed authentication, rate limiting, observability, and policy enforcement from every microservice and moved them into a shared infrastructure layer. That lesson took roughly a decade to settle, and the cost of learning it was paid in production breaches, audit findings, and engineering rework done late.

The same pattern is now playing out at the AI layer, on a faster timeline, with larger blast radii. The cross-cutting concerns specific to AI traffic, including prompt safety, content filtering, model fallback, agent authorization, token cost governance, and tool poisoning defense, are being implemented in application code, imported Python libraries, and inside individual MCP servers. The deployment data is already mainstream. The case studies are already public. The architectural answer is not new, but the conditions that make it the right answer are stronger in 2026 than at any prior moment.

What Counts as a Cross-Cutting Concern, and What Does Not

A cross-cutting concern is behavior that needs to apply consistently across many parts of a system but does not belong to any one of them. Authentication is the textbook example. Every protected endpoint needs it. None of them defines it. Logging, observability, rate limiting, retries, audit, content safety, encryption in transit, and policy enforcement work the same way. They appear everywhere, but they do not encode the business knowledge of any specific application.

Domain logic is the opposite. The rule that a Premium customer can keep more saved searches than a Standard customer is domain logic. It depends on knowledge of what the application does. It cannot be implemented uniformly across the company because it is specific to this product. It belongs in the application that owns the product.

The line is not always obvious, and that is exactly where things end up in the wrong place. A three-question test sharpens the call.

The uniformity test: would the organization expect every team to implement this exactly the same way? Authentication, yes. Refund eligibility, no.

The domain knowledge test: does the implementation require knowing what the application does? Rate limiting, no. Approving a wire transfer, yes.

The blast radius test: when this behavior is wrong, is the failure systemic or local? An auth bypass shipping in fifty applications is a company-wide incident. A pricing bug in one product is a local incident.

When a behavior is uniform, requires no domain knowledge, and fails systemically when wrong, it is cross-cutting and belongs at a layer that enforces it once for everyone. When it goes the other way, it is domain logic and belongs in the application.

OWASP's API Security Top 10 cleanly illustrates the line. Function-level authorization, the check that asks whether this user is allowed to call this kind of endpoint at all, is cross-cutting. The same logic applies across every endpoint that requires the same role. Object-level authorization, the check that asks whether this user owns this specific record, is closer to domain logic. It depends on the business relationship between the caller and the object. OWASP's guidance is explicit that object-level authorization checks must live in the function that handles the object, because that is where the domain knowledge lives. The rule is not "everything goes in the gateway." It is "each cross-cutting concern goes at the right shared layer, and each domain check stays in the application that knows the domain."

The easiest way to think about this principle is cross-cutting concerns become dangerous when they are implemented locally but expected to behave globally. The risk to watch for is treating a cross-cutting concern as application code and depending on every team across the organization to implement it correctly, consistently, and in lockstep over time.

Why This Matters More in 2026 Than Ever

Six conditions, each more pronounced now than at any prior moment, push cross-cutting concerns out of application code and into a dedicated layer.

1. Scale

A typical enterprise deployment in 2026 runs hundreds to thousands of services, not dozens. Wiz Research's State of AI in the Cloud 2026 report found that 90 percent of analyzed cloud environments run self-hosted AI software on top of an existing microservices estate, with managed AI services in 81 percent and Model Context Protocol servers in roughly 80 percent. Each additional service is another place where authentication, rate limiting, observability, and policy enforcement could drift. The probability that fifty independently implemented copies of an authorization check are all correct at any given moment approaches zero. At a thousand copies, it is zero. Modern scale makes scattered cross-cutting code a statistical certainty of inconsistency.

2. Velocity

Continuous deployment pipelines push code to production multiple times a day, and AI traffic compounds the pressure with model swaps, prompt updates, and tool catalog changes happening on an even faster cadence. When cross-cutting policy lives inside the application, every deployment is a chance to silently disable it. A teammate refactors the auth middleware. A pull request "simplifies" the logging path. A library update changes the default rate-limit behavior. None of these are reliably caught at code review, because the team owns its own policy. They are caught, if at all, by an audit weeks later. A policy enforced at a separate layer, with its own change-management surface and its own deployment cadence, does not drift by accident.

3. Compliance Density

Regulatory frameworks now stack on top of each other in ways they did not in 2015. SOC 2 Type II, ISO 27001, HIPAA, PCI DSS, GDPR, the EU AI Act, the EU Cyber Resilience Act, and sector-specific regulations all require uniform enforcement of access control, data handling, and audit trails. Demonstrating compliance is straightforward when policy is expressed once and enforced uniformly. It is nearly impossible when policy is reimplemented in every application in every language by every team. Auditors do not accept "the policy is in the code" when there are 200 copies of the code.

4. Threat Economics

Attackers automate. The economics of exploitation in 2026 reward adversaries who can detect inconsistencies at machine speed. When one application enforces a rate limit, and another does not, automated reconnaissance finds the gap in minutes. When one MCP server validates tool inputs and another does not, the inconsistency is the attack surface. The IBM X-Force 2026 Threat Intelligence Index reports a 44 percent year-over-year increase in attacks that began with the exploitation of public-facing applications, citing missing authentication controls and AI-enabled vulnerability discovery as primary drivers. Wiz Research has separately observed malware that uses LLMs to dynamically generate commands and adapt execution at runtime. Defenders cannot afford the inconsistency that scattered cross-cutting code produces, because attackers will find it before the audit does.

5. Supply Chain Reality

Modern application code imports tens of thousands of dependencies transitively. Every imported library that handles a cross-cutting concern is a credential and privilege boundary that an attacker can pursue. The LiteLLM compromise in March 2026 made this concrete. When an LLM proxy lives as a Python library inside the application process, compromise of the library is compromise of every credential the application holds. Externalizing the logic to a separate, hardened service that the platform team operates contains the blast radius by design. It is the difference between one credential leak and forty thousand.

6. Polyglot Reality

The modern enterprise stack is not one language. A typical AI-adjacent deployment includes Python application code calling Go inference servers, Rust ingest pipelines, Node.js front ends, and Java backend services. Cross-cutting concerns implemented as Python decorators do not protect the Go service. Rate limiting implemented in a Java filter does not apply to the Rust pipeline. The only way to enforce consistent policy across a polyglot estate is to enforce it at the network boundary that all the languages share.

A seventh condition belongs alongside the sixth, and it is unique to this decade. Token-level cost governance, model fallback and failover, semantic caching, prompt safety, jailbreak detection, PII redaction, and tool-use authorization are cross-cutting concerns that did not exist a few years ago. They cut across applications the same way authentication does, but they also cut across model providers, agent runtimes, and tool catalogs. Centralizing them is not a generalization of an old principle. It is the only practical way to handle them.

These conditions are stronger now than ever. Each one independently argues for moving cross-cutting concerns out of the application runtime. Together, the case is overwhelming.

The API Gateway: A Pattern That Already Worked

The principle has already been proven for HTTP traffic. The API gateway pattern, formalized in the microservices canon and now standard in enterprise architecture, exists because the industry confronted these forces a decade ago at the network layer and concluded that authentication, rate limiting, request validation, observability, and policy enforcement belonged in a single layer rather than scattered across hundreds of services. Microsoft's API gateway guidance states the case directly: the gateway centralizes client-service interaction and performs cross-cutting tasks such as authentication, SSL termination, mutual TLS, and rate limiting. Microsoft's Gateway Offloading pattern, the related architectural reference, identifies certificate management, authentication, SSL termination, monitoring, protocol translation, and throttling as concerns that should be moved out of individual services and into the gateway tier.

The validation came from production telemetry, not from theory. Organizations that placed cross-cutting concerns at the gateway layer ran more reliably, audited more cleanly, recovered from incidents faster, and onboarded new services without re-implementing security primitives. Organizations that did not, paid the cost in breaches, audit findings, and engineering rework. After a decade of empirical settlement, the API gateway became default infrastructure: a tool that platform teams reach for early in any architecture conversation.

That conclusion still holds even though the traffic shape has changed. If anything, the traffic pattern has changed in ways that strengthen the case.

A New Dimension: The Caller May Be an Agent

The original API gateway pattern made an implicit assumption about the caller. The caller was a known type: a human user, a front-end application, an internal service, or a partner integration. Each of those has a stable identity, a request shape that maps cleanly to a documented API, and behavior that does not surprise the operator.

The AI stack changes the caller. An agent, executing a multi-step plan against a model and a tool catalog, may now be the entity deciding which API to call, which MCP tool to invoke, and which prompt to send to which model. The caller is not just sending a request. The caller is interpreting intent, selecting actions, and chaining tool invocations on the fly.

That changes what the cross-cutting concern surface needs to cover.

The controls that gate API access (rate limit, auth, schema validation) are no longer sufficient on their own, because the caller may invoke a sequence of perfectly authorized API calls that, taken together, exfiltrate data the user never had permission to expose, or take actions the user never explicitly approved. OWASP's LLM Top 10 names this risk explicitly as excessive agency: an agent with broad tool access can compose actions in ways no individual permission grant intended. Unbounded consumption is the same risk pointed at cost rather than data.

The caller's inputs are also different. Prompts, retrieved documents, tool descriptions, and tool results are content, not structured requests. Prompt injection therefore becomes a class of vulnerability that a traditional API gateway is not built to detect. OWASP's LLM Top 10 ranks prompt injection as the leading risk to LLM-integrated systems. The MCP specification's own security guidance warns about confused deputy risks, broad tokens, expanded blast radius, audit noise, and privilege chaining as protocol-level concerns that the implementer has to address.

The right response is not to abandon the API gateway. It is to extend the same principle to two new boundaries the AI stack has introduced: the conversation boundary, where prompts and completions live, and the tool-and-action boundary, where agents discover and invoke tools. Each new boundary is its own enforcement point. All three need to operate from the same identity, the same audit trail, and the same policy fabric.

Guardrails Are Not Governance

A useful distinction is becoming clearer in the academic and industry literature on agentic AI risk: prompt-level guardrails, model safety filters, and LLM-as-judge checks are useful signals, but they are not governance. They sit inside the model's probabilistic reasoning loop and depend on the model’s decision to comply with them. That is a layer of defense. It is not enforcement.

A recent paper by Szpruch, Sudjianto, Bhatti, and Ang on scalable runtime governance for agentic AI in financial services, drawing on Model Risk Management practice, makes the architectural case directly. Outcomes in agentic systems emerge from execution trajectories, rather than from a stable input-output mapping. The failures that matter (unsafe tool use, skipped approvals, privacy breaches, uncontrolled side effects) are process failures that surface during runtime, across the sequence of model calls, retrievals, tool invocations, and state transitions that make up the agent's actual work. Effective oversight requires runtime enforcement: governance-semantic telemetry, continuous authorization, temporal policy conformance checking, drift monitoring over trajectory health, and tier-based containment. Related work, including the MI9 framework from researchers at Barclays' Model Risk Management group and the "Policies on Paths" formulation in recent EU AI Act compliance research, converges on the same architectural conclusion.

The implication is direct. Governance for agentic AI cannot live inside the application's own prompt template, inside an in-process guardrail library, or inside the model's instruction tuning. It needs to live at the runtime boundary, where actions, tool calls, identity, approvals, and side effects pass through enforcement points that are independent of the model's probabilistic reasoning.

This is the cross-cutting concern argument restated in the language of regulated AI risk. The runtime governance that the literature is calling for needs a runtime enforcement point. If policy, telemetry, authorization, and containment are embedded separately inside every AI application, every MCP server, and every backend API, the organization has governance in theory but fragmentation in practice.

The AI Stack Is Re-Running the Anti-Pattern

Current AI infrastructure deployment patterns systematically violate the principle the API gateway pattern proved out.

LLM proxy libraries are imported into the application code. LiteLLM, the most widely deployed example, is typically loaded as a Python module running in the application process, holding API keys and routing requests across providers. The proxy lives in the runtime. Every application that wants centralized routing imports its own copy. Every application that imports it inherits the proxy's full security surface, including all the credentials it holds.

Guardrail libraries are imported into the application code. NVIDIA's NeMo Guardrails ships as a Python package that wraps a LangChain chain or any Runnable. The library does what it claims. The architectural placement, however, means that prompt safety lives within whichever team's application imports it. When five applications in the same enterprise each import their own guardrail library with their own configuration, the organization's policy footprint is five distinct rule sets drifting in five different directions.

MCP servers ship with ad-hoc per-server authentication. The Model Context Protocol, introduced by Anthropic in November 2024, has seen extraordinary adoption with over 5,000 community-developed servers within months of release. The current specification references OAuth 2.1, protected resource metadata, and authorization server discovery, but the protocol delegates the actual enforcement to each implementer. MCP standardizes how tools are discovered and invoked. It does not, on its own, standardize how they are governed. When every MCP server is responsible for its own auth, the result is dozens of different auth implementations across dozens of MCP servers, plus a tool-poisoning attack surface where a malicious tool description can hijack the model that is reading it.

Run the three-question test against any of these patterns. Is prompt safety expected to behave uniformly across applications? Yes. Does prompt safety require domain knowledge of each application's functionality? No, with rare exceptions. Is a prompt safety failure systemic when it ships in dozens of apps? Yes. Prompt safety is cross-cutting by every test, and yet the deployed pattern treats it as application code.

The same test applies to LLM cost governance, model fallback, content filtering, agent-tool authorization, and audit. They are cross-cutting by every test. The default deployment pattern places them in the runtime anyway. The result is the AI version of pre-gateway microservices sprawl: every application validates prompts differently, every agent handles tool permissions differently, every MCP server makes its own authorization assumptions, every model integration has different rate limits and retry logic and cost ceilings, every team produces a different audit trail, and security cannot answer the simple question, what did this agent see, decide, call, and change?

This is the pre-API-gateway pattern, applied to AI. It violates all six conditions outlined above. It does not scale. It drifts on every deployment. It does not produce uniform compliance evidence. It hands attackers exploitable inconsistencies. It widens the supply chain’s attack surface. It does not work in polyglot stacks. And it cannot solve the AI-specific concerns it was deployed to solve, because each application solves them differently.

The Case Studies Are Already Public

The cost is not hypothetical.

In March 2026, attackers compromised the LiteLLM Python package. Two malicious versions, 1.82.7 and 1.82.8, were published to PyPI on March 24 after the maintainer's publishing token was stolen through an upstream supply chain compromise of the Trivy security scanner. Wiz Research observed LiteLLM running in approximately 36 percent of the cloud environments it analyzed. The package is downloaded more than three million times per day, and FutureSearch's analysis of BigQuery PyPI download logs found that the malicious versions were installed roughly 47,000 times before PyPI quarantined the package. Because the proxy ran inside the application process, the malicious payload had access to environment variables, API keys, cloud credentials for AWS, GCP, and Azure, Kubernetes configs, CI/CD secrets, and SSH keys. The investigation, based on reports by Snyk, Sonatype, Datadog Security Labs, and Trend Micro, traced the campaign to a group tracked as TeamPCP. Datadog Security Labs and other responders advised that any host or CI job that installed the affected versions should be treated as a full credential exposure event, not a routine package update gone wrong.

Tool poisoning is the MCP equivalent. First published by Invariant Labs and now identified in the emerging OWASP MCP Top 10 framework, currently in beta, tool poisoning exploits the fact that MCP tool descriptions are injected directly into the model context. A tool that appears benign by name can carry hidden instructions in its description that the model will follow. Academic threat modeling using STRIDE and DREAD identifies tool poisoning as the most prevalent and impactful client-side vulnerability in MCP today. The defense is not a smarter individual MCP server. It is an enforcement layer that inspects and authorizes every tool call before the model sees it.

The scale of exposure is no longer experimental. Wiz's 2026 report found that MCP servers were running in roughly 80% of analyzed cloud environments, with 5%of those environments running at least one internet-facing MCP server. Self-hosted AI agent technology was found in 57% of organizations. The footprint expanded faster than any infrastructure standard in recent enterprise computing memory, largely without architectural review.

Reliability data points the same way. Datadog's analysis of customer LLM call traces found that in February 2026, 5% of all LLM call spans returned errors, and roughly 60% of those errors were rate-limit failures from upstream providers. In March, rate-limit errors alone accounted for nearly 8.4 million failed spans. When fallback, retry, caching, and budget enforcement live in each application's code, each application owns its own resilience problem, and each solves it differently. Centralizing that logic, exactly as the API gateway pattern centralizes HTTP rate limiting, is the obvious response.

The Triple Gate Pattern: One Control Plane, Three Trust Boundaries

AI traffic in 2026 crosses three distinct trust boundaries, each with its own cross-cutting concerns.

The conversation boundary, where prompts and completions move between users and models, is the AI Gateway's domain. Prompt injection defense, jailbreak detection, topic control, PII redaction, semantic caching, model routing, token-level cost control, model failover, and model observability all belong here. Every one of these passes the three-question test for cross-cutting.

The tool-and-action boundary, where agents discover MCP tools and invoke them with parameters, is the MCP Gateway's domain. Task-based access control, tool filtering, parameter validation, on-behalf-of token delegation, session routing, and tool-level audit belong here. Tool poisoning defense lives here as well, because the gateway is the only layer in the chain with full visibility into the tool description before the model sees it.

The backend-and-system boundary, where the result of all that activity becomes an actual API call into a system of record, is the API Gateway's domain. Authentication, function-level authorization, schema validation, rate limiting, WAF, and lifecycle governance all continue to belong here, exactly as they have for the past decade.

These three boundaries are the Triple Gate Pattern. They need to operate as one control plane, not three separate systems.

This is not the same as putting everything in the gateway. Each gate enforces the cross-cutting policy envelope at its trust boundary. The applications behind the gates still own their domain decisions: business rules, workflow logic, and the object-level authorization checks that depend on the relationship between the caller and the specific data being accessed. The gateway should enforce the shared runtime concerns. The application still owns the domain-specific ones. That distinction is what keeps the pattern useful rather than overreaching.

When each gate is implemented as application runtime code rather than infrastructure, the pattern fragments. The API gateway is in the infrastructure layer, where it has always been. The AI gateway is a library inside one team's Python service. The MCP gateway is a sidecar pattern built by another team last quarter, with different policy semantics and a different config language. The three gates do not see each other. The audit trail does not connect. A request that enters through the API gateway as an authenticated user X can emerge as a privileged service account at the MCP gateway because the application in between handled identity its own way. The cost-governance policy at the AI gateway is unaware of the rate-limit policy at the API gateway, because they are different systems run by different teams on different release cadences.

The compounding effect is operational. An LLM proxy sees the model interaction but not the agent actions that follow. An MCP proxy sees tool calls but not the LLM conversation that triggered them. An API gateway sees the inbound user request but loses sight of what the agent did downstream. Each layer knows part of the story. No one knows the full chain. When an incident occurs, the investigation has to stitch together logs from three systems with three different identity models, three different policy languages, and three different teams to find out what happened.

What security and platform teams actually need: one control plane, one declarative policy expressing what is allowed at each gate, one identity propagated across all three, one audit trail covering the full request chain, and one operational surface for incident response. That is the case for treating AI Gateway and MCP Gateway as infrastructure-layer concerns rather than application-layer libraries. The same case that the industry already accepted for the API gateway applies consistently to the new traffic types.

A Few Points That Deserve Their Own Spotlight

Supply chain placement is part of the cross-cutting concern surface. The LiteLLM incident is widely framed as a supply chain attack, which it was. It is also, more fundamentally, an architectural placement failure. A cross-cutting concern that had to be implemented somewhere was implemented inside every application's process space. The supply-chain attack succeeded because the proxy lived in the runtime. An infrastructure-layer gateway, deployed as a single hardened service that the platform team operates, has a fundamentally different attack surface than a Python library imported by every application team in the company. Both can be compromised. The blast radius is not the same.

Identity has to flow through the chain. An end user authenticates at the API gateway. The request hits an agent. The agent invokes an MCP tool. The tool calls a backend API. If the same identity does not flow end-to-end through this chain, the system either has to grant the agent broad service-account privileges, which violates least privilege, or invent application-level identity translation, which fragments yet again. Token delegation, on-behalf-of flows, and task-based access control at the gate layer are the mechanisms that keep this chain coherent. They cannot be implemented well as application code, because they require a privileged position in the request path.

Memory safety and language choice matter at the gate. CISA's January 2026 memory safety roadmap deadline, NSA and FBI guidance, and the EU Cyber Resilience Act all push critical infrastructure toward the adoption of memory-safe languages. The control plane sitting in front of every API, every LLM call, and every MCP tool invocation is critical infrastructure by any reasonable definition. The gate cannot be a place where C and C++ vulnerabilities still live.

Where to Go From Here

Cross-cutting concerns belong in a layer where they can be expressed once and enforced consistently. That is true today, in 2026, for the same reasons that made it true for HTTP traffic a decade ago, plus several reasons that did not exist then: AI-specific concerns, agent-driven callers, faster threat economics, and a much larger supply chain blast radius.

The API gateway pattern absorbed this lesson at the HTTP layer over the past decade. Production telemetry validated it. The pattern is now standard architecture, applied without much debate, because the costs of the alternative were paid out in public over many incidents.

The AI stack is now in the position the API stack was a decade ago. The architectural pressures are the same, the early warning signs are already on the public record, and the lessons are available to read. The LiteLLM compromise, tool poisoning attacks against MCP, the scattered guardrail-library deployment pattern, and the fragmented identity flow across LLM and agent traffic all point in the same direction. The Triple Gate Pattern, with API Gateway, AI Gateway, and MCP Gateway operating as a unified infrastructure-layer control plane, is the architecturally consistent response.

Cross-cutting concerns become dangerous when they are implemented locally but expected to behave globally. The teams that handle the AI infrastructure transition well will not be the ones with the most guardrails scattered across the most applications. They will be the ones whose runtime architecture governs every request, every prompt, every tool call, and every API behind it from a single, coherent control plane.

The principle holds. Agentic AI is making it concrete: runtime governance needs runtime enforcement, and runtime enforcement needs gates. The choice for teams building AI infrastructure in 2026 is which costs to absorb: the small, planned cost of getting the architecture right at the start, or the much higher costs of the API gateway era already documented when the same principle was learned in production. The data, case studies, and deployment patterns suggest that choice is closer at hand than it might feel.

Frequently Asked Questions

What is a cross-cutting concern, and how do I know if something qualifies?

A cross-cutting concern is behavior that needs to apply consistently across many parts of a system but does not belong to any one of them. Three tests identify it cleanly. First, the uniformity test: would the organization expect every team to implement this exactly the same way? Second, the domain knowledge test: does the implementation require knowing what the application does? Third, the blast radius test: when this behavior is wrong, is the failure systemic or local? If the answer is uniform, requires no domain knowledge, and fails systemically when wrong, it is cross-cutting and belongs at a shared enforcement layer, not inside each application.

Isn't a guardrail library good enough for prompt safety?

Guardrail libraries do what they claim at the function level. The architectural problem is placement. When five teams each import their own guardrail library with their own configuration, the organization has five policy rule sets drifting in five different directions. More fundamentally, a guardrail that lives inside the model's probabilistic reasoning loop depends on the model's decision to comply with it. That is a layer of defense. It is not enforcement. Enforcement requires an independent runtime boundary that the model cannot reason its way around.

What made the LiteLLM compromise so damaging, and could better security hygiene have prevented it?

Better hygiene at the package level would have helped at the margins, but the root cause was architectural. The proxy ran inside the application process, which meant that compromising one package version gave the attacker access to every credential the application held (e.g., API keys, cloud credentials, Kubernetes configs, CI/CD secrets). An infrastructure-layer gateway deployed as a single hardened service has a fundamentally different attack surface. Both can be compromised, but the blast radius is not the same. The LiteLLM incident is a supply chain story on the surface and an architectural placement failure underneath.

We already have an API gateway. Does that cover our AI traffic?

It covers the backend-and-system boundary—i.e., authentication, function-level authorization, schema validation, rate limiting—exactly as it always has. It does not cover the two new boundaries the AI stack introduces. The conversation boundary, where prompts and completions move between users and models, needs prompt injection defense, jailbreak detection, PII redaction, semantic caching, model routing, and token-level cost control. The tool-and-action boundary, where agents discover and invoke MCP tools, needs task-based access control, tool filtering, parameter validation, and tool poisoning defense. An API gateway was not built to inspect content at either of those boundaries. Extending the same architectural principle to all three trust boundaries is what the Triple Gate Pattern addresses.

How does identity work across the full agent request chain?

An end user authenticates at the API gateway. The request hits an agent. The agent invokes an MCP tool. The tool calls a backend API. If the same identity does not flow end-to-end through that chain, the system either grants the agent broad service-account privileges, which violates least privilege, or invents application-level identity translation, which fragments governance again. Token delegation, on-behalf-of flows, and task-based access control at the gate layer are the mechanisms that keep the chain coherent. They require a privileged position in the request path that application code cannot reliably hold.

What is tool poisoning and why can't individual MCP servers defend against it?

Tool poisoning exploits the fact that MCP tool descriptions are injected directly into the model context. A tool that appears benign by name can carry hidden instructions in its description that the model will follow, including redirecting actions, exfiltrating data, or chaining to unauthorized tool calls. Individual MCP servers cannot defend against this because the attack targets the model reading the description, not the server serving it. The only layer with full visibility into the tool description before the model sees it is a gateway sitting between the agent and the MCP server catalog. That is where tool poisoning defense has to live.

]]>

Traefik Proxy 3.7: Ingress NGINX, Without Ingress NGINX

Emile Vauge — Wed, 06 May 2026 15:33:00 GMT

Traefik Proxy 3.7, codenamed Langres, is the most consequential release we've shipped this year. It brings 85+ Ingress NGINX annotations now natively supported (and the experimental flag finally dropped), a brand-new certificates view in the dashboard, middlewares on services, full Gateway API v1.5 support, and retries and failover driven by HTTP status codes.

If you're running Ingress NGINX today, Traefik Proxy v3.7 is your drop-in replacement. Bring your existing manifests, enable the provider, and Traefik takes over. No YAML rewrite. No annotation conversion. No big-bang cutover. That's the headline. Let's get into the rest.

85+ Annotations, 90% of Snippets Use Cases, and It's All Production Ready

When we first introduced the Ingress NGINX provider in v3.5, we framed it as a migration foundation. Nine months, dozens of community PRs, and one official Ingress NGINX retirement announcement later, Traefik has become a true drop-in replacement.

In v3.7, the Ingress NGINX provider supports more than 85 of the most common annotations natively, covering more than 90% of real-world usage patterns we have observed across hundreds of production clusters. We didn't pick them by guessing. The open-source Ingress NGINX-migration tool (part of the Ultimate Migration Kit we shipped earlier this year) sends us anonymized usage data when users opt in (e.g., which annotations are deployed, in what combinations, and at what frequency). Every annotation we added in this release was prioritized from real production patterns across hundreds of clusters.

The breadth covers the entire long tail: authentication, session affinity, routing and redirects, proxy tuning, load balancing, rate limiting, canary deployments, custom headers, custom error pages, default backends, access control, and observability. The complete list of 85+ supported annotations and their behavioral notes lives in the NGINX Ingress Provider documentation.

The most strategically important addition is partial support for configuration-snippet, server-snippet, and auth-snippet (#12715, #12778 by @juliens). Snippets are the universal escape hatch of Ingress NGINX, used heavily in production: they let users embed raw NGINX directives that the controller pastes verbatim into the generated nginx.conf file before reloading the worker processes. Because the user-provided text becomes part of the runtime configuration without parsing or validation, a malicious or simply malformed value can inject arbitrary NGINX behavior into the controller. Snippets aren't the specific vector exploited by IngressNightmare (that was a different annotation), but they are the most extreme expression of the same architectural pattern: user input templated into the controller's runtime config.

The Kubernetes maintainers explicitly cited snippets as one of the "serious security flaws" that drove the retirement decision. But refusing snippets in Traefik would break too many real migrations, and copying Ingress NGINX's approach would recreate the vulnerability class. So we did neither: Traefik parses snippet content into structured input, maps it against a curated allowlist of common, safe directives (header manipulation, rewrites, variable interpolation for client IPs and URI components), and rejects everything else. Unlike Ingress NGINX, Traefik does not generate a raw config file at all, so there is nothing for the raw user text to be pasted into. The 90% of real-world snippet use cases just work, on a simpler and safer foundation.

Behind the scenes, what does this mean operationally? There is no YAML to rewrite. Bring your existing Ingress resources, enable the provider with a few lines of static configuration, and Traefik translates them into its internal object model securely:

providers:
  kubernetesIngressNginx:
    enabled: true

The full step-by-step path, including running the audit, deploying Traefik alongside your existing controller, and progressively shifting traffic, is in the Ingress NGINX to Traefik migration guide. We're also publishing a 3-part blog series alongside this release that walks through it end-to-end. The first post, Ingress NGINX Migration: Measure Twice, Cut Once, covers the audit step in depth: how to inventory your annotations with the migration tool and translate the NGINX ConfigMap into Traefik's configuration model. Parts 2 (install) and 3 (progressive cutover) follow weekly.

And after a year of community-driven maturation, the provider has graduated. The experimental flag is gone. The Ingress NGINX Provider is now a first-class citizen in Traefik. If you are still on Ingress NGINX, this is your drop-in replacement, validated by IBM Cloud, Nutanix, OVHcloud, SUSE, TIBCO, and a long list of organizations that have already made the switch.

TLS Certificates in the Dashboard: A Community Gift

Some of the best Traefik features come from outside our core team. v3.7 ships a brand-new Certificates menu in the dashboard (#12628), authored by @holomekc as an external contribution that we are particularly proud of.

For years, certificates were the invisible part of Traefik's runtime. You configured them, you deployed them, and you trusted that everything was working. The new dashboard view brings that visibility front and center: you can now see every TLS certificate in use, the domains it covers, its expiration, and where it is attached across HTTP and TCP routers.

This is exactly the kind of operator-quality-of-life feature that does not show up in benchmarks but transforms day-to-day operations. Renewal anxiety, expired certs in production, missing domains on a wildcard cert: the dashboard now answers all of these at a glance.

Middlewares on Services: A Cleaner Routing Model

Until now, middlewares in Traefik attached to routers or entryPoints. That mostly worked, but it produced a pattern that anyone running Traefik at scale knows well: when a service is exposed by ten routers, you end up duplicating the same middleware reference ten times. And when that middleware is a forwardAuth or a rate limiter that fundamentally belongs to the service, not the route, the duplication is a smell.

v3.7 fixes this with middlewares on services (#12544 by @juliens, co-authored with @lbenguigui). You can now attach middlewares directly to a service definition, and they apply to every router that targets that service.

http:
  services:
    api:
      loadBalancer:
        servers:
          - url: "http://api-backend:8080"
      middlewares:
        - rate-limit
        - auth

This change matters beyond the convenience of fewer YAML lines. It moves middleware semantics closer to where they belong conceptually. Authentication that protects a service should live on the service. Rate limiting that protects a backend's capacity should live on the backend. The router stays focused on what it is for: matching requests.

The same change also unlocks Gateway API filters on HTTP backends, a long-standing gap in our Gateway API support that this single PR closed.

Gateway API v1.5: Leading the Standard, Setting the Pace

Traefik v3.7 ships with full support for Gateway API v1.5.1 (#12768 by @mmatur), the latest revision of the Kubernetes networking standard. Traefik is part of the very small circle of gateways that keep pace with the Gateway API project with the regularity of a metronome: every minor release of the spec lands in a Traefik release within days. This release brings two notable additions on top of v1.5 baseline:

Multiple certificateRefs on Gateway listeners (#12590 by @mortennordbye). A listener can now reference multiple certificates, with Traefik selecting the correct one on each SNI. This eliminates a class of workarounds that involved either splitting a single logical Gateway into several, or fronting it with extra TLS termination.

Secret support for BackendTLSPolicy.caCertificateRefs (#12927 by @kevinpollet). End-to-end TLS to backend services with private CAs now works with secret-stored CA bundles, removing the last friction point for production-grade BackendTLSPolicy deployments.

Combined with the new middlewares-on-services semantics that bring Gateway API filters to HTTP backends, v3.7 closes the remaining feature gap between the Ingress provider and the Gateway API provider.

Retry and Failover Driven by Status Codes: Resilience That Resembles Reality

Traefik has had a Retry middleware and a Failover service since the early days. Both were good, but both shared the same blind spot: they only fired on connection failures and explicit health-check signals, not on what your application was actually telling them.

That changed in v3.7.

The Retry middleware now supports retries based on HTTP response status codes, a configurable per-attempt timeout, and an opt-in flag for non-idempotent methods (#12667 by @LBF38). If your backend occasionally returns a 502 under load, you can configure Traefik to absorb that and retry without the client noticing:

http:
  middlewares:
    smart-retry:
      retry:
        attempts: 3
        initialInterval: 100ms
        retryOn:
          statusCodes:
            - 502
            - 503
            - 504
        timeout: 2s

The Failover service now triggers on response status codes too (#12596 by @lbenguigui). Combined with the new service failover support in the TraefikService CRD (#12733 by @jspdown), you can declare blue-green and active-passive setups directly in Kubernetes without leaving Traefik's CRD model:

apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: api-failover
spec:
  failover:
    service: api-primary
    fallback: api-backup
    healthCheck: {}
    errors:
      status:
        - "500-504"

Real production failures rarely look like clean connection refusals. They look like 502s during a slow rollout, 503s when an autoscaler is catching up, 504s when a downstream dependency degrades. v3.7 lets Traefik respond to those signals natively, without external service meshes or sidecar acrobatics.

More Goodies

A non-exhaustive selection of the rest, because every release has a long tail of quality-of-life improvements that do not need their own section but absolutely deserve a mention:

Routing & Rules

Wildcard host in Host and HostSNI matchers (#12884 by @juliens)
Providers routing precedence configuration (#12895 by @juliens)
Per-Ingress entry points selection on Nginx Ingresses (#12727 by @ajacques)
ipAllowListStrategy option for allowlist-source-range and whitelist-source-range (#12932 by @mathieuherbert)

Middleware

New encodedCharacters middleware (#12555 by @gndz07)
authSignInURL in forward auth (#12293 by @kyounghunJang)
errorRequestHeaders in Errors middleware (#13034 by @gndz07)

TLS

Graceful handling of missing TLSStore secrets (#12522 by @david-garcia-garcia)
Fragmented TLS Client Hello support (#12787 by @rtribotte)
CertificateTimeout ACME configuration option (#12278 by @ceko)

Observability

Stdio access logs alongside OTLP logging (#12307 by @Mulgish)
OTel-conformant trace context attributes in access logs (#12801 by @mmatur)
Kubernetes Ingress logs fields (#12913 by @rtribotte)
File path support for metrics.influxdb2.token (#12458 by @barhun)

Web UI

Server weight in service detail view (#12325 by @murataslan1)
Dashboard name configuration (#12410 by @gndz07)

MISC

Global option to disable X-Forwarded-For appending (#12374 by @lbenguigui)
Cipher suites configuration with ServersTransport (#11965 by @NEwa-05)
maxResponseBodySize on HTTP provider (#12788 by @gndz07)
maxResponseBodySize on ForwardAuth (#12694 by @gndz07)
Knative v1.20.0 support (#12441 by @idurgakalyan)

Several security advisories were also addressed across the v3.7 cycle. The full list, with CVE IDs, severity, and patched versions, is on the Traefik Security Advisories page. Security continues to be a first-class part of every release.

A Toast to the Community

Traefik 3.7 is the work of a community that takes its time and gets it right. More than 80 contributors shipped code into this release, from first-time PR authors fixing typos in documentation to long-time maintainers landing entire subsystems. With 63k+ GitHub stars and billions of downloads behind us, every release is a community release, and v3.7 is more than most.

To everyone who opened an issue, reviewed a PR, sent us migration data through the audit tool, contributed an annotation, or shared their production war stories: thank you. You are why Traefik continues to set the standard for cloud-native ingress. Want to add yours to v3.8? traefik/traefik#12631 tracks the remaining Ingress NGINX annotations and is a great on-ramp for first-time contributors, but contributions are welcome across the entire codebase, on any topic.

Ready to upgrade?

Essential Resources and Next Steps

Download Traefik 3.7: GitHub Release | Docker Hub
Documentation: Complete Documentation
NGINX Migration: ingressnginxmigration.org | Migration Tool | NGINX Migration Guide
Community: GitHub Repository | Community Forum

]]>

Ingress NGINX Migration Guide (1 of 3): Measure Twice, Cut Once

Emile Vauge — Wed, 06 May 2026 15:32:08 GMT

Every migration story starts the same way: with a team staring at a production cluster, a long changelog of annotations, and a ConfigMap nobody has touched in three years. The gap between "we should migrate off Ingress NGINX" and "we know exactly what to do Monday morning" feels enormous. Opaque. Risky. Expensive.

It isn't.

In the vast majority of clusters I've seen since we started helping teams move off Ingress NGINX, 99% of the migration is mechanical: the annotations translate directly, the ConfigMap settings have an obvious Traefik equivalent, and the cutover is a matter of shifting traffic route by route. Only about 1% of a typical deployment actually requires human judgment. The hard part isn't doing the work. The hard part is knowing, before you start, which 1% deserves your attention.

That's what an audit is for. And it's the first thing you should do, before you install Traefik, before you touch a single Ingress resource, before you write a migration plan.

This is post 1 of a 3-part series on migrating from Ingress NGINX to Traefik, published alongside the Traefik v3.7 GA. We've broken the series into three steps:

Audit (this post): inventory your annotations and translate the NGINX ConfigMap.
Install: deploy Traefik with Helm next to your existing NGINX controller.
Migrate: cut over route by route, using IngressClass routing, L4 TLS pass-through, and a blue-green pattern.

The Real Cost of Skipping the Audit

The instinct when a project is under deadline pressure is to start moving. Install the new controller. Flip the first route. Fix things as they break. I understand the appeal. I've shipped enough infrastructure to know that "just start" works, sometimes.

This is not one of those times.

Ingress NGINX accumulates complexity in two places that are easy to overlook. The first is annotations on individual Ingress resources, which have grown to several hundred over the years. Most teams use a handful; a few teams use dozens; almost nobody has a complete inventory. The second is the controller ConfigMap, which quietly sets cluster-wide defaults for timeouts, TLS policy, body sizes, logging formats, and a long tail of NGINX-specific tuning that your application teams have never seen and would not recognize.

Skip the audit and these two surfaces bite you at the worst possible moment: after you've already put Traefik in the traffic path. A missing annotation mapping causes a silent behavior change for one team. A forgotten ConfigMap setting breaks TLS for a specific client. You roll back under pressure, lose trust, and the migration stalls for a quarter.

The audit is not a box-ticking exercise. It's the difference between a migration that takes an afternoon and one that takes a fiscal year.

Why Grep is Not an Audit

The obvious temptation is to kubectl get ingress -A -o yaml | grep annotations and call it a day. Don't. That approach fails on three fronts.

First, it tells you which annotations exist but not which ones actually matter. An annotation that Traefik supports natively is a non-event. An annotation that requires a new middleware is a small piece of work. An annotation with no direct equivalent is a conversation with the team that owns that Ingress. Grep cannot distinguish between these three, and treating them as equivalent is how migrations slip.

Second, it ignores the ConfigMap entirely. The ingress-nginx-controller ConfigMap is where your proxy-body-size, your ssl-ciphers, your hsts-max-age, and your use-proxy-protocol live. None of these appear on any Ingress resource. None of them will show up in a grep of your Kubernetes manifests. All of them affect production behavior.

Third, it produces a list, not a plan. A list of annotations is not actionable until someone sits down and maps each one to its Traefik equivalent, identifies gaps, and decides what to do about them. That work takes hours or days depending on cluster size. Worse, it gets redone every time a new engineer picks up the project.

This is exactly the kind of problem that tooling solves well and humans solve badly.

The Two-Step Audit

Here is the approach we recommend, and it has a deliberate order: Ingress annotations first, ConfigMap second. Both matter. Doing them in this sequence means you can scope the work and start planning before you touch the controller itself.

Step 1: Audit Your Ingress Annotations with the Migration Tool

We built ingress-nginx-migration, an open-source CLI that connects to your cluster and analyzes every Ingress resource in the namespaces you choose (or all of them, by default). It serves an interactive HTML report at http://localhost:8080.

Install and run it against your live cluster:

curl -sSL https://raw.githubusercontent.com/traefik/ingress-nginx-migration/main/scripts/install.sh | bash
ingress-nginx-migration --kubeconfig ~/.kube/config

The tool needs read-only access (list, get, watch) on ingresses and ingressclasses. Nothing else. It never modifies your cluster.

The report classifies your annotations into four buckets:

Vanilla: Ingresses that use no Ingress NGINX-specific annotations at all. They migrate as-is, on any recent Traefik version. For many clusters, this is a surprisingly large share.
Supported: Ingresses whose every nginx annotation has a Traefik equivalent. These also migrate as-is, with no YAML rewriting. The report tells you the minimum Traefik version each annotation requires (v3.6, v3.7, or Traefik Hub for ModSecurity), so you know exactly which version to deploy.
Unsupported: Ingresses that use at least one annotation Traefik does not yet support. The report lists those annotations and their frequency across your cluster, so you can quickly see whether you are looking at a couple of edge cases or a load-bearing pattern that needs a workaround.
Invalid: Annotations the tool does not recognize at all - neither in the supported list nor in the known-unsupported list. These are typically typos, custom extensions, or annotations specific to a fork. They get their own tab in the report so you can review them and clean up before the migration rather than during it.

A report generated by our migration assessment tool.

In a matter of minutes, the tool turns "hundreds of Ingresses in production" into a prioritized queue. You know exactly which resources migrate seamlessly today, what minimum Traefik version covers the bulk of your fleet, and which Ingresses need a closer look before you touch them.

If you choose to share your report with us via the "Share Report" button (anonymized by design - only counts and annotation names, never resource names, namespaces, or configurations), you also help us prioritize which annotations we add support for next. Many of the 80+ annotations now supported by Traefik v3.7 were prioritized from data that users sent us through exactly this channel.

Step 2: Translate your ConfigMap

Once you know the state of your Ingress annotations, turn to the other half: the controller ConfigMap. The full mapping table lives in the Ingress NGINX ConfigMap migration guide.

In Ingress NGINX, the ConfigMap is a single cluster-wide configuration layer. In Traefik, the same behavior is split across four surfaces, each with a clear purpose:

The providers.kubernetesIngressNGINX static configuration, for Ingress NGINX compatibility defaults like proxyBodySize, proxyConnectTimeout, and proxyBuffering.
EntryPoints, for listener behavior like HTTP-to-HTTPS redirection and PROXY protocol.
Dynamic tls.options and HTTP middlewares, for TLS policy, HSTS, and header behavior.
Traefik access log configuration, for request logging.

This split is intentional. Each surface has a single, well-defined responsibility, which makes Traefik's behavior easier to reason about per route. It also means that when you migrate, you rarely need to translate a ConfigMap setting to a single equivalent. You translate it to the right surface.

Start by exporting what you actually use today:

kubectl get configmap ingress-nginx-controller -n ingress-nginx -o yaml

Most keys fall into one of three categories. The first is the direct-mapping keys: proxy-connect-timeout, proxy-body-size, client-body-buffer-size, proxy-next-upstream, custom-http-errors, and similar. These have a one-to-one Traefik equivalent under providers.kubernetesIngressNGINX. The only gotcha is unit conversion: NGINX accepts values like 16k or 1m, Traefik expects raw bytes or integer seconds.

The second category is the cross-cutting keys: ssl-redirect, ssl-protocols, hsts, use-proxy-protocol. These don't belong on the provider; they belong on entryPoints, TLS options, or a headers middleware attached cluster-wide. This is where Traefik's per-surface model pays off: a headers middleware attached to an entryPoint gives you a cluster-wide HSTS default, but you can override it per-route without editing a central ConfigMap and rolling the whole controller.

The third category is the drop-it keys: worker-processes, worker-cpu-affinity, Lua shared dict settings, main-snippet, http-snippet, and their friends. These are NGINX internals. Traefik doesn't expose them because Traefik isn't NGINX. If you find one of these in your ConfigMap, translate the intent, not the directive. Ask yourself what behavior the team actually wanted, and configure that behavior in Traefik.

The complete, section-by-section translation table is in the ConfigMap migration step of the guide. It covers every ConfigMap key we've seen in production clusters and tells you exactly which Traefik surface takes over.

What the Audit Gives You

When you finish the two-step audit, you have three artifacts:

A categorized list of every Ingress in your cluster (vanilla, supported, unsupported, plus a separate tab for invalid/unknown annotations), with a per-annotation breakdown and the minimum Traefik version required to cover each one.
A line-by-line translation of your ConfigMap into Traefik static configuration, entryPoints, middlewares, and access logs.
A concrete scope, for the first time in the project. You can tell your team how many Ingresses are in the 99% versus the 1%. You can estimate the work. You can plan in sprints instead of quarters.

This is what I meant earlier when I said the hard part isn't doing the work. Migrating a vanilla or fully-supported Ingress is a kubectl command. Translating a direct-mapping ConfigMap key is a YAML change. The reason migrations feel impossible is not that the work is hard. It's that teams don't know, until they start, how much of the work is hard and how much isn't.

An audit tells you.

What Comes Next?

This is the first post in a series covering the full migration from Ingress NGINX to Traefik alongside our v3.7 GA. The next post will walk through installing Traefik with Helm next to your existing NGINX controller, in a configuration that serves the same Ingress resources in parallel so you can validate behavior without moving any traffic. The third will cover the progressive cutover: route by route, using IngressClass routing, L4 TLS pass-through, and a blue-green pattern that lets you roll back at any point with a single change.

For now, run the audit. You don't need v3.7 installed. You don't need to commit to a migration plan. You just need to know where you stand. Fifteen minutes from now, you will.

Resources

Migration website: ingressnginxmigration.org
Migration tool: github.com/traefik/ingress-nginx-migration
Migration guide: doc.traefik.io/traefik/migrate/nginx-to-traefik (and the ConfigMap migration section)
Annotation reference: Traefik NGINX Ingress Provider docs
Community forum: community.traefik.io

]]>

The CISO's Seven-Layer Cake: A Modern Defense Blueprint for the Frontier AI Era

Sudeep Goswami, Immánuel Fodor — Tue, 05 May 2026 19:06:28 GMT

Modern enterprise defense is built in functional layers. Each layer has its own capabilities and its own prevention mode. Identity is the foundation on which every other layer depends. Traffic governance is the tier where policy does its heaviest work. Security operations sit at the top, correlating signals from everything beneath.

Below is an illustration of this layered defense:

The reference architecture that follows is the field guide for how these layers are actually deployed. No vendors are named. This is deliberate. The goal is a capability framework that 1) survives vendor churn and acquisition cycles and 2) that CISOs can use to assess any product against a consistent standard.

Why This Blueprint Exists

The economics of enterprise defense have shifted. Frontier AI models have collapsed the cost and time required to discover vulnerabilities, chain them into exploit paths, and execute autonomous attack sequences. What used to take a skilled team weeks now runs in minutes. Defender playbooks built around human-speed response assume a luxury the attacker no longer grants.

The Capability Asymmetry is Now Live

What was recently a forward-looking argument is now a live deployment. Multiple frontier AI labs have released cyber-capable models under divergent access philosophies. One approach concentrates the most offensive-capable model in the hands of a small coalition of roughly a dozen partners, on the explicit position that broader distribution is too dangerous at present. Another approach scales defender-focused cyber models to thousands of enterprises, on the premise that defenders fall behind if strong tooling is withheld. Both positions can be correct simultaneously. Both produce the same structural consequence: offensive capability has already outpaced the rate at which defender architectures can absorb it.

The proliferation window is short. Industry estimates suggest the time for offensive frontier-class capabilities to diffuse from the initial holders to the broader threat landscape at six to eighteen months. Enterprises outside the initial coalitions have that long to mature their deterministic layers before the asymmetry reaches their threat model. This blueprint treats that window as the clock against which architectural decisions should be measured.

On the defender side of the same window, the largest identity and security platforms are converging on a single architectural pattern: bundling credential issuance with traffic and tool-invocation enforcement under one control plane. The convenience is real. The architectural cost is the subject of this blueprint.

On the offensive side, three structural changes follow from this shift.

The volume of attempted exploitation will rise by orders of magnitude, not percentages.
The window between vulnerability disclosure and active weaponization will compress toward zero.
Autonomous attack agents will operate at machine speed across the full kill chain, which means defender architectures that rely on human review at any critical point will be outpaced.

This blueprint answers that shift. It is organized around a small set of design principles, names the capabilities required at each layer of the defense stack, and describes how the layers work symbiotically to produce an architecture that is both economically rational and operationally defensible.

Design Principles

Principle One: Minimize the Attack Surface

Every component in the stack either reduces the attack surface or does not. Components that reduce surface earn their cost. Components that merely observe the surface are valuable but secondary. The primary architectural question at every layer is: what surface does this component remove, and at what cost per event?

Surface reduction happens in three ways:

Policy-level reduction removes traffic, calls, or actions that violate declared rules before they reach anything of value.
Structural reduction removes entire classes of vulnerability through design choices (memory safety in the data path, least-privilege defaults, network segmentation).
Supply chain reduction removes the untrusted code, dependencies, and artifacts that expand the blast radius of any single compromise.

A surface that does not exist cannot be attacked. This is not a slogan. It is the only defense that scales against adversaries operating at machine speed.

Principle Two: Prefer Deterministic Prevention Over Behavioral Judgment

Where an attack class can be expressed as a structured policy decision, evaluated with certainty against structured inputs, it should be prevented deterministically. Where it cannot, behavioral approaches are necessary. Conflating these two modes is the single most common architectural error. Pushing as much prevention as possible into the deterministic mode is the single most consequential design choice in a modern defense architecture.

Deterministic prevention denies or allows based on rules that can be evaluated with certainty. The inputs are structured: protocol fields, identity claims, policy declarations, cryptographic signatures, declared schemas. The decision is a function of those inputs, not a probability. A request either carries a valid token with the required scope, or it does not. A container image either matches a signed, approved hash, or it does not. An agent either has the task-level authorization to call a given tool, or it does not.

Deterministic prevention has three properties that make it economically dominant where it applies. The false positive rate is near zero because the decision is not a judgment. The latency is microseconds because the evaluation is trivial. It requires no human in the loop because there is nothing to review. When an attack class can be reduced to a deterministic rule, preventing it at the wire costs effectively nothing per event.

The limitation is equally clear. Deterministic prevention cannot catch what its rules do not describe. Novel attacks, behavioral anomalies, and anything requiring contextual judgment fall outside its reach.

Behavioral approaches operate on patterns, statistical models, and machine learning classifiers trained on past attacker and defender behavior. They are necessary for the attack classes that cannot be reduced to rules: lateral movement, insider abuse, novel malware behavior, anomalous data access, and sophisticated social engineering. Their false-positive rates are non-zero by design, their latency is higher, and their efficacy depends on training data quality, model freshness, and telemetry coverage.

The two modes are complementary, not competing. Deterministic prevention handles the knowable and removes an enormous volume from the behavioral layer. Behavioral detection then operates on a smaller, cleaner signal where its higher cost per event is justified. An architecture that uses behavioral detection for problems that should have been prevented deterministically wastes analyst capacity on routine violations and starves the behavioral layer of the signal quality it needs to catch what actually matters.

Principle Three: Detect at Machine Speed What Could Not Be Prevented

No architecture prevents everything. Novel attacks, insider threats, supply chain compromises, and zero-day exploits will bypass preventive controls. For these, the measure of defense is time to detection and time to first response, both of which must operate at machine speed against machine-speed adversaries.

Detection is only as good as its input signal. An architecture that has minimized the attack surface through strong prevention, and that handles the knowable deterministically, produces detection telemetry with far higher signal-to-noise than one that relies solely on detection. Prevention makes detection work.

These three principles are interdependent. Minimization reduces the space the other two principles have to cover. Deterministic prevention removes whole attack classes from the behavioral layer and feeds a cleaner signal to what remains. Machine-speed detection closes the gap where prevention cannot reach.

The blueprint that follows is organized around making all three operate at their strongest.

The Seven Layers of Modern Defense

The blueprint below describes seven functional layers. Each layer is defined by its capabilities, not by the products that provide them. Some products span multiple layers. Some layers can be partially covered by other layers. The goal is coverage of every capability, not one product per layer.

This Modern Defense Blueprint, a field guide.

In the blueprint above, identity pervades every layer. The traffic governance plane is the policy fulcrum, unifying API, AI inference, and agent tool traffic under a single deterministic policy model. The learning loop at the base shows behavioral detections at Layers 5 and 7 being promoted into deterministic rules at Layers 1, 3, and 4. This is how the cake is actually assembled.

Layer 1: Identity and Access Fabric

Purpose. Establish and enforce who (or what) is acting, what they are authorized to do, and under what conditions. This is the foundation on which every other layer stands.

Core Capabilities.

Strong authentication with phishing-resistant factors
Single sign-on with centralized session control
Conditional access based on device posture, location, time, and behavioral signals
Workload and machine identity (TLS, SPIFFE, service accounts)
Non-human identity governance for agents and automated systems, including discovery, lifecycle, attribution, and revocation of agent identities at scale
Privileged access management with just-in-time elevation
Session anomaly detection

Prevention Mode. Primarily deterministic at the policy evaluation level. Behavioral for anomaly detection (impossible travel, session hijacking indicators).

Design Intent. Identity is the first and most consequential policy decision in any request, which is why it is Layer 1. Every downstream layer consumes identity as an input; none can operate coherently without it. An architecture that treats identity as a boolean (authenticated or not) rather than a rich policy input has ceded most of its authorization power before any other layer has a chance to act. Agents and workloads require first-class identity, not an afterthought.

Layer 2: Perimeter and Edge

Purpose. Reduce the volume and class of traffic that reaches internal systems. Establish the first boundary of trust in network terms.

Core Capabilities.

Volumetric attack absorption (DDoS mitigation, connection rate limits)
Protocol hygiene (TLS termination, cipher policy, protocol version enforcement)
DNS security (filtering, DNSSEC, tunneling detection)
Web filtering and category-based access control
Geographic and reputation-based denial
Secure enterprise browsing with isolation capabilities for untrusted content

Prevention Mode. Predominantly deterministic at the network level. Behavioral for threat intelligence overlays and anomaly detection.

Design Intent. Every request that can be denied here is a request that does not consume downstream capacity. This layer earns its cost through volume reduction more than sophistication. Perimeter is positioned as Layer 2 rather than Layer 1 because a modern zero trust architecture starts with identity; perimeter is a reinforcing boundary, not the primary one.

Layer 3: Traffic Governance Plane

Purpose. Enforce fine-grained policy on every request, call, and action that traverses the environment. This is the layer at which deterministic prevention produces its greatest leverage.

Core Capabilities.

API gateway functions: schema validation, authentication enforcement, scope-based authorization, rate limiting, quota enforcement, request/response transformation
Web application firewall with both signature and positive-model rules
AI and inference governance: model routing, prompt policy, token budgets, output filtering, inference-layer identity
Agent and tool invocation governance: task-based access control, tool-level authorization, transaction scoping, delegated authority flows, and Model Context Protocol enforcement as the emerging standard for agent-to-tool traffic
Observable policy decisions with structured telemetry for every evaluated request

Prevention Mode. Overwhelmingly deterministic. This is the layer at which the deterministic prevention thesis is strongest, because the inputs (protocol semantics, identity claims, declared policy) are all structured, and the decisions are binary.

Design Intent. Three properties make this layer structurally irreplaceable. It sits between every external caller and every protected workload, which makes it the enforcement point with the broadest reach. It operates on structured protocol data, enabling deterministic prevention at very low false-positive rates. It produces the richest per-request telemetry in the stack, which means its observability output is what enables effective downstream detection.

A critical architectural requirement at this layer is heterogeneity. Modern environments carry three distinct classes of traffic: application API calls, AI inference requests, and agent tool invocations. Each has its own policy model, identity semantics, and abuse patterns. Fragmenting these into three separate governance systems produces operational drift, policy inconsistency, and gaps at the boundaries. A unified policy plane across all three is a structural requirement, not a preference.

The same heterogeneity logic applies to identity. The traffic governance plane consumes identity claims from multiple providers across the modern enterprise: workforce identity systems, customer identity systems, machine and workload identity, and increasingly agentic identity issued through standards-based protocols. A traffic plane bound to a single identity provider is a fragmentation point, not a unification point. The enforcement layer must be neutral across identity sources by construction, because the issuance of identity and the enforcement of what that identity is permitted to do are different architectural responsibilities operating on different telemetry, at different cadences, and with different failure modes. Collapsing those responsibilities into a single control plane is consolidation, not coherence.

An additional non-negotiable is memory safety in the data path. Any component that sits in every request path and is written in a memory-unsafe language is a perpetual patch-cadence problem. The vulnerability deluge anticipated by frontier AI research has now been empirically demonstrated. Recent frontier model disclosures surfaced thousands of previously unknown zero-day vulnerabilities across foundational software, including flaws that had survived decades of human review. Memory-unsafe data path components are not a feature to be patched on cadence. They are a structural liability on a short clock.

Layer 4: Application and Supply Chain Integrity

Purpose. Ensure that the code, dependencies, and artifacts running in the environment are trusted by construction, not by hope.

Core Capabilities.

Secure software development lifecycle with enforced gates
Software bill of materials generation and policy-based admission control
Signed artifact enforcement at deployment
Hardened, minimal base images with continuous vulnerability remediation
Dependency provenance tracking
Secrets scanning in source, builds, and images
Static and dynamic application security testing integrated into CI/CD
Model and dataset provenance for AI workloads

Prevention Mode. Deterministic at the policy gate level (signed or not, on SBOM allowlist or not, scan-clean or not). Behavioral in the static analysis phase.

Design Intent. The inside-out attack pattern (a compromise propagated through the supply chain, ending inside the perimeter with full trust) is the most consequential shift in the threat model over the last two years. This layer exists to make that pattern structurally harder. Every artifact that enters the environment should carry cryptographic evidence of its provenance, and admission should be contingent on policy satisfaction, not reputation.

The compounding pressure on this layer comes from downstream flow. When frontier-scale discovery surfaces thousands of vulnerabilities in foundational operating systems, browsers, and widely used open-source libraries, those findings propagate into every enterprise software bill of materials as new critical advisories. A supply chain posture built on manual review and best-effort patching cannot process that volume. The admission gate must be deterministic, and the dependency set must be actively minimized.

Layer 5: Workload Runtime Protection

Purpose. Detect and contain attacks that have bypassed preventive controls and are operating within the environment.

Core Capabilities.

Endpoint detection and response with behavioral analytics
Cloud workload protection (container, VM, serverless)
Runtime network segmentation and microsegmentation
Kernel-level telemetry (eBPF or equivalent) for syscall and process behavior
Lateral movement detection
Memory-level attack detection (exploitation techniques, injection, escalation)
Container and Kubernetes-specific runtime controls

Prevention Mode. Predominantly behavioral. Some deterministic capability through policy (Pod Security, network policies, seccomp, capability dropping).

Design Intent. This is where behavioral prevention and detection do their heaviest work. The inputs are unstructured (process trees, syscall patterns, memory access, network flows). Deterministic rules exist, but cover a smaller fraction of the attack space than at Layer 3. Accepting this means funding this layer for what it does well (behavioral judgment) and not asking it to do the work that should have happened deterministically upstream.

Layer 6: Data Protection

Purpose. Protect data at rest, in transit, and in use, and prevent its unauthorized egress.

Core Capabilities.

Encryption at rest with managed key lifecycle
Encryption in transit with strong protocol policy
Data classification and labeling
Data loss prevention with channel coverage (email, web, endpoint, API)
Secrets management with rotation and ephemeral issuance
Database activity monitoring
Tokenization and format-preserving encryption for sensitive fields
Model output filtering for sensitive data exposure in AI contexts

Prevention Mode. Deterministic for encryption enforcement, access policy, and classification-based blocking. Behavioral for anomalous access patterns and egress detection.

Design Intent. Data is the ultimate objective of most attacks. This layer makes data valuable only where it should be valuable and visible only to those who should see it. Irreversibility is the key concept: a leaked key, an exfiltrated dataset, and an exposed model weight cannot be recovered. Preventive controls at this layer are therefore disproportionately valuable.

Layer 7: Security Operations and Response

Purpose. Correlate signals across layers, detect what individual layers cannot see in isolation, and respond at machine speed.

Core Capabilities.

Centralized telemetry ingestion from every layer
Correlation and detection engineering
Machine learning and behavioral analytics across combined signals
Threat intelligence integration
Orchestration and automated response (SOAR capabilities)
Case management and investigation tooling
Metrics instrumentation for time to detection and time to response
Threat hunting capabilities with full-fidelity data access

Prevention Mode. Primarily detection. Behavioral by nature of the problem space. Deterministic correlation rules exist, but are the floor, not the ceiling.

Design Intent. This layer succeeds or fails on signal quality. An architecture that feeds it noise produces analyst burnout and false confidence. An architecture that feeds it structured, policy-contextual telemetry from strong preventive layers produces detection at the speed the threat requires. AI-augmented security operations are no longer an aspirational capability. Defender-focused cyber models and agent governance platforms are now actively deployed at enterprise scale across major vendors.

That shift raises the ceiling on what this layer can do, but does not raise the floor. A behavioral analytics engine, whether classical or AI-driven, delivers value proportional to the quality of the signal it consumes. Time to detection and time to first response in single-digit minutes are the emerging standard, and that standard is only achievable when the upstream layers have done their job.

How the Layers Work Symbiotically

The seven layers are not independent. The value of the architecture lies in how it reinforces itself.

Prevention Feeds Detection

Every policy decision at Layers 1, 2, and 3 is also a telemetry event. A denied request generates a log line that enriches the SOC's understanding of attacker behavior. A granted request carries a structured context (identity, scope, policy decision) that makes Layer 7 correlation dramatically more effective. The prevention layers are not parallel to the detection layer; they are upstream tributaries feeding it.

Detection Informs Prevention

Behavioral detection at Layers 5 and 7 identifies new attack patterns that can then be encoded as deterministic rules at Layers 1 through 3. Every novel attack caught behaviorally should produce a question: can this now be prevented deterministically in the future? The architecture learns.

Identity Pervades Every Layer

The identity fabric at Layer 1 is not consulted once at login. It is the input to policy decisions at the traffic layer, the basis for workload permissions at the runtime layer, the access gate for data at Layer 6, and the pivot for correlation at Layer 7. An identity architecture that cannot service all of these consistently produces gaps at the seams. Identity is Layer 1 precisely because nothing downstream works without it.

Supply Chain Integrity Underwrites Runtime Assumptions

The behavioral detection at Layer 5 is calibrated against an assumed baseline of legitimate workload behavior. When supply chain integrity at Layer 4 is weak, the baseline is contaminated, and Layer 5 loses its ability to distinguish benign from malicious. Strong supply chain controls are not a parallel investment to runtime detection; they are its precondition.

The Traffic Governance Plane is the Policy Fulcrum

More policy decisions are made at this layer than anywhere else in the stack, because every request, call, and invocation passes through it. The quality of policy at this layer compounds. A well-governed traffic plane removes enormous volume from downstream layers, delivers structured context to the SOC, and handles entire attack classes deterministically. A poorly governed traffic plane pushes work downstream that downstream layers are not designed to handle efficiently.

Defender AI is Itself a Governed Workload

The AI-augmented SOC, the AI-driven AppSec platform, and any automated remediation agent introduced into the environment are not outside this architecture. They are subjects of it. A defender AI agent is a non-human identity at Layer 1, a traffic-generating client at Layer 3, and a software artifact with its own dependency set at Layer 4. Its training data provenance, invocation patterns, and authorized actions all produce telemetry at Layer 7. Organizations deploying defender AI without applying the same architectural controls to those agents inherit a new class of privileged insider risk. The blueprint applies to itself.

The Economic Case for This Blueprint

The case for this architecture rests on four economic realities.

1. Cost Per Event is Not Linear Across Layers

A policy denial at the traffic layer costs nothing effectively. A SOC alert costs analyst time. A contained incident costs an incident response engagement. An uncontained breach costs the full regulatory, reputational, and operational burden. Shifting events leftward in this cost curve is the single most impactful thing an architecture can do.

2. Irreversibility has Infinite Cost

Certain attack outcomes cannot be undone. Exfiltrated data, leaked keys, destructive agent actions, and poisoned model weights. For these outcomes, prevention is not a cost optimization; it is the only available defense. Architectures that under-fund prevention for outcomes in this category are accepting unbounded downside.

3. Detection has a Latency Floor

Human-in-the-loop detection cannot be compressed below certain thresholds. Automated detection can be faster, but it still operates in non-zero time. The current generation of defender-focused AI tooling closes part of that gap, but the floor does not disappear.

Adversaries with access to frontier discovery models operate below that floor by construction, because their decision cycle is set by inference cost rather than by analyst review. Deterministic prevention operates in microseconds. For the attack classes that can be expressed deterministically, prevention is the only mode that keeps pace with machine-speed adversaries, regardless of how capable the defender's own AI layer becomes.

4. Signal Quality Compounds

Strong upstream prevention produces a cleaner downstream signal. A clean signal enables faster and more accurate detection. Accurate detection justifies higher automation in response. Each layer's effectiveness multiplies through the one downstream. Weak upstream controls degrade every layer that follows.

A Maturity Model for Self-Assessment

In this maturity model, there are four stages of architectural coherence. Stage 1 is a single wall with trust assumed inside. Stage 2 layers multiple defenses that operate independently. Stage 3 shifts prevention deterministically to the wire and makes identity pervasive. Stage 4 closes the loop, with behavioral detections becoming new deterministic rules on a short cadence. The goal is not Stage 4 everywhere. The goal is knowing where the next investment produces the greatest marginal improvement.

A CISO using this blueprint can assess the current state against the following progression. The goal is not to be at the highest stage everywhere; it is to understand where the organization sits and where the next investment produces the greatest marginal improvement.

Stage 1: Perimeter-Centric

Defense is concentrated at the network edge. Identity is coarse. Inside the perimeter, trust is broadly assumed. Detection is signature-based. This stage is structurally unsound against modern threat models and is increasingly rare in mature organizations.

Stage 2: Defense in Depth

Multiple layers exist but operate independently. Identity is federated but not pervasive. Detection is present but overwhelmed by volume. This is the most common stage. It survives but does not scale against frontier AI-enabled threats.

Stage 3: Deterministic First

Prevention has been deliberately shifted leftward into layers where it can operate deterministically. The traffic governance plane enforces fine-grained policy. Identity is pervasive across humans, workloads, and agents. Behavioral detection operates on a cleaner signal because routine violations are prevented upstream. Time to detection is measured in minutes.

Stage 4: Symbiotic

The layers operate as a single system. Telemetry flows consistently. Detection findings become new preventive rules on a short cadence. Supply chain integrity underwrites runtime assumptions. Identity governs every policy decision. Response is largely automated for the volume of routine events, freeing analysts for the novel and complex. Time to detection and response is measured in single-digit minutes. The organization can defend against machine-speed adversaries because its own architecture operates at machine speed.

Stage 4 is achievable. It requires discipline in capability selection, commitment to deterministic prevention wherever viable, and architectural coherence across layers. Most organizations pursuing this trajectory find that the constraint is not technology but integration quality, which is why capability coherence matters more than any individual product choice.

Implementation Sequence

In the implementation sequence above, there are five investments in priority order. Identity is the foundation. Traffic governance is the fulcrum where deterministic prevention produces its greatest leverage. Supply chain integrity closes the inside-out attack path. SOC modernization comes fourth, not first, because an AI-augmented SOC built on weak upstream layers is a fast analyst drowning in the same noise. Step five is a loop, not a terminus.

For an organization assessing where to invest next, the recommended sequence is:

First, identity (Layer 1). No downstream layer works well if identity is incoherent. Pervasive, strong, conditional identity for humans, workloads, and agents is the prerequisite for everything else. The numbering of this blueprint places identity at Layer 1 for precisely this reason: it is the foundation on which all other investment depends.

Second, the traffic governance plane (Layer 3). This is where the largest volume of deterministic prevention happens and where the richest detection telemetry is produced. Unifying API, AI, and agent traffic governance under a single policy model is a strategic investment with compounding returns. Perimeter (Layer 2) is typically mature in existing organizations, which is why new investment is usually better directed at Layer 3.

Third, supply chain integrity (Layer 4). The inside-out attack pattern is the fastest-growing threat class. Investment here produces disproportionate risk reduction because it closes entire attack paths structurally.

Fourth, SOC modernization (Layer 7). An AI-driven SOC operating on the cleaner signal produced by the first three investments reaches its potential. This is a live purchasing decision for most CISOs today, because defender AI platforms are being actively marketed to SOC teams right now. The sequence matters. An AI-augmented SOC built on top of weak identity, a fragmented traffic governance plane, and a porous supply chain produces a very fast analyst drowning in the same noise. Upstream first, then SOC, in that order.

Fifth, continuous refinement of all layers. The architecture is never complete. New detection findings become new preventive rules. New threat intelligence updates every layer. Maturity is a direction, not a destination.

Closing Principle

The shift to frontier AI threat models rewards architectural coherence over point-product sophistication. An organization with a coherent seven-layer architecture operating deterministically where possible, and behaviorally where necessary, with clean signal flow between layers, will outperform one with superior individual products and weaker integration every time.

Coherence is not the same as consolidation. The layers are deliberately distinct because they perform different functions on different inputs at different cadences, and collapsing them into a single product surface erodes the structural defense in depth that makes the architecture work. The system that issues credentials should not be the only system enforcing what those credentials can do. The system that hosts a workload should not be the only system observing it. Architectural coherence is achieved through clean interfaces between distinct layers, not through bundling those layers into a single control plane.

The measure of a modern defense is not how many tools it contains. It is how small its attack surface has become, how quickly it detects what did get through, and how little human intervention either of those requires. That is the standard the frontier AI era imposes on defenders. This blueprint is how to meet it.

]]>

Breaking the Monolith with Nutanix Cloud Infrastructure and Traefik Hub

Steve Eyler, Zaid Albirawi — Thu, 23 Apr 2026 22:17:30 GMT

You have a monolith running on a VM. It works. It has worked for years. But the pressure to modernize is real: new features take too long, scaling is expensive, and the teams building on it are stepping on each other's toes.

The instinct is to rewrite everything. Stand up a Kubernetes cluster, containerize the whole thing, and flip the switch. But if you have been in this industry long enough, you know how that story usually ends. Months of work. A missed deadline. A rollback. A very quiet retro.

There is a better way. And if you are running Nutanix, you already have the infrastructure to do it.

The Real Blocker Is Not Containers

Most modernization efforts stall before a single microservice is shipped. The blocker is not Kubernetes. It is not containerization. It is the gap between the VM world, where the monolith lives, and the container world where you want to go.

You can’t extract a service if you can’t route traffic to it. You can’t validate a new service if you can’t observe it handling real requests. And you can’t do any of this safely if authentication is applied differently at every layer.

The first move is not to break anything apart. The first move is to unify your traffic layer.

With a unified traffic layer, you get a single control point across all environments:

Why Nutanix Changes the Equation

Nutanix is unique here because it runs both compute substrates natively. Your monolith sits on AHV virtual machines. Nutanix Kubernetes Platform (NKP) runs containers right next to them on the same compute cluster, same storage, same network fabric. You don’t need to build a bridge between two clouds or two data centers. The bridge already exists at the infrastructure level.

What you need is an application-layer bridge: something that understands HTTP, can discover services across both VMs and Kubernetes, and gives you the traffic controls to move between them safely.

That is where Traefik Hub comes in.

Step One: Deploy Your Unified Ingress Across VMs and Kubernetes

Before you think about microservices, you need a single entry point for all traffic. Today, your monolith probably handles its own ingress, or sits behind a legacy load balancer that knows nothing about your containers.

The first step is to deploy Traefik Hub in two places:

On NKP, upgrade to Traefik Hub. NKP already ships with a Kommander-managed Traefik instance for platform ingress, so the technology is familiar. You install Traefik Hub via Helm alongside it, using ingress classes to separate responsibilities. The Kommander instance keeps its existing ingress class for platform traffic, while Traefik Hub uses a dedicated class for application traffic. The NKP instance becomes your parent gateway, the central routing brain for all traffic.

On AHV, install Traefik Hub directly on a VM in the same VPC or subnet as your monolith. No Kubernetes required. Traefik Hub runs as a native binary on Linux, provisioned via cloud-init when you spin up the VM in Prism Central. It connects to the Nutanix Prism Central API and automatically discovers your VMs using category tags. Tag a VM with TraefikServiceName: orders and Traefik picks it up within seconds.

Now, let's connect them. Traefik Hub's multi-cluster capability uses a parent-child architecture where the NKP instance acts as the parent and the AHV instance registers as a child. The child exposes an uplink entry point on a dedicated port, and the parent automatically discovers all advertised workloads. Communication between them is secured with mutual TLS.

The result: a unified gateway on NKP that can route traffic to containers running in Kubernetes and to services running on VMs. One control plane. One set of routing rules. One place to enforce policy. Your monolith is now accessible through the same gateway that will serve your future microservices.

Lock Down the Network with Nutanix’s Flow Microsegmentation

Unified ingress solves the north-south problem: how traffic gets in. But a multi-cluster architecture also creates new network communication paths that need to be secured. There are two distinct concerns here, and Flow Network Security addresses both.

First: securing the Traefik-to-Traefik multi-cluster communication. The parent gateway on NKP communicates with child instances on AHV VMs and other NKP clusters via mTLS on the uplink port (9443 in this example). At the application layer, mutual TLS ensures that only the parent can connect to the child and vice versa. But at the network layer, you should also lock this down with Flow.

First, we need to define an Entity Group that will reference the Traefik Hub pods inside the NKP clusters, and tag the Traefik Hub VMs with AppType: traefik-hub category. Then we can create an application policy that allows traffic on port 9443 only between VMs tagged with AppType: traefik-hub and pods selected with the Traefik Entity Group. Everything else trying to reach that port gets dropped. This ensures that even if something else on the network tries to probe the uplink endpoint, it never gets through.

Second: securing east-west traffic between workloads. This is where Flow microsegmentation shines. Flow is a distributed, stateful firewall built into the AHV hypervisor that enforces policies at the VM vNIC level, before a packet ever reaches another workload. When using NKP, Flow extends the same policy model to Kubernetes pods and services. The result is consistent network security enforcement across both compute substrates, managed from a single Prism Central console.

Traefik Hub Nutanix Prism Central provider uses Nutanix Categories for service discovery in an AHV environment. Tag your monolith VMs with categories like AppTier: backend, and group your containerized microservices into Entity Groups based on their cluster, namespace and labels. Flow can then define exactly which workloads can talk to each other, on which ports, and in which direction, regardless of whether they run on VMs or Kubernetes.

Together, these two layers create defense-in-depth. Traefik Hub handles application-level concerns at Layer 7—e.g., who is making the request, whether they are authenticated, whether they should be rate-limited, and which backend version should receive the traffic. Flow handles the network-level concerns at Layers 3 and 4—e.g., can this VM talk to that pod at all, on which ports, and in which direction.

In practice, this means you can tighten security as you decompose. When the orders service moves from the monolith to its own container on NKP, you create a Flow policy that allows the new orders pod to reach the database VM on port 5432, and nothing else. The monolith keeps its existing access. The new microservice gets only what it needs. If the orders pod is ever compromised, the blast radius is contained to what Flow explicitly permits.

Flow also supports quarantine policies. If your security tooling detects anomalous behavior from a workload, Flow can instantly isolate it, cutting off all network access while allowing forensic investigation traffic. This is especially valuable during a migration, where new services are being validated, and the risk profile is temporarily elevated.Finally, the same security model Flow applies to VMs extends directly to NKP. You define Entity Groups to identify which pods should receive which traffic, then reuse your existing VM rules, swapping the category tags that identified the monolith for the Entity Groups covering its containerized replacement. Both rule sets can run in tandem during the cutover, so the monolith and its microservice counterparts stay protected under a single, consistent policy throughout the migration.

Step Two: Establish Observability Before You Touch Anything

You now have a gateway in front of everything. Before you extract a single service, start leveraging it.

Traefik Hub emits OpenTelemetry metrics, traces, and logs out of the box. Point Traefik Hub's OpenTelemetry exporter at the your telemetry suite, and you will immediately get visibility into every request flowing through the gateway.

On the network side, Nutanix Flow also gives you something most teams wish they had before starting a decomposition: a visual map of actual traffic patterns across your monolith. Flow's traffic visualization shows you which VMs are talking to which other VMs, how often, on which ports, and in which direction. This is invaluable for understanding data access patterns. If you can see that the orders module only talks to the orders database and the notification queue, you have evidence that it is a good extraction candidate. If you can see that the pricing module reaches into six different backends, you know to leave it alone for now.

This is not optional. This is the foundation on which everything else depends. Between Traefik Hub's request-level telemetry and Flow's network-level traffic visualization, you need to answer questions like:

Which endpoints get the most traffic?
What are the current latency baselines?
Which parts of the monolith are already isolated in terms of data access?
Which VMs communicate with which backends, and how tightly coupled are they?
Where are the error hotspots?

The answers to these questions tell you what to extract first and, just as importantly, what to leave alone. Many teams make the mistake of extracting the service they think is most important. The right choice is usually the service that is most independent, has the clearest boundaries, and causes the least disruption if something goes wrong. Flow's traffic map gives you hard evidence for that decision instead of guesswork.

Observability also gives you the before picture. When you start routing traffic to a new microservice, you need to compare its behavior against the monolith's baseline. Without that baseline, you are flying blind.

Step Three: Centralize Authentication

Before you split traffic between the monolith and new services, you need a consistent authentication layer. The goal is simple: one place where identity is verified, so every service behind the gateway can trust who is making the request.

If your organization already uses an OIDC-compatible identity provider (Keycloak, Auth0, Okta, Entra ID, or similar), Traefik Hub makes this straightforward. Configure the gateway to validate JWT tokens at the edge, so that every request that reaches your services is already authenticated. If you are not there yet, that is perfectly fine. Traefik Hub also supports basic auth, API keys, and custom header-based authentication. You can start with whatever your monolith uses today and evolve toward centralized identity at your own pace. The important thing is that the gateway becomes the single enforcement point, regardless of the mechanism behind it.

Here is the pragmatic part: it is perfectly fine to validate at the gateway and still have the monolith validate internally. You do not need to rip out the monolith's auth layer on day one. Configure Traefik to forward the validated credentials downstream so the monolith can continue to verify them as it always has. Over time, as you gain confidence, services behind the gateway can rely on the gateway's validation and skip their own.

This approach means you do not need to change the monolith's auth logic to get started. The gateway adds a layer of centralized enforcement without requiring the monolith to change.

Step Four: Decide What to Extract

With the foundation in place, the real work begins: deciding what to decompose, in what order, and why.

How to Pick the First Service

Not every service is a good candidate for early extraction. Here is what to consider:

Domain Independence
Look for functionality with clear boundaries around data ownership and API surface. An orders service that reads from its own tables and exposes a clean REST API is a better first candidate than a pricing engine that reaches into six different database schemas.

Team Ownership The team that owns the domain should own the extraction. If you are extracting the notifications service, the team that builds and maintains notifications should drive it. They know the edge cases, the failure modes, and the quirks that no architecture diagram captures.

Risk ToleranceStart with something that has a fallback. If the new notifications microservice fails, can you queue the notifications and retry? That is a much safer first extraction than the payment service, where failure means lost revenue.

Data CouplingIf the candidate service shares a database with other parts of the monolith, you have a harder extraction ahead of you. Shared databases are the number one reason monolith decompositions stall. Either decouple the data first, or pick a service that already has its own data store. This is where Flow's traffic visualization pays off again.

Instead of relying on architecture diagrams that may be out of date, you can look at the actual network communication graph and see exactly which components access which data stores. A module that Flow shows talking to a single database is a clean extraction target. A module that Flow shows reaching into multiple different backends is one you should consider deferring.

Traffic PatternsYour observability data shows the request volume, latency profile, and error rate of each endpoint. Pick something with moderate traffic for your first extraction. Too low and you will not learn anything. Too high and a mistake impacts too many users.

What the Team Should Think About

Beyond picking the service, the team needs to align on a few things before code gets written:

API contracts. Define the API surface of the new service before you build it. The monolith already exposes these endpoints. The new service needs to be a drop-in replacement from the client's perspective.
Data migration strategy. If the new service needs its own data store, how do you migrate the data? Do you do it upfront, or run dual writes for a period?
Error handling and fallback. What happens when the new service is down? Does the gateway fall back to the monolith? Does it return a cached response? Does it fail loudly? Decide this before you ship.
Deployment independence. The whole point of microservices is that teams can ship independently. If deploying the new service still requires coordinating with the monolith team, you have not actually decomposed anything.

Step Five: Implement the Strangler Fig Pattern

Now the migration starts. The pattern is called the Strangler Fig: new functionality grows around the old system, gradually replacing it piece by piece, until the original is no longer needed.

In a Nutanix environment with Traefik Hub, this pattern is controlled entirely through routing. You never need to change DNS records, update load balancer configs, or redeploy the monolith. Every change is a routing rule update.

Phase 1: Mirror Traffic

Deploy your new microservice on NKP. Configure Traefik Hub to mirror a percentage of production traffic to it. Mirroring sends a copy of each request to the new service while continuing to serve all responses from the monolith. Users see no difference. The mirror's responses are discarded.

Start at 10%. Watch the new service's logs, traces, and error rates. Compare its latency against the monolith's baseline. If it handles 10% cleanly, ramp to 25%, then 50%, then 100%.

Mirroring is zero-risk validation. You are testing with real production traffic patterns, real payloads, and real edge cases. No amount of synthetic load testing gives you this level of confidence.

Phase 2: A/B Testing with Live Traffic

Once the microservice handles 100% of the mirrored traffic cleanly, you are ready to send real traffic. But not all of it. When the microservice successfully processes 100% of the mirrored traffic without issues, you can then begin directing live traffic to it. However, start with a partial rollout.

Use Traefik Hub's weighted round-robin to split traffic between the monolith and the new service. Start with 10% going to the new service and 90% staying on the monolith. Monitor closely.

For more control, use identity-aware routing. Traefik can extract claims from JWT tokens and route based on user attributes. Send internal employees to the new service first. Then beta users. Then a broader cohort. This gives you surgical control over who experiences the new path.

Sticky sessions ensure that once a user is assigned to a version, they stay there for their session. No one bounces between the old and new checkout flow mid-transaction. To prevent users from being switched between the old and new checkout processes mid-transaction, sticky sessions are employed. This guarantees that once a user is assigned to a specific version, they remain with that version for the duration of their session.

Phase 3: Gradual Cutover

As confidence grows, shift the weights. 5/95 becomes 10/90, then 25/75, then 50/50. At each stage, compare error rates, latency, and throughput between the two paths. If anything degrades, shift the weight back. No redeployment. No rollback pipeline. Just a configuration change.

When you reach 100% on the new service, keep the monolith endpoint alive for a while as a failover target. Traefik Hub's failover service automatically redirects traffic back to the monolith if the new service fails health checks.

Only decommission the old path when the team is confident, and the data confirms it.

Phase 4: Repeat

Pick the next service. The process is the same: mirror, A/B test, shift, cut over. Each extraction gets easier because the patterns are established, the tooling is in place, and the team has done it before.

The Multi-Cluster Architecture

What makes this work on Nutanix is the multi-cluster topology. The parent Traefik Hub instance on NKP is the single entry point for all external traffic. It connects to child instances running on:

NKP workload clusters hosting your containerized microservices
AHV VMs hosting your monolith and any services that have not yet been containerized
Additional NKP clusters if you run dedicated clusters for different teams or environments

The parent automatically discovers workloads advertised by each child. When a new service comes online in any child cluster, the parent sees it and can route to it immediately. This works across Kubernetes and VMs transparently because the multi-cluster provider is infrastructure-agnostic. It just needs HTTP connectivity.

This also means your Day 2 operations benefit from the same architecture. Need to spin up a new NKP cluster for a team? Add it as a child, and the parent can route to it. Need to run a canary deployment across clusters? Use weighted routing on the parent to split traffic between the old and new clusters. Need high availability? Configure failover services so traffic shifts automatically when a cluster goes down.

What You Get at the End

By the end of this process, you have:

A unified gateway that manages traffic across VMs and containers from a single control plane
Defense in depth: L7 application security from Traefik Hub and L3/4 microsegmentation from Nutanix Flow, working in concert
Centralized authentication that does not require the monolith to change
End-to-end observability across every service, regardless of where it runs
A proven, repeatable process for extracting services safely
Infrastructure that supports the next evolution, whether that is AI workloads, edge deployments, or API management

The monolith does not disappear overnight. It should not. Some parts of it will run for years, and that is fine. The point is that you can extract what needs to move, at the pace that makes sense for your teams, with the confidence that every step is reversible.

Traffic control is migration control. And on Nutanix with Traefik Hub, you have all the control you need.

Additional Resources

]]>

One Platform. One Gateway. Every Request Governed.

Troy Topnik, Sudeep Goswami — Sun, 19 Apr 2026 09:00:00 GMT

How Traefik Labs and SUSE power unified application delivery governance across VM and container estates, covering VMware migration, Kubernetes ingress, and AI agent control.

Three waves are hitting enterprise infrastructure simultaneously, and they are converging on the same answer.

The first wave is VMware displacement. Two years after Broadcom completed its acquisition of VMware, the predicted mass exodus has not materialized, but a measured, sustained unwind is well underway. CloudBolt's January 2026 survey of 302 enterprise IT decision-makers found that 86 percent of organizations are actively reducing their VMware footprint, while 88 percent report that anticipated future price increases are already shaping infrastructure decisions. Most organizations saw price increases of 25-49%; outlier cases have been dramatically higher. The organizations evaluating SUSE Virtualization, Nutanix, and OpenShift as destinations are not making minor adjustments. They are re-platforming their entire estate, without realizing that infrastructure migration and application delivery migration are two different problems.

The second wave is Kubernetes networking consolidation. The Kubernetes community retired ingress-nginx in March 2026, ending all maintenance, bug fixes, and security patches for one of the most widely deployed components in production infrastructure. SUSE formalized Traefik as the default ingress controller for RKE2 starting with v1.36, extending what K3s has delivered for years. IBM Cloud, Nutanix, OVHcloud, and TIBCO each made the same choice independently.

The third wave is enterprise AI governance. Organizations are deploying AI inference workloads, autonomous agents, and MCP-connected tool pipelines into production. API gateway, AI gateway, and MCP governance are absent across every segment of the enterprise infrastructure landscape today. They have to be added, and the architecture of how they are added determines how well-governed, auditable, and portable the result is.

The joint architecture described in this post addresses all three waves with a single platform and a single upgrade path.

The Architecture at a Glance

Traefik Hub sits in front of both SUSE estates: the VM estate (SUSE Virtualization/Harvester) and the Kubernetes estate (RKE2 + K3s), both managed by SUSE Rancher Suite.

Two Estates, One Management Plane

A common misconception is that SUSE Rancher Prime is a Kubernetes-only management platform and SUSE Virtualization is a VM-only platform. Neither is correct, and the distinction matters significantly for how Traefik is positioned in this architecture.

SUSE Rancher Prime is the unified management control plane for both estates. It manages Kubernetes clusters (RKE2 and K3s) and SUSE Virtualization (Harvester) VM clusters from a single interface and API surface. Platform teams operating mixed environments do not need separate consoles for virtualization and Kubernetes.

SUSE Virtualization (Harvester), the VM platform, is itself built on Kubernetes. It runs RKE2 internally, and VMs are managed as Kubernetes objects via KubeVirt. This means VMs have a Kubernetes API surface: they can be declared as YAML manifests, exposed as Kubernetes services, and managed through the same GitOps workflows as containerized workloads. The current release of Harvester defaults to ingress-nginx. Traefik becomes the default ingress controller in Harvester v1.9, which is in active development. Once v1.9 ships, the Traefik application delivery layer will be consistent across both SUSE estates by default, with no additional configuration required.

On the Kubernetes estate, RKE2 defaults to Traefik from v1.36, and K3s has defaulted to Traefik for years. Together with the Harvester v1.9 roadmap, this means Traefik is converging as the standard ingress and application delivery layer across the entire SUSE estate, from edge K3s clusters through enterprise RKE2 to Harvester-managed VM infrastructure.

The Gateway Layer: Traefik Hub Across Both Estates

One of the consistent findings in analyzing application delivery across VMware customer segments is how wide the gap is. Across every segment reviewed, vSphere-only, VVF, and VCF with or without Avi, API gateway is absent, AI gateway is absent, and MCP governance is absent or limited to transport-layer session persistence. These gaps do not close when organizations migrate to SUSE. They close when organizations deploy Traefik Hub. The Triple Gate architecture addresses all three with a single deployment, a single control plane, and a single audit surface, applied uniformly to both the VM and Kubernetes estates.

API Gateway

The API governance gap is older and deeper than most organizations realize. A typical VMware estate routes API traffic through NGINX or F5 without any catalog of what APIs exist in production, no record of which consumers are calling which endpoints, no deprecation workflow, and no API-level access controls. When organizations begin migrating workloads, they frequently discover that their API surface is both larger and less understood than anyone believed. The migration moment is when the gap becomes operationally expensive rather than theoretically concerning.

Traefik's API Gateway closes this gap at the ingress boundary, before any backend on either estate receives a request. The Coraza WAF is a Go-native implementation of the OWASP Core Rule Set, the same ruleset used in ModSecurity-compatible deployments, but without the C/C++ data plane that regulators are asking vendors to move away from. It applies equally to traffic destined for VM-hosted services on Harvester and container-hosted services on RKE2. Authentication is enforced at the gateway: JWT validation, OAuth2 flows, API key management, and policy-based access control are all handled before the request propagates. Rate limiting operates at the consumer level: per user, per team, and per application, using token bucket or sliding window algorithms. API lifecycle management provides the developer portal, access plans, versioning, and deprecation controls that transform a collection of endpoints into a governed API platform.

The regulated industry argument is direct. PCI-DSS requires a WAF for card data environments. HIPAA requires access logging and breach notification capability. The combination of Traefik's API Gateway for north-south enforcement and SUSE Security's container security for east-west micro-segmentation provides the two-perimeter architecture that zero-trust frameworks prescribe: enforce at the boundary, enforce between workloads. The same deployment that handles API governance also closes the memory safety compliance gap, since Coraza is written entirely in Go.

AI Gateway

AI inference infrastructure creates two simultaneous problems that traditional ingress controllers were never designed to solve.

The first is cost. GPU inference is expensive: a frontier model call that involves a long context window can cost orders of magnitude more than a typical API request, and an application making unconstrained inference calls can generate thousands of dollars in cloud spend within hours. Without a gateway, that spend is invisible until the invoice arrives.
The second is risk. Model outputs can contain PII extracted from training data, prompt injection attacks can manipulate model behavior in ways that produce harmful or policy-violating responses, and models from different providers have different safety characteristics that need to be accounted for in regulated deployments.

Traefik's AI Gateway addresses both dimensions at the infrastructure layer. Multi-provider model routing with weighted failover involves defining a primary inference endpoint and a fallback chain. If the primary provider is rate-limited, unavailable, or over budget, traffic shifts to the next provider automatically, with no application code changes, no DNS updates, and no manual intervention. Token budget enforcement operates as a gateway policy: per-user, per-team, and per-application token budgets are enforced before inference happens. An HTTP 429 response is returned before GPU cycles are consumed, which means cost controls are enforced at the infrastructure layer rather than discovered after the fact through billing reconciliation.

Safety pipelines run as middleware in the request and response chain. NVIDIA Safety NIMs, purpose-built safety models covering PII detection, prompt injection detection, and content filtering, execute in parallel with the primary inference call. The safety check result can gate the response before it reaches the consumer. For organizations running AI workloads on NVIDIA GPU nodes in RKE2, this creates a coherent safety architecture: NVIDIA hardware for inference, NVIDIA safety models for guardrails, and Traefik as the orchestration and enforcement layer that applies those guardrails consistently across every inference endpoint on the platform.

MCP Gateway

The Model Context Protocol is the emerging standard for how AI agents call external tools: databases, APIs, file systems, shell interfaces, and internal services. The governance question MCP raises is different in kind from traditional API governance. A conventional API consumer is a deterministic program calling a known endpoint with known parameters. An MCP-connected AI agent is a probabilistic system whose tool calls are shaped by natural-language instructions, and whose behavior can be redirected by a sufficiently crafted prompt. The blast radius of a prompt injection attack scales directly with the breadth of tool access available to the targeted agent. An agent with unrestricted MCP access to production systems is, in effect, an insider threat that can be triggered from the outside.

Task-Based Access Control (TBAC) is Traefik's answer to this problem, and it operates at a different layer than anything in the traditional load-balancing market. The distinction is worth making precisely: Avi Networks announced MCP session persistence as a tech preview at VMware Explore 2025. Session persistence keeps the same agent talking to the same MCP server connection across requests: this is an L4 transport concern. TBAC defines what tasks an agent can perform, on what resources, under what conditions, at the application layer. A TBAC policy does not say "this agent can connect to this MCP server." It says, "this agent can call this MCP server to perform read operations on cluster configuration data, for clusters in the production namespace, at a rate not exceeding one hundred calls per hour, between 08:00 and 22:00 UTC." Every call outside those parameters is blocked and logged. Session persistence and task authorization are not competing capabilities; they operate at different layers of the stack and address entirely different problems.

The AI Assistant (aka “Liz”) in SUSE Rancher Prime illustrates the practical governance model. It uses MCP to call external tools to retrieve cluster state, diagnose anomalies, and execute remediation workflows. TBAC defines what the assistant can do: which data sources it can query, which cluster operations it can initiate, and under what conditions. A policy that allows the assistant to read monitoring data from a CMDB MCP server does not automatically allow it to write to a production configuration store, even if both are reachable MCP servers. Every tool call is logged: agent identity, tool called, parameters passed, response received, and timestamp, creating the audit trail that compliance teams require when AI is part of a production operational workflow. Critically, the governance layer is independent of the agent. Whether the deployment involves Liz, a third-party agent framework, or a custom-built pipeline, the same TBAC policies apply at the gateway without modifications to the agent itself. The control surface belongs to the platform operator, not to the agent vendor.

All three gates (API, AI, & MCP) run on the same Traefik Hub instance, activated with a single Helm chart upgrade from Traefik Proxy. No re-architecture, no migration of existing routing rules, no new deployment footprint. A platform team that deploys Traefik Proxy as the default ingress controller on RKE2 today has a direct path to closing the API governance gap, the AI cost and safety gap, and the agent authorization gap across both the VM and Kubernetes estates, through a single upgrade operation.

The VMware Exit: Traefik as the Application Delivery Constant

The most common entry point for this joint architecture in 2026 is not a greenfield deployment. It is a VMware exit.

A typical VMware estate runs F5 BIG-IP, NGINX, or HAProxy for load balancing and a standalone WAF from F5 or Cloudflare. Every one of those components is written in C or C++. The regulatory picture on this is no longer advisory. CISA's "Product Security Bad Practices" guidance set January 1, 2026 as the deadline for software manufacturers to publish a memory safety roadmap for any existing product written in memory-unsafe languages used in serving critical infrastructure. That deadline has passed. Any vendor supplying network-facing infrastructure to regulated environments that cannot produce a published memory-safety roadmap is now operating against explicit CISA guidance. A June 2025 joint NSA/CISA Cybersecurity Information Sheet reinforced the same position, identifying memory safety vulnerabilities as serious risks to national security and critical infrastructure. The EU Cyber Resilience Act adds a parallel European mandate. Every legacy application delivery component in the VMware ecosystem (F5, NGINX, HAProxy, Avi, and Envoy) has a C or C++ data plane. Traefik, written in Go, has no memory-safety debt to address. It is already compliant by design.

Both vSphere 7 (October 2025) and vSphere 8 (October 2027) are on fixed end-of-life timelines, with no standalone vSphere 9. VCF 9.0 removed general-purpose NSX load balancing from the base entitlement. The migration follows three stages, with Traefik as the constant across all of them:

Migrate (DAY 0)

Deploy Traefik in front of existing VMware workloads before any infrastructure moves. This replaces C/C++ tools with a single memory-safe Go binary and decouples application identity from infrastructure identity. SUSE VM Import controller migrates VMs from vCenter to SUSE Virtualization (Harvester). Traefik auto-discovers workloads at their new location because they are still exposed as Kubernetes services. Traffic continues to flow without any upstream configuration changes.

Modernize

As workloads are containerized on RKE2, Traefik can manage canary routing between VM-hosted and container-hosted services simultaneously. One policy set governs both estates. No big-bang cutover. RKE2 v1.36 arrives pre-integrated with Traefik, so new clusters join the same governance model immediately.

Transform

AI inference workloads on RKE2 with NVIDIA GPU nodes become a third estate behind the same gateway. Token-level rate limiting governs GPU resource costs. A single Helm chart upgrade from Traefik Proxy to Traefik Hub activates API Gateway, AI Gateway, and MCP Gateway across all three estates simultaneously.

The organizations that deploy Traefik on Day 0 of their VMware exit arrive at their destination with application delivery governance already in place, across VMs and containers alike.

Why Regulated Industries Should Pay Attention

For organizations in regulated environments, Traefik Hub provides a supported, sanctioned path to FIPS 140-2 compliance. The implementation uses BoringCrypto, a FIPS 140-2 validated cryptographic module derived from BoringSSL, compiled directly into the Go binary. This means FIPS compliance is not a configuration option or a wrapper: it is enforced at the cryptographic primitive level, covering TLS, JWT validation, API key handling, and HMAC operations. Organizations operating under FedRAMP, HIPAA, PCI-DSS, or EU CRA requirements can deploy Traefik Hub with a fully documented, auditable cryptographic baseline. Air-gapped deployments are supported through RKE2’s offline image tarball distribution, which bundles Traefik natively, combined with Traefik Hub’s offline activation model.

SUSE Security handles east-west security policy within clusters while Traefik handles north-south governance at the estate boundary. Together, they form the two-perimeter model that zero-trust frameworks require. This applies across both the VM and Kubernetes estates managed by Rancher Prime. For organizations that missed the January 1, 2026 CISA deadline for publishing a memory safety roadmap, the SUSE and Traefik migration is itself the roadmap: moving application delivery to a Go-based stack eliminates the class of vulnerability CISA is targeting, rather than scheduling a future plan to do so.

The Upgrade Path is the Commercial Insight

Every Kubernetes cluster on K3s or RKE2 already runs Traefik Proxy as the default ingress controller. Every Harvester VM cluster runs K3s with Traefik inside. Organizations already running Traefik Proxy across their SUSE estate are one Helm chart upgrade away from Traefik Hub, which adds API Gateway, AI Gateway, MCP Gateway, and full API lifecycle management with no changes to the underlying cluster or existing ingress resources.

For VMware exiters, the same logic applies from Day 0. Adopting Traefik before the first VM moves means the upgrade path to full AI governance is already established before the migration completes. No second integration project, no second vendor, no second control plane.

SUSECON 2026 and What Comes Next

SUSECON 2026 runs from April 20–23 in Prague. The theme is Shape Your Resilient Future, with tracks covering AI adoption, digital sovereignty, Cloud Native modernization, and the shift from legacy virtualization. The SUSE Sovereign Summit opens on April 20 and focuses on infrastructure independence and regulatory compliance. The VMware displacement motion that SUSE is leading into SUSECON is sovereign infrastructure by another name: open source, portable, auditable, not subject to the pricing decisions of a single acquirer.

The architecture described here is deployable today. Teams attending SUSECON can begin the first stage of the migration path before the conference ends.

Summary: The Three Waves are Converging

The joint Traefik and SUSE stack addresses VMware displacement, Kubernetes networking consolidation, and enterprise AI governance across both SUSE estates:

Traefik is already the default ingress inside both SUSE estates: K3s inside Harvester (VM estate) and RKE2/K3s (Kubernetes estate), providing a consistent application delivery model across the entire platform
A three-stage VMware migration path (Migrate, Modernize, Transform) with Traefik as the constant, replacing C/C++ application delivery tools with a memory-safe Go platform from Day 0
SUSE Rancher Prime as the unified management plane for both VM and Kubernetes estates, with Traefik Hub as the unified governance layer in front of both
A single Helm chart upgrade from Traefik Proxy to Traefik Hub activates API Gateway, AI Gateway, and MCP Gateway across all estates simultaneously
A two-perimeter security model: Traefik for north-south governance, SUSE Security for east-west policy, across both VM and container workloads
FIPS-certified, air-gap-ready sanctioned & supported images from Traefik Labs for regulated industry deployments
A memory-safe, Go-based implementation that is compliant by architecture with CISA's January 2026 memory safety roadmap deadline (now passed) and with NSA, FBI, and EU CRA guidance, replacing every C/C++ component in the legacy VMware application delivery stack

Platform teams, CTOs, and CISOs navigating VMware exits, Kubernetes modernization, and enterprise AI adoption in 2026 are not choosing between stability and modernization. With this architecture, they are choosing both on a single platform, across every estate, with a single upgrade path.

The Five Levels of AI Sovereignty: A Maturity Model for Infrastructure Control

Sudeep Goswami — Thu, 16 Apr 2026 07:59:00 GMT

Not all sovereignty is created equal. Here's how to figure out where your organization actually stands, and where it needs to be.

We regularly talk to enterprises that tell us they have "sovereign AI infrastructure." Then we ask a few questions:

What happens if your internet connection goes down?
Where does your AI safety filtering actually run?
If you wanted to move to a different cloud provider next quarter, how much would you have to rewrite?

The answers are usually uncomfortable. The "sovereign" infrastructure turns out to depend on a SaaS control plane in Virginia. The AI guardrails make an API call to a model hosted by a hyperscaler. The governance policies are configured in a vendor console that can't be exported.

This isn't sovereignty. It's a sovereignty theater.

The word has become so overloaded that it's nearly meaningless. Data residency gets called sovereignty. Hybrid clouds get called sovereignty. "We'll run the data plane in your VPC" gets called sovereignty. None of these are wrong, exactly. But they describe very different levels of control, and conflating them is creating real problems for organizations trying to make architecture decisions.

So we built a framework to make the distinctions explicit. We're calling it the AI Sovereignty Maturity Model. Two moving parts: five levels that describe what sovereignty looks like at each stage, and seven dimensions that show where to look when you're assessing your own architecture. It's not a sales tool, it's a diagnostic tool. We use it internally to understand where prospects actually are, and we're publishing it because the industry needs a common language for these conversations.

From Compliance Theater to True Control

The model defines five levels. Each one represents a meaningfully different relationship between an organization and its AI infrastructure.

Level 1: Data Residency

Data is stored in a specific geography. A Frankfurt data center. A US-East region. A specific availability zone that satisfies compliance requirements.

But the control plane managing that data lives somewhere else. The AI models processing queries make calls to external APIs. The policies governing access are configured in a vendor's cloud console.

Level 1 satisfies a checkbox, but it does not provide control. If the vendor has an outage, operations stop. If a geopolitical event disrupts connectivity, operations stop. If the vendor changes their terms or pricing, there's no recourse.

Most organizations that claim sovereignty are actually here.

Level 2: Bring Your Own Cloud

The data plane runs in the customer's environment. Usually their own AWS, Azure, or GCP account. This is a real improvement over Level 1. Cost economics are better because you're using your own committed spend and enterprise discounts. Latency is better because the data doesn't leave your environment for processing. Compliance posture is cleaner because your security team can apply their own controls to the data plane.

But the control plane is still external. The vendor operates the "brain" as a managed service. Configuration, orchestration, policy management, and often authentication all depend on that external connection.

This is where most "hybrid" and "BYOC" architectures land. It's a legitimate improvement over fully hosted SaaS. But it's not sovereignty. If the vendor's control plane goes down, your data plane keeps running on its last known configuration, but you can't update policies, push new routes, or respond to changing conditions. If the vendor decides to change the API, deprecate a feature, or raise prices, you're along for the ride.

Level 3: Self-Hosted (Connected)

Both the control plane and the data plane run on the customer infrastructure. This could be on-premises, in a private cloud, or in a customer-managed public cloud account. The customer operates the full stack.

But external dependencies remain. Licensing validation that phones home. Telemetry that gets sent to the vendor. AI safety checks that call cloud-hosted models. Software updates that require connectivity to pull.

Level 3 organizations have meaningful control over their infrastructure on a day-to-day basis. But they're not independent. An internet outage degrades operations. An air-gapped deployment isn't possible without punching holes in the egress firewall. The architecture has hidden connective tissue to the outside world.

We see many financial services organizations here. They want to operate on-premises for compliance, but their architecture still has dependencies that would fail an air-gap test.

Level 4: Self-Hosted (Autonomous)

The full stack runs on customer infrastructure with no external connections required. None.

Internet connectivity is optional. You can plug in for updates when you want them, but operations continue indefinitely without any vendor communication. Licensing works offline. Governance is policy-as-code that lives in your repo. AI safety systems (guardrails, content filters, jailbreak detection) run on local models rather than using cloud APIs.

If you pull the network cable, everything keeps working.

This is what defense and intelligence organizations require for classified workloads. It's increasingly what healthcare organizations need for sensitive AI processing. It's what any organization should want if they're serious about operational resilience.

Level 5: True Sovereignty

Level 4 capabilities, plus full portability.

This is the distinction that matters most and gets talked about least. An organization can be fully self-hosted and autonomous (Level 4) but still locked into proprietary formats, vendor-specific APIs, and architectures that can't be migrated without a rewrite.

Level 5 means escape velocity. This is what cloud native architecture was always supposed to deliver: the same configuration that runs in AWS runs on-premises. The same policies that work in your US deployment work in your EU deployment. If you decide to change vendors, or change clouds, or bring everything in-house, you can migrate your entire stack without rearchitecting.

You own the architecture, not just the deployment.

This is rare. Most vendors don't want you to have it because it eliminates lock-in. But it's the only level that provides true strategic optionality.

Knowing the levels exist is one thing. Knowing where your own architecture actually sits on them is another. That's the job of the seven dimensions. Each one is a place where sovereignty either holds or quietly fails, and assessing them is how you figure out your real level. There's also a rule about how they combine that catches most organizations off guard, but that comes later. First, the dimensions themselves.

Where Dependencies Hide

Each dimension matters on its own. The tables below show what different situations look like at different levels. The five-level definitions above remain the source of truth; these tables are shortcuts to help you locate yourself quickly. Find the row that describes your situation, and the right column tells you what level that puts you at.

1. Control Plane Location

Where does the "brain" of your AI infrastructure live?

Your situation is…	Achievable level
Control plane is vendor-managed in the vendor's environment. You configure via their console or API, but they operate it.	L1-L2
Control plane runs in your environment but requires periodic external validation (license checks, external IdP calls ).	L3
Control plane is fully self-hosted. No external dependencies. You can operate indefinitely without vendor communication.	L4-L5

This is usually the dimension where organizations overestimate their level. "We run on-premises" doesn't mean Level 4 if every authentication request routes through an external identity provider.

2. Data Plane Location

Where does the actual processing of sensitive data occur?

Your situation is…	Achievable Level
Data plane runs in the vendor's environment (you chose the region, but they operate it).	L1
Data plane runs in your environment (your cloud account, your data center, your hardware).	L2-L5

This is the dimension most vendors emphasize because it's the easiest to solve. BYOC architectures get you to Level 2 on this dimension. But it's only one of seven.

3. Network Dependency

What happens if your internet connection goes down?

Your situation is…	Achievable Level
Operations halt or severely degrade. Control plane is unreachable. New configurations can't be applied.	L1-L2
Core operations continue, but some features are unavailable. Maybe you can't update policies or pull new models.	L3
Full operations continue indefinitely. Internet is optional. Air-gapped deployment is possible.	L4-L5

This is the test we use most often. If the answer is "we'd have problems," that tells you where the hidden dependencies are.

4. Governance Model

How are policies defined, enforced, and audited?

Your situation is…	Achievable Level
Policies configured in the vendor's console. Limited or no export capability. You can't version control your governance.	L1-L2
Policies exportable, but in vendor-specific formats. Partial GitOps support.	L3
Full policy-as-code. Governance lives in your repo. Same policies deploy to any environment. Portable across vendors.	L4-L5

This matters more than most organizations realize. If your governance can't move with you, you don't own it.

5. AI Safety Architecture

Where do guardrails and safety checks execute?

Your situation is…	Achievable Level
Safety checks (topic control, content filtering, jailbreak detection) require external API calls to cloud-hosted models.	L1-L3
Safety systems run locally on your hardware. No external calls. Works offline.	L4-L5

This is becoming critical. Organizations are deploying guardrails to satisfy responsible AI requirements, but those guardrails often introduce the very dependencies sovereignty is supposed to eliminate. If your jailbreak detection calls a cloud API, you've added latency to every request, expanded your attack surface, and potentially violated data sovereignty requirements.

We see organizations that are Level 4 on everything else fall to Level 3 here because their safety stack wasn't designed for offline operation.

6. Agent Governance

How are AI agents authorized to access tools and take actions?

Your situation is…	Achievable Level
No agent governance layer. AI tools are accessed via standard API authentication. Once authenticated, agents have standing access.	L1-L2
Agent governance exists, but it depends on an external policy engine or authorization service.	L3
Dedicated agent gateway with continuous authorization, running entirely on your infrastructure. Per-request policy evaluation.	L4-L5

This dimension didn't exist two years ago. Now it's arguably the most important.

The rise of agentic AI (systems that don't just generate text but take actions via protocols like MCP) breaks traditional security models. An agent might be authenticated, but that doesn't mean every action it takes should be authorized. Agents can hallucinate. Agents can be manipulated by adversarial prompts. Agents can explore solution paths you didn't anticipate.

The security model needs to shift from "authenticate once and trust" to "authorize continuously at the gateway." And that gateway needs to run on your infrastructure. If you're calling an external service to decide whether an agent can read a database, you've introduced latency, dependency, and metadata leakage into every tool call.

Most organizations haven't built this yet. The ones who have are differentiated.

7. Vendor Lock-in / Escape Velocity

Can you migrate your entire stack without rewriting?

Your situation is…	Achievable Level
Migration requires significant rearchitecture. Policies, configurations, and integrations are tied to vendor-specific formats.	L1-L3
Portable deployment artifacts (containers, Helm charts), but some vendor-specific dependencies in configuration or governance.	L4
Full portability. The same configuration works in any environment. Vendor change is a small migration, not a complete rewrite.	L5

This is escape velocity. It's what turns sovereignty from a compliance checkbox into strategic leverage.

When you have Level 5 portability, every vendor conversation changes. You're not negotiating from captivity. You're negotiating as a partner with options. That optionality has real monetary value in every contract renewal.

The Rule That Changes Everything

Now for the rule we promised earlier: your overall sovereignty level is determined by your lowest-scoring dimension.

An organization can be Level 4 on control plane location, Level 4 on data plane, and Level 4 on network dependency model. But if their AI governance model requires their policies to be written in the vendor’s console and cannot be version controlled, they're Level 2 overall. That single dependency defines their ceiling.

This is why "we run the data plane in your VPC" isn't enough. If the control plane is external, you're Level 2. If the safety checks are external, you're Level 3. If you can't migrate without rewriting, you're Level 4.

The model forces you to look at every dependency, not just the ones vendors want to talk about.

Assessing Your Position

Here's a simple worksheet. Rate your organization on each dimension. Be honest. The goal isn't to get a high score. The goal is to understand where you are and where the gaps are.

Dimension	Current Level (1-5)	Target Level (1-5)	Gap
Control Plane Location
Data Plane Location
Network Dependency
Governance Model
AI Safety Architecture
Agent Governance
Vendor Lock-in
Overall (lowest score)

Note: Your overall level is your lowest dimension. If you're Level 4 on six dimensions and Level 2 on one, you're Level 2. There is no averaging. There is no partial credit.

What "Good" Looks Like by Industry

Different industries have different requirements. Here's what we see in practice:

Defense and Intelligence: Level 4-5 required for classified workloads. Air-gapped operation is non-negotiable. Most DoD contracts now explicitly require AI infrastructure capable of operating offline. ITAR and FedRAMP requirements are pushing even unclassified workloads toward Level 4.

Healthcare: Moving toward Level 3-4 for AI processing of PHI. HIPAA doesn't explicitly require it yet, but the Office for Civil Rights (OCR) guidance is trending toward "know where your AI runs." Organizations using AI for clinical decision support are increasingly uncomfortable with cloud-dependent architectures.

Financial Services: Level 3-4 is typical for production AI. The pattern we see most often: develop in the cloud (for speed), deploy on-premises (for compliance). This requires Level 4+ governance portability to avoid maintaining two separate configurations. Level 5 is becoming a competitive differentiator for firms that want to move fast without accumulating technical debt.

Public Sector: Varies widely by jurisdiction. EU sovereign cloud mandates are pushing toward Level 4. APAC data localization requirements are creating demand for Level 3+. The US federal government is increasingly Level 4 for AI workloads, especially post-executive order.

Enterprise (General): Most are Level 2, think they're Level 3, and should be planning for Level 4. The organizations moving fastest on AI are also accumulating the most sovereignty debt. Shadow agents, scattered API keys, ungoverned model access. At some point, the bill comes due.

Why This Matters Now

The AI governance conversation has shifted faster than most organizations realize.

A few months ago, boards were asking, "how do we adopt AI?" Now they're asking, "how do we control AI?" That's a different question, and it requires different infrastructure.

We're seeing it in every enterprise conversation. The initial inquiry is about capability: can you do AI gateway, can you do MCP governance, can you integrate with our LLM providers? But within a few minutes, the conversation shifts to control. Where does this run? What happens if the connection goes down? Can we operate this in our air-gapped environment? How do we avoid getting locked in?

The organizations that build for sovereignty now will have options. The organizations that optimize for convenience will spend the next three years explaining to auditors and boards why they can't answer basic questions about where their AI runs and who controls it.

The maturity model doesn't make the decision for you. But it makes the tradeoffs explicit. And in our experience, once executives see the tradeoffs clearly, they rarely choose Level 2 when they could have Level 4 or Level 5.

For the strategic argument behind why sovereignty is operating leverage (not just compliance), read AI Sovereignty: Why Control Is Your Ultimate Operating Leverage.

]]>

One Gateway. Three Estates. Zero Compromises.

Aniket Daptari, Sudeep Goswami — Fri, 03 Apr 2026 18:06:00 GMT

How Nutanix and Traefik Labs Are Unifying Application Delivery Across VMs, Containers, and AI.

A joint perspective from Nutanix and Traefik Labs.

The infrastructure world is in the middle of a tectonic shift, and it's not a single earthquake. It's three hitting at once.

Enterprise IT teams are navigating a VMware displacement wave triggered by the Broadcom acquisition. They're managing the coexistence of legacy applications on virtual machines alongside modern microservices running in Kubernetes. And they're under pressure to operationalize AI, embedding inference directly into production applications, not as a science experiment but as a business-critical capability.

Each of these shifts alone would be a multi-year transformation. Together, they're redefining what enterprise infrastructure must look like.

Add to this the rise of sovereign computing. Financial services firms bound by data residency mandates. Healthcare organizations navigating patient data regulations across geographies. Government agencies requiring air-gapped deployments. These organizations can't simply lift and shift to hyperscalers. They need infrastructure that keeps data, models, and governance under their control, on their terms, in their locations.

Here's what makes this moment unique: AI is not a separate workload category. It's becoming embedded inside every application. A single user action, logging into a banking app, checking an account balance, initiating a transfer, can touch a containerized frontend, a VM-hosted core system, and an AI inference endpoint within the same request flow. The user doesn't know. The user doesn't care. They just expect it to work.

The question isn't whether enterprises will run workloads across VMs, containers, and AI infrastructure simultaneously. They already do. The question is whether they have a unified way to route, observe, secure, and govern traffic across all three.

The Real Challenges Behind the Buzzwords

When we talk to enterprise infrastructure teams, the conversation quickly moves past technology preferences and into operational reality.

VMs and containers must coexist for years. The notion that every organization will "just move to Kubernetes" ignores the reality of core banking platforms, ERP systems, mainframe-adjacent workloads, and thousands of VM-based applications that will run for another decade. These aren't candidates for re-architecture. They're candidates for better management. At the same time, every new application, every new capability, is being built cloud-native. Infrastructure teams don't get to choose one world. They have to operate in both.

Complexity is the real enemy, not any single vendor. The skills gap in enterprise IT is well documented. Kubernetes expertise remains scarce. Networking teams trained on traditional load balancers are being asked to manage service meshes. Security teams need policy enforcement that spans environments they barely understand. Every additional tool, console, or abstraction layer compounds this problem. Organizations need fewer moving parts, not more.

Escaping one lock-in shouldn't create another. The VMware migration conversation is often framed as "pick a new hypervisor." But the smarter organizations are thinking beyond that. They want strategic decoupling: the ability to move workloads between infrastructure providers without rearchitecting their application delivery, security policies, or observability pipelines. The governance layer must be portable, even if the compute layer changes.

Day 0 preparation is more important than Day 1 migration. Most migration solutions focus on Day 1: getting workloads moved. But the riskiest moment in any migration is the cutover itself, when application traffic must shift from old infrastructure to new without disruption. Organizations that couple application identity directly to infrastructure identity (hardcoded IPs, DNS-dependent routing, infrastructure-specific load balancers) face downtime windows, DNS propagation delays, and split-brain risks every time a workload moves. The smarter approach is to decouple the application layer from the compute layer before migration begins, so the underlying infrastructure becomes interchangeable. And then comes Day 2 and beyond: who handles traffic management across a heterogeneous environment? How do you enforce consistent rate limiting, authentication, and access policies when workloads span VMs and Kubernetes? What happens when you need to add an AI inference call into an existing application flow without redesigning the entire pipeline?

AI insertion demands infrastructure readiness. The organizations best positioned to operationalize AI are not the ones with the most GPUs. They're the ones that already have unified ingress and governance across their infrastructure. If you can't consistently route, observe, and policy-govern traffic across VMs and containers today, you can't add AI inference as a third estate tomorrow. AI readiness is a side effect of good infrastructure, not a separate initiative.

Unified Application Intelligence: The Nutanix and Traefik Labs Joint Solution

Nutanix and Traefik Labs have built a joint solution designed around a simple but powerful principle: one application intelligence layer that follows your workloads, regardless of where they run.

Nutanix provides the foundational compute, storage, and Kubernetes platform. Traefik provides the unified Layer 7 ingress, routing, and governance that spans across it all. Together, they deliver something neither can offer alone: a consistent application delivery experience across VMs, containers, and AI workloads, managed from a single operational plane.

Here's what this looks like in practice.

Unified ingress across three estates. Traefik provides a single entry point for all application traffic, whether it's destined for a virtual machine on Nutanix AHV, a containerized service on Nutanix Kubernetes Platform (NKP), or an AI inference endpoint running on GPU-accelerated infrastructure. One gateway. One set of policies. One place to observe and troubleshoot. No stitching together separate load balancers, API gateways, and AI-specific proxies.

A lightweight footprint from edge to cloud. Traefik's single binary architecture means it runs everywhere: on a Nutanix cluster in a data center, on a small edge appliance at a retail location, or on a GPU node processing inference requests. There's no heavyweight sidecar tax or complex dependency chain. This matters enormously for organizations running Nutanix at the edge for use cases like manufacturing, retail, and healthcare imaging.

Default ingress in NKP. Traefik ships as the default ingress controller in Nutanix Kubernetes Platform. This isn't a marketplace listing or a "works with" certification. It's an engineering decision by Nutanix to make Traefik the native traffic management layer for every NKP deployment. Customers get enterprise-grade ingress out of the box, with a seamless upgrade path to advanced capabilities.

A clear upgrade path without rip-and-replace. Organizations can start with the open-source Traefik Proxy, which already powers millions of deployments worldwide with over 3.4 billion Docker Hub downloads, and upgrade to enterprise-grade capabilities as needs evolve: API gateway functionality, AI gateway with model routing and token-level governance, MCP gateway for agentic AI frameworks, and full API lifecycle management. The same ingress layer grows with the organization.

Under the Hood: How the Integration Works

The Nutanix and Traefik Labs integration goes deeper than simple co-deployment. It's a set of native integrations that make the unified ingress story operational.

Auto-Discovery Across AHV Virtual Machines

Traefik integrates directly with Nutanix Prism Central to auto-discover virtual machines and their associated services. When a new VM spins up or an existing service changes, Traefik detects it automatically and attaches the appropriate routing rules, security policies, and traffic management configurations. No manual reconfiguration. No configuration drift between what's running and what's being managed. This is critical for organizations managing hundreds or thousands of VMs as part of a VMware migration. As workloads land on AHV, they're immediately visible to and governed by Traefik.

Native Kubernetes Integration on NKP

On the Kubernetes side, Traefik integrates natively with NKP using standard Kubernetes Ingress resources and Traefik's own IngressRoute CRDs. Container workloads are auto-discovered as pods scale up and down, services are registered dynamically, and routing policies are applied consistently. Because Traefik is the default ingress in NKP, there's no separate installation or configuration step. It's ready from the first cluster deployment.

Complementing Flow Networking at Layer 7

Nutanix Flow Virtual Networking and the newly released Flow CNI provide robust Layer 4 network segmentation, microsegmentation, and connectivity across the Nutanix stack. Traefik complements this with Layer 7 application intelligence: content-based routing, header inspection, path-based policies, rate limiting, authentication, and API-level governance. Think of it as two layers working in concert. Flow handles the network plumbing and segmentation. Traefik handles the application-aware traffic decisions. Together, they provide defense in depth without requiring organizations to choose between network-level and application-level controls.

AI Workload Routing and Governance

For AI inference workloads, whether running on GPU-accelerated clusters on-premises, at the edge, or across hybrid environments, Traefik provides the same Layer 7 governance it applies to VMs and containers. This includes intelligent model routing (directing requests to the right model version or endpoint based on content, headers, or load), token-level rate limiting to manage expensive GPU resources, and security guardrails that ensure AI endpoints are subject to the same authentication and access policies as every other service. The key architectural insight: AI inference is just another endpoint behind Traefik's unified ingress. It doesn't need a separate gateway, a separate policy framework, or a separate observability pipeline.

What This Looks Like: A Real Application Flow

Consider a banking application, a pattern representative of what we see across financial services, healthcare, and other regulated industries.

A customer opens their mobile banking app. That initial request hits Traefik and is routed to the React frontend running as a containerized service on NKP. The user logs in, and Traefik routes the authentication request through middleware enforcement to identity services, also on NKP. Once authenticated, the dashboard loads account data. Traefik routes that request to the core banking platform, a VM-based system running on Nutanix AHV that has served the institution reliably for years. The user initiates a funds transfer. Before the transaction is processed, Traefik routes a fraud detection request to an AI inference endpoint for real-time scoring. The check passes, and Traefik routes the final transaction request back to the core banking system on AHV to complete the transfer.

Five steps. Three infrastructure types. One gateway managing the entire flow.

The user experienced a seamless interaction. The infrastructure team manages one ingress layer with one set of policies. The security team has one audit trail. The compliance team has one governance framework. And when the next capability gets added, whether it's a new AI model for personalized recommendations or a containerized notification service, it plugs into the same architecture without re-plumbing.

The Time to Act Is Now

For organizations evaluating their infrastructure strategy, three use cases make the Nutanix and Traefik Labs joint solution immediately actionable.

Migrate: Start with Day 0, not Day 1. Most migration strategies focus on Day 1: the moment a VM moves from one hypervisor to another. But the smartest organizations start at Day 0, before a single workload moves, by deploying Traefik as the unified ingress layer in front of their existing VMware environment.

Why? Because the riskiest part of any migration isn't moving the VM. It's maintaining application availability during and after the move. When application traffic is coupled directly to the infrastructure, a VM migration means DNS changes, propagation delays, potential downtime windows, and user-visible disruption. But when Traefik sits in front as the application's ingress layer, the VM becomes a backend detail. You move it from ESXi to Nutanix AHV. Traefik auto-discovers the new location via Prism Central integration. Traffic keeps flowing. The user never knows the underlying substrate changed.

This is the architectural principle that makes migration resilient by design: decouple the application identity from the infrastructure identity. Traefik owns the routing, the policies, and the traffic management. The hypervisor becomes interchangeable. No split-brain risk during cutover. No "please allow 24-48 hours for DNS propagation." No gap between migration and governance. The same ingress layer that manages your VMs on ESXi today manages them on AHV tomorrow and manages your containers and AI workloads the day after that.

Modernize: Decompose monoliths at your own pace. As applications evolve from monolithic VMs to containerized microservices on NKP, Traefik provides seamless traffic shifting between the old and new. Route 90% of traffic to the legacy VM. Shift 10% to the new container-based service. Validate. Adjust. No big-bang cutover. No separate tooling for each environment. The migration from VM to container happens behind the same gateway, with the same policies, at whatever pace the business requires.

AI-Enable: Insert intelligence without re-architecture. When the time comes to add AI capabilities, whether fraud detection, recommendation engines, document processing, or predictive analytics, Traefik's AI and MCP Gateway provides the same routing, governance, and observability for inference and MCP endpoints that it already provides for VMs and containers. There's no new gateway to deploy, no new policy framework to learn, and no new blind spot in your observability pipeline.

These aren't three separate journeys. They're stages of a single infrastructure evolution: prepare at Day 0, migrate at Day 1, modernize at your own pace, and AI-enable when the business demands it. The organizations that put the right application intelligence layer in place before the first VM moves will navigate each stage faster, with less risk, and with fewer late-night fire drills.

The infrastructure world has shifted. VMs, containers, and AI workloads aren't converging someday. They've already converged, inside your applications, inside your request flows, right now. The only question is whether you have one gateway managing all of it, or three separate tools creating three separate problems.

Nutanix and Traefik Labs joint solution chose one gateway. We think you should too.

To learn more about the Nutanix and Traefik Labs joint solution, visit https://traefik.io/solutions/nutanix-and-traefik.

To see the integration in action, join us at .NEXT 2026 in Chicago April 7^th- 9^th.

]]>

Breaking the Monolith: How an API Gateway Turns a Risky Migration into a Controlled Journey

Zaid Albirawi — Wed, 01 Apr 2026 20:01:19 GMT

The monolith isn't the villain. It got you to production, handled your first million users, and shipped features when the team was small enough to fit in one room. But somewhere between the third deploy queue of the day and the incident where a payment change broke the notification system, the conversation starts: "We need to break this thing apart."

The instinct is to plan a big rewrite. Spin up a Kubernetes cluster, redesign the domain model, rebuild from scratch. Eighteen months later, the new system handles 30% of the old one's functionality, and the monolith is still running in production.

There's a better way. Instead of a big-bang rewrite, you migrate incrementally, routing traffic, testing in production, and shifting workloads one piece at a time. The key ingredient is an API gateway that gives you fine-grained control over where every request goes. That's the thesis of this post: traffic control is migration control, and the right gateway turns a risky migration into a series of small, reversible, low-risk moves.

We'll walk through the journey using Traefik Hub as the gateway, but the patterns apply broadly. What makes Traefik particularly well-suited is that it combines routing, load balancing, identity-aware traffic splitting, and multi-platform connectivity into a single platform. Which means the same gateway that handles your Day 1 pass-through can handle your Day 3 or Day 100 canary deployment across VMs and Kubernetes. And as your architecture evolves beyond traditional microservices, the same gateway extends into Kubernetes Gateway API, AI model routing, and MCP, but more on that at the end.

The Gateway as the Starting Point

Before you break anything, you need a control plane for your traffic. If you don't have an API gateway in front of your monolith, that's step zero. If you have a basic load balancer, put Traefik in front of it or replace it entirely.

The setup is deliberately boring: point DNS at Traefik, configure a catch-all route that forwards everything to the monolith. Nothing changes for users. But now every request flows through a layer you own, and that layer is programmable.

This is the foundation. Every pattern that follows, strangler fig routing, canary deployments, and identity-based traffic splitting, depends on having this control point in place. It costs almost nothing to set up, it gives you options you didn't have before, and it works whether your monolith runs on Kubernetes, VMs, or bare metal.

Observability: See Before You Cut

Here's something teams often overlook: before you break anything apart, you need to understand what you have. The moment all traffic flows through the gateway, you gain a single observation point for your entire application, and that's worth pausing on before you start extracting services.

Traefik Hub emits OpenTelemetry metrics, traces, and logs natively. That means the same gateway you just stood up as a pass-through is already producing data you can pipe into Dash0, Datadog, Splunk, Grafana, Dynatrace, or whatever your observability stack looks like. You don't need to instrument the monolith. You don't need to retrofit tracing into a codebase that was never designed for it. The gateway sees every request, every response code, every latency spike, and it tells you about all of it.

This matters for the migration because it gives you a baseline. Before you extract a single service, you know: which endpoints get the most traffic, what the latency distribution looks like, where errors cluster, and which paths are candidates for extraction. You're building the dashboards and alerts before the migration, not scrambling to add them after something breaks.

It also pays off at every subsequent stage. When you start canary deployments, you're comparing the canary's error rate and latency against a baseline you already have. When you do A/B testing, you're measuring real business metrics through the same pipeline. When you shift traffic with identity-aware routing, you can see exactly how internal users experience the new service versus how everyone else experiences the monolith. Observability turns every migration decision from a guess into a data point.That visibility at the gateway is the foundation, but it’s not the finish line. As you start extracting services, tracing needs to extend with the architecture. Instrumenting your microservices as they’re built lets you carry that context forward, turning edge visibility into true end-to-end distributed tracing. Requests no longer stop at the gateway; they can be followed across every service boundary, every downstream dependency, and every internal interaction. That’s when observability stops being a perimeter view and becomes a full system understanding.

Additional Reference: Metrics, Tracing & Logs configuring OpenTelemetry export in Traefik Hub.

Centralizing Auth: Your First Win

With traffic flowing through the gateway, you get an immediate win that has nothing to do with microservices: centralized authentication.

Move auth to the edge. Traefik Hub supports OIDC and JWT middleware natively, so you can authenticate requests at the gateway before they ever reach the monolith. This means new services you extract later don't need to implement their own auth; the gateway handles it.

How you handle tokens downstream depends on your architecture. In a high-trust internal network, you might strip the token entirely and forward only the claim headers. In a zero-trust model, you forward the verified JWT to the backend so it can also make its own authorization decisions. Either way, the gateway becomes the centralized authentication point for all API traffic, ensuring that every request is consistently authenticated before reaching your services.

This matters for the migration story because the identity information flowing through the gateway is exactly what you'll use later to route specific users to new services. Setting up auth now pays dividends down the road.

Additional References:

OIDC Authentication setting up OpenID Connect middleware in Traefik Hub
JWT Authentication setting up JWT middleware in Traefik Hub

Bridging the Infrastructure Gap: No Re-Platforming Required

Before you start extracting services, there's a common blocker worth addressing: "Our monolith runs on VMs. We can't do any of this until we re-platform onto Kubernetes." That's wrong, and getting this piece in place now means every pattern that follows works regardless of where your monolith lives.

Traefik Hub's multi-cluster capability bridges VMs and Kubernetes natively. The architecture is straightforward:

A Traefik instance in Kubernetes acts as the parent cluster and the entry point for all traffic. A Traefik instance on the VMs (in the same network as the monolith) acts as a child cluster, advertising the monolith's workloads via uplinks. The parent discovers these workloads automatically and routes traffic to them. The connection between parent and child is secured with mutual TLS, and both sides verify each other against a trusted CA.

With this bridge in place, every pattern that follows, strangler fig routing, mirroring, canary deployments, and identity-aware routing, works across the VM-to-Kubernetes boundary. You can route specific paths to a new microservice in Kubernetes while everything else stays on the VM monolith. You can canary between the two, split traffic by identity, or mirror requests across the infrastructure divide. The gateway abstracts the boundary, and you migrate workloads at whatever pace makes sense for your team.

As confidence grows and services move to Kubernetes, you adjust the uplink weights. The VM cluster handles less and less traffic. Eventually, the monolith's uplink weight drops to zero. The fig has fully grown.

Additional Reference: Multi-Cluster Traffic Distribution setup guide for parent/child clusters, uplinks, mTLS, and cross-cluster routing patterns.

The Strangler Fig: Carving Out Your First Service

Now the actual decomposition begins. The pattern is called the strangler fig, after the tropical plant that grows around a tree and gradually replaces it. You don't rewrite the monolith, you intercept specific routes at the gateway and redirect them to new services, one endpoint at a time.

Say your monolith handles everything at example.com. You've extracted the orders domain into a new microservice. At the gateway, you add a route: requests to /api/orders go to the new service; all other requests go to the monolith. Clients don't know anything has changed.

The beauty of this approach is that it's incremental and reversible. If the new orders service has a problem, you revert to the monolith. No redeployment, no rollback, just a routing change. Over time, you extract more domains: payments, notifications, and user profiles. The monolith shrinks. The fig grows.

But there's a question lurking here: how do you know the new service actually works correctly under real traffic? You don't want to route 100% of order requests to an untested service and hope for the best. That's where traffic splitting comes in.

Testing in Production Safely

The strangler fig tells you where to route traffic. But how do you know the new service actually works before you commit? You don't flip a switch, you build confidence in stages: first, you observe, then you compare, then you roll out. Traefik gives you a different mechanism for each stage.

Mirroring: Zero-Risk Shadow Testing

The safest way to test a new service is to send it real traffic without affecting users. That's mirroring. Traefik copies a percentage of production requests to the new service but discards its responses; users only ever see the monolith's response. The new service processes real requests, real payloads, real edge cases, and you watch its logs and metrics for anything unexpected.

Since responses from the mirror are discarded, you won't see failures in your client-facing metrics. You need to watch the microservice directly, its logs, its traces, its error rates, and its latency. This is where the observability foundation you set up earlier pays off: the gateway's OpenTelemetry data gives you the monolith's baseline, and the microservice's instrumentation shows how it handles the same traffic. Compare the two. If the microservice is throwing errors on 2% of mirrored requests, you know exactly what to fix before any real user sees that service.

This is also where you catch the problems you'd never find in a staging environment: the malformed request from that one legacy client, the query parameter nobody documented, the header your integration partner sends at 3 am. Mirroring lets you discover it all without risking users. Start at 10%, ramp to 100% as the new service stabilizes.

A/B Testing: Comparing Old and New

Once the new service handles mirrored traffic cleanly, you're ready to send it to real users. A/B testing splits live traffic between the monolith and the microservice using weighted round robin (WRR), so you can compare their behavior side by side.

Sticky sessions matter here. If a user's first request goes to the new service, their subsequent requests should too; otherwise, they'll get inconsistent behavior mid-session.

Traefik supports cookie-based stickiness at the service level, so each user gets a consistent experience even during the split.
For situations where cookies aren't an option, API clients, mobile apps, or sessionless connections, Traefik also supports Highest Random Weight (HRW) hashing, which deterministically routes requests to the same backend based on attributes like the client's IP address. Same consistency guarantee, no cookies required.

The intent of A/B testing is comparison: which implementation is better? You're gathering data to make a decision.

Failover: The Monolith as Your Safety Net

All the patterns above assume you're actively watching and making decisions. But what about 3 am on a Saturday? Failover gives you an automatic safety net: set the new microservice as the primary and the monolith as the fallback. If the microservice goes down or fails health checks, Traefik automatically routes all traffic back to the monolith. No human intervention required.

This is different from weighted round robin. WRR splits traffic intentionally; you're choosing to send some percentage to each service. Failover is binary: use the new thing, but if it breaks, the old thing is still there. During a migration, this means you can go to 100% on the microservice while keeping the monolith warm as a fallback. It's the confidence boost that lets you make the final cutover without holding your breath.

Circuit Breaker: Automatic Damage Control

Failover handles the case where the microservice is completely down. But what about degraded performance, the service is technically responding, but 40% of requests are returning 500s? That's where the circuit breaker comes in.

Traefik's circuit breaker middleware monitors the service's error rate. When errors exceed a threshold you define (say, 50% of requests over a rolling window), the circuit "trips" and stops sending traffic to the failing service entirely. It periodically lets a few requests through to check if the service has recovered, and automatically closes the circuit once it has.

Pair this with failover, and you have a complete safety story: the circuit breaker detects degradation before it becomes an outage, and failover ensures traffic lands on a healthy node. During the migration, this combination means you can be aggressive about shifting traffic to new services, knowing that the system will self-heal if something goes wrong.

Canary Deployments: Shipping New Features Safely

It's worth noting that canary deployments aren't really a migration pattern; they're what you adopt after the migration. Once the orders microservice owns 100% of its traffic, every future release of that service uses canary: roll out v1.1 to 5% of users, watch the metrics, increase to 50%, then cut over. The same WRR mechanism that helped you compare monolith vs. microservice now helps you ship new features safely.

Health checks on the canary version gate each step automatically: if the canary fails its health check, Traefik stops sending traffic to it. No rollback deployment, just a weight shift back to 100% stable.

This is one of the compounding benefits of the gateway-first migration approach. The weighted routing infrastructure you built for the migration doesn't go away; it becomes your standard deployment strategy for every service you extract.

The full toolkit looks like: a mirror to validate with zero risk, an A/B test to compare monolith vs. microservice and decide which wins, a failover and a circuit breaker to catch problems automatically. Once the microservice owns its traffic, the same weighted routing mechanism becomes your canary strategy, same percentage splits, different purpose: now you're shipping new features safely, not choosing between old and new. Each pattern builds on the same gateway foundation, and every step is reversible.

Additional References:
Traffic Mirroring | A/B Testing | Canary Deployments configuration guides.

Identity-Aware Routing: Migrating by Who, Not Just What

Weighted round robin splits traffic randomly. But sometimes you don't want random, you want surgical. Instead of sending 10% of all users to the new service, you want to send specific users: your internal team, your beta testers, your most adventurous enterprise customers.

This is where Traefik Hub's multi-layer routing and identity integration come together. The idea is straightforward: the gateway already authenticates requests (remember the OIDC setup from earlier). The JWT claims contain information about who the user is, their role, their organization, and their tier. You use those claims to make routing decisions.

The mechanism is multi-layer routing. A parent router authenticates the request and extracts claims into headers. A child router matches on those headers and routes accordingly:

This gives you a migration strategy based on user segments. Internal team first, they're forgiving, and they file better bug reports. Then, beta users who opted in. Then a broader cohort. Enterprise customers on the monolith until you're absolutely confident. You control exactly who sees the new code, and you can change the targeting at any time without a deployment.

You can even combine this with a weighted round-robin: route 50% of beta users to the new service and 50% to the monolith. Identity routing for the who, weighted splitting for the how much.

Additional References:
Multi-Layer Routing | Identity-Based Routing identity-aware routing configuration and patterns.

The Triple Gate: API, AI, and MCP

The migration from monolith to microservices isn't the end of the journey; it's the first gate. The gateway you built to control your migration is the same gateway that controls what comes next, and what comes next is a three-gate architecture where a single platform governs all your traffic: services, AI models, and AI agents.

Gate 1: API Gateway. This is where you are now. This layer handles everything you've discussed throughout this post. Routing, authentication, observability, traffic splitting, and identity-aware decisions. Traefik Hub also supports the Kubernetes Gateway API standard, so as the ecosystem moves toward it, you have a native path to adopt it without re-architecting. This gate handles every request from users to services.

Gate 2: AI Gateway. Your newly extracted microservices will start calling LLMs. When they do, the gateway becomes the control plane for AI traffic too. Traefik Hub's AI Gateway routes requests to different model providers, enforces token-based rate limiting, applies content guardrails, and caches responses, all at the gateway layer. The same infrastructure that canary-tested your orders microservice now canary-tests your switch from GPT-4 to Claude. This gate handles every request from services to models.

Gate 3: MCP Gateway. The Model Context Protocol (MCP) is the emerging standard for connecting AI agents to tools and data sources. As your microservices become the tools that agents call, the gateway needs to understand MCP's stateful, session-based communication patterns. Traefik Hub's MCP Gateway routes agent-to-tool traffic, enforces tool-based access control, and maintains session affinity across requests. The services you extracted from the monolith become first-class tools in the agent ecosystem, and the gateway governs access to all of them. This gate handles every request from agents to tools.

The thread connecting all three gates is the one we started with: traffic control is migration control — and it doesn't stop at microservices. The same gateway that manages your monolith decomposition manages your AI model routing and your agent tool access. One platform, three gates, every request.

The Migration Is a Journey

The monolith doesn't die overnight, and it doesn't have to. The worst thing you can do is treat the migration as a binary event, old system off, new system on. The best thing you can do is build the infrastructure that lets you migrate incrementally, test continuously, and roll back instantly.

The API gateway is that infrastructure. It's the control plane for your migration, and for what comes after. Start with a simple pass-through. Set up observability to understand what you have. Centralize auth. Bridge your VMs and Kubernetes so the infrastructure is ready. Extract your first service behind a strangler fig route. Split traffic to test it. Use identity-aware routing to target specific user segments. And when you're ready, extend that same gateway through the triple gate, API, AI, MCP, into whatever your architecture becomes next. At every step, you're in control, and at every step, you can stop and come back to it later.

The monolith took years to build. Give yourself permission to take time to break it apart safely.

Every pattern in this guide, weighted routing, mirroring, canary releases, and load-balancing strategies, is available in Traefik Hub today. To try them on your own cluster, get started with Traefik Hub.

]]>

Implementing Forrester's AEGIS Framework at the Infrastructure Layer

Immánuel Fodor, Zaid Albirawi — Fri, 27 Mar 2026 13:50:51 GMT

Forrester's AEGIS framework (Agentic AI Enterprise Guardrails For Information Security) defines 39 controls across six domains for securing AI agents in the enterprise. It is the most comprehensive analyst framework for agentic AI security to date, with regulatory cross-mapping to NIST AI RMF, ISO/IEC 42001:2023, OWASP Top 10 for LLMs, the EU AI Act, and MITRE ATLAS. CISOs are adopting it as the evaluation standard for agent governance.

The framework is strong on what organizations need. It is intentionally silent on how to implement it. That's by design: AEGIS is vendor-neutral and architecture-agnostic. But it leaves platform engineers and security architects with a practical question: where do these 39 controls actually get enforced?

This post maps AEGIS's six domains and three core principles to Traefik Hub's Triple Gate architecture (API Gateway, AI Gateway, MCP Gateway), with configuration examples showing how specific AEGIS controls translate to Kubernetes CRDs that enforce at the infrastructure layer.

Key Takeaways

Forrester's AEGIS framework defines 39 controls across 6 domains. Its Zero Trust domain explicitly calls for "policy gateways" that all agent traffic must flow through.
Traefik Hub's Triple Gate architecture maps to 4 of 6 AEGIS domains (IAM, Data Security, Application Security, Zero Trust) with strong, product-native implementation from a single platform.
AEGIS's core principle, "least agency," is the design philosophy behind TBAC (Tools/Tasks/Transactions-Based Access Control): infrastructure-layer enforcement of what agents can do, independent of the agent runtime.
The remaining 2 domains (GRC and Threat Management) require organizational processes and complementary solutions, as expected for any infrastructure product.
We also published a companion post mapping infrastructure-layer enforcement to the OWASP Top 10 for Agentic Applications.

Why Infrastructure-Layer Enforcement Matters for AEGIS

Before mapping individual domains, it's worth understanding why AEGIS repeatedly calls for enforcement at the infrastructure layer rather than inside the agent runtime.

AEGIS's Zero Trust domain is explicit: "all access to APIs or data [must] flow through Zero Trust enforcement layers such as policy gateways or conditional access controls." The Application Security domain requires "channeling agent interactions through secure gateways with runtime policy enforcement and input sanitization."

The reason is architectural. Agent platforms are adding application-level controls: RBAC, audit logging, policy SDKs, signed skills. These operate inside the agent runtime. If the runtime is compromised through prompt injection, jailbreak, or privilege escalation, the governance is compromised with it. Application-layer controls and infrastructure-layer enforcement serve different purposes. Application-layer tools govern the agent's logic: what it should do. Infrastructure-layer enforcement governs the traffic: what it's allowed to do, regardless of what the runtime thinks is appropriate.

AEGIS treats both as necessary. This post focuses on the infrastructure layer. For a deeper look at how infrastructure-layer enforcement maps to specific agentic risks, see our companion post on the OWASP Agentic Top 10.

AEGIS Core Principles: Least Agency, Continuous Assurance, and Securing Intent

Before mapping individual domains, AEGIS is built on three core principles. Each maps to a Traefik Hub architectural decision.

Principle 1: Least Agency

AEGIS introduces "least agency" as its foundational principle. It extends the concept of least privilege beyond access control to cover decisions and actions. Agents should receive only the minimum permissions, capabilities, tools, and decision-making authority necessary for a specific task.

Traefik Hub implements least agency through TBAC (Tools/Tasks/Transactions-Based Access Control) in the MCP Gateway. TBAC enforces actions an agent identity is authorized to commit at the infrastructure layer: which tools it can invoke, which tasks it can perform, which transaction parameters are permitted, and under what conditions.

The enforcement is scoped to the agent's JWT identity. The same identity context that authenticates an API call (Gate 1), governs token budgets (Gate 2), and authorizes tool invocations (Gate 3). There's no gap where an agent can authenticate as one identity and act with the permissions of another.

Here's what least agency looks like in practice. Consider an inventory management agent that should be able to look up stock levels and create alerts, but should never be able to approve purchase orders:

Request flow for agent identity "inventory-agent-prod-01":

Gate 1 (API Gateway):
  → JWT validated: agent=inventory-agent-prod-01, team=warehouse, role=inventory-reader
  → Rate limit checked: 1,000 requests/hour for this identity based on the agent scope

Gate 2 (AI Gateway):
  → Token budget checked: 50,000 tokens/day remaining
  → Safety pipeline: all configured tiers execute in parallel
  → Token consumption recorded against this identity

Gate 3 (MCP Gateway):
  → TBAC evaluated:
    - Tool: inventory-lookup (authorized, read-only)
    - Tool: stock-alert (authorized, max 10 creates/hour)
    - Tool: purchase-order (DENIED, requires manager-agent role)
  → Agent tool call budget checked: 10,000 calls/day remaining
  → Invocation executed and logged with full parameter capture

The agent can look up inventory. It can create stock alerts, up to 10 per hour. It cannot approve purchases. These constraints are enforced at the traffic layer, below the agent runtime. If the agent is prompt-injected into believing it has purchase authority, TBAC still blocks the tool invocation because the authorization check happens in the gateway process, not in the agent process.

This is what AEGIS means by least agency: not just controlling access to systems, but controlling what an agent can do within those systems, enforced independently of the agent's own understanding of its permissions.

Principle 2: Continuous Risk Management

AEGIS's second principle replaces periodic audits with continuous evaluation of data, model, and agent integrity. Continuous assurance, not point-in-time compliance.

In Traefik Hub, this translates to per-request enforcement. The composable safety pipeline runs on every request and every response. TBAC evaluates every tool invocation. Token budgets are checked before every LLM interaction. There is no sampling, no periodic scan, no batch evaluation.

JWT validation happens on every request, not once per session. If a token expires, the next request is denied. If a TBAC policy is updated via a Kubernetes CRD change, the updated policy is enforced on the next request, with no redeployment required.

Deployment via Kubernetes CRDs enables GitOps-driven policy management. TBAC policies and safety pipeline configurations are version-controlled, auditable, and continuously reconciled by the Kubernetes controller. Drift from the declared state is automatically corrected. This means governance policies are not just continuously enforced; they're continuously reconciled to the desired state.

Principle 3: Securing Intent

AEGIS's third principle shifts the focus from infrastructure-centric controls to intent-centric controls. Organizations need to understand whether an agent's actions are malicious or benign, intentional or unintentional.

The Triple Gate architecture provides intent visibility across the full AI workflow:

Gate 1 (API Gateway): Who is this user or agent, and are they authorized to access this API?
Gate 2 (AI Gateway): What is the user asking the LLM? Is the content safe? Is the LLM's response factually grounded and free of harmful content?
Gate 3 (MCP Gateway): What tool is the agent invoking, with what parameters? Is this action authorized for this agent identity?

The same JWT identity is tracked across all three gates. When an incident occurs, the audit trail shows the full chain: who authenticated, what they asked, what the model responded, what tool the agent invoked, with what parameters, and what the governance decision was at every step. This is the evidence chain AEGIS requires for securing intent.

Mapping the Six AEGIS Domains

With the principles established, here's how each AEGIS domain maps to infrastructure-layer enforcement. Domains are covered in order. Coverage depth varies: four domains map strongly to product-native capabilities, two require organizational processes where we provide the enforcement layer but not the full domain.

Domain 1: Governance, Risk, and Compliance

AEGIS's GRC domain covers organizational governance: oversight functions, AI system inventories, risk classification, acceptable use policies, and cross-functional governance groups.

Coverage: Partial (enforcement layer). This is primarily an organizational domain, not a product-layer domain. No gateway product provides governance committees or risk classification taxonomies. What Traefik Hub provides is the enforcement infrastructure that makes GRC decisions machine-executable.

GRC policies that exist only in documents don't enforce themselves. A policy that says "agents must not access customer PII without explicit authorization" is only as strong as the system that enforces it. In Traefik Hub, that policy becomes a TBAC CRD that blocks unauthorized access at the traffic layer, plus a safety pipeline rule that masks PII in transit. The GRC team defines what should be allowed; the infrastructure enforces it.

For AI system inventory and risk classification, integration with an existing CMDB or governance platform is required. Traefik Hub's telemetry (which agents exist, what they access, how much they consume, what governance decisions were made) provides the operational evidence that feeds inventory and risk processes.

Domain 2: Identity and Access Management

AEGIS's IAM domain treats agents as a new identity class (neither fully human nor fully machine) that requires unique identification, just-in-time privileges, human oversight, and contextual authentication.

Coverage: Strong. Traefik Hub's implementation centers on JWT-based identity continuity. Every agent interacting with the Triple Gate is identified by its JWT claims. Token rate limiting, safety pipeline logging, TBAC authorization, and observability telemetry all track against this unique identity. This enables:

Per-agent token consumption tracking and budget enforcement
Per-agent distributed traces across all three gates, capturing authentication decisions, guard results, and tool invocations per call via OpenTelemetry
Per-agent safety pipeline results
Per-agent tool invocation history with full parameter capture

For just-in-time privileges, TBAC supports conditional, scoped authorization. High-risk tool invocations can require human approval. Time-windowed permissions can restrict agent capabilities to specific operational hours. Parameter-level constraints can limit the scope of approved actions.

Authentication is continuous. JWT validation occurs on every request at Gate 1. The validated identity propagates through Gates 2 and 3, where it is re-evaluated against the relevant TBAC policies and token budgets. There is no session-based trust. Every interaction is independently validated.

Domain 3: Data Security and Privacy

AEGIS's Data Security domain covers data classification for agent contexts, controls for agent memory, data enclaves, anonymization, privacy-preserving AI operations, and data provenance.

Coverage: Strong. The composable safety pipeline is Traefik Hub's implementation of these controls. It operates bidirectionally (on both requests and responses) across four tiers, each with different detection capabilities.

Here's a configuration that implements multiple AEGIS data security controls in a single middleware:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: aegis-data-protection
  namespace: ai-services
spec:
  plugin:
    content-guard:
      engine:
        regex: {}

      request:
        # Prevent data extraction via prompt manipulation
        - jsonQueries:
            - ".messages[].content"
          block: true
          reason: "data_extraction_detected"
          entities:
            - "(?i)repeat.*(training|system|original).*(data|prompt|instructions)"
            - "(?i)output.*(everything|all).*(you know|in your|you were)"

        # Mask Social Security numbers before they reach the LLM
        - jsonQueries:
            - ".messages[].content"
            - ".context"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{3}-\\d{2}-\\d{4}"

        # Mask credit card numbers
        - jsonQueries:
            - ".messages[].content"
            - ".customer.payment"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}[-\\s]?\\d{4}"

      response:
        # Prevent infrastructure data leakage in LLM responses
        - jsonQueries:
            - ".choices[].message.content"
          mask:
            char: "X"
            unmaskFromLeft: 0
            unmaskFromRight: 0
          entities:
            - "192\\.168\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "10\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "(?i)(postgres|mysql|redis)://[^\\s]+"

This configuration covers three AEGIS data security requirements:

Data loss prevention (request-side): The first rule blocks prompts that attempt to extract training data or system instructions. This is deterministic, sub-millisecond, and catches the most common data extraction patterns.

PII anonymization (request-side): The second and third rules mask SSNs and credit card numbers before they reach the LLM. The data is redacted in-place: the request continues with ***-**-1234 instead of the full SSN. The LLM never sees the original value.

Infrastructure data protection (response-side): The fourth rule prevents the LLM from leaking internal IP addresses and database connection strings in its responses. This catches a common failure mode where models trained on internal documentation inadvertently expose infrastructure details.

This is Tier 1 (Regex Guard) only. For PII that doesn't follow strict patterns (misspelled names, partial phone numbers, context-dependent personal data), add Tier 2 (Microsoft Presidio). For semantic threats (prompt injection attempts that don't use obvious keywords, off-topic drift, harmful content), add Tier 3 (NVIDIA NIMs) and Tier 4 (IBM Granite Guardian). All tiers run in parallel. For a detailed walkthrough of the full four-tier pipeline, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

A note on deployment model: Traefik Hub is self-hosted. All safety pipeline processing runs on the customer's infrastructure. No data leaves the customer's environment for governance evaluation. For organizations subject to data residency requirements (EU AI Act, GDPR, NIS2), this is not a feature but a requirement. Air-gapped deployment is fully supported.

Domain 4: Application Security and DevSecOps

AEGIS's Application Security domain covers the agent software lifecycle: risk evaluation, DevSecOps practices, supply chain assessment, adversarial testing, embedded telemetry, sandbox interactions, input sanitization, and runtime policy enforcement.

Coverage: Strong. Two aspects of this domain are particularly relevant at the infrastructure layer: runtime policy enforcement and supply chain security.

Runtime policy enforcement: TBAC policies are expressed as Kubernetes CRDs and enforced on every MCP tool invocation. This is runtime enforcement that operates independently of the agent's application code. The agent framework does not evaluate TBAC policies, cannot modify them, and has no mechanism to bypass them.

AEGIS also requires that governance controls work with agent architectures, not against them. Traditional gateways return an HTTP 403 Forbidden when a policy blocks a request. For autonomous agents executing multi-step workflows, a 403 breaks the agent's control flow. The agent can't distinguish between "the guardrail blocked this specific request" and "something is fundamentally wrong with the system." Most agent frameworks treat non-2xx responses as system errors.

Traefik Hub's onDenyResponse feature addresses this. When a guard blocks a request, the response can be configured to return a structured, schema-compliant refusal (HTTP 200 with a conversational message) instead of a raw 403:

blockConditions:
  - reason: harmful_content
    condition: Contains("yes")
    onDenyResponse:
      statusCode: 200
      message: >
        This request was blocked by organizational security policy.
        Please reformulate your request within acceptable use guidelines.

An agent receiving this structured refusal can parse it as a valid conversational turn, understand that the request was refused (not that the system failed), and decide what to do next. The middleware chain downstream continues to function normally. This is what makes infrastructure-layer governance agent-aware: enforcement that works with autonomous workflows rather than breaking them.

Supply chain security for agent tools: AEGIS calls for software supply chain assessment, which in the agentic AI context means securing the MCP tool supply chain. The risk is real: malicious MCP servers have already been documented in the wild (the Postmark MCP backdoor in September 2025 shipped 15 clean versions before injecting a backdoor in v1.0.16, affecting approximately 500 organizations).

The MCP Gateway addresses this at the infrastructure layer:

Server allowlisting: TBAC policies explicitly define which MCP servers each agent identity can connect to. Connections to unlisted servers are denied at the gateway.
Tool-level authorization: Even within an allowed MCP server, TBAC controls which specific tools the agent can invoke.
Parameter constraints: TBAC can limit the parameters an agent passes to authorized tools (read-only database queries, restricted file paths, capped transaction values).
Outbound inspection: The MCP Gateway can inspect outgoing API calls from agent tool invocations, preventing data exfiltration through tool parameters.

An agent that gets prompt-injected into connecting to a malicious MCP server is blocked by the TBAC allowlist. These are infrastructure-layer controls that the agent runtime cannot override.

Domain 5: Threat Management and Security Operations

AEGIS's Threat Management domain covers real-time monitoring, anomaly detection, comprehensive logging, rapid containment, hallucination detection, and incident response readiness.

Coverage: Partial (detection and containment). Traefik Hub covers the detection and containment aspects of this domain through the composable safety pipeline and operational controls.

Hallucination detection deserves specific attention because AEGIS calls it out as a required capability and because it is architecturally distinct from other threat types. IBM Granite Guardian provides hallucination detection and RAG quality assessment through the LLM Guard middleware. The useRequestHistory: true parameter provides the request context so the model can evaluate whether the response is factually supported. Regex patterns and NLP-based entity detection cannot detect hallucinations because they match surface patterns, not factual accuracy. This requires a semantic model.

Parallel threat detection across multiple guard providers (NVIDIA NIMs for 22+ harm categories and jailbreak detection, IBM Granite Guardian for hallucination and harm) runs concurrently. Total enforcement time equals the slowest guard, not the sum.

Rapid containment via the Failover Router implements circuit-breaker chains. When a provider returns error status codes (429, 500-504), traffic is automatically rerouted to fallback providers. All safety policies remain enforced regardless of which provider is serving the request. The governance doesn't degrade when the model does.

Comprehensive logging across every gate emits structured telemetry: API authentication results (Gate 1), safety pipeline guard decisions with reasons (Gate 2), and TBAC authorization decisions with tool and parameter details (Gate 3). All telemetry includes the agent identity for cross-gate correlation and is exported via OpenTelemetry to the observability platform of your choice.

Request mirroring provides a distinct and complementary capability worth calling out specifically. Traefik Hub can forward a live copy of every request and response to an external destination in parallel with the primary traffic flow, with no impact on latency for the agent workload. For SIEM and UEBA integration, this means security tooling receives the actual payloads in real time, not just metadata after the fact. Threat detection models can analyze live traffic, identify anomalous patterns, and feed updated policy signals back to the gateway. This is the integration point where infrastructure-layer enforcement connects to SOC workflows.

A note on scope: Traefik Hub provides detection and containment at the traffic layer. Behavioral anomaly detection (establishing baseline invocation patterns and alerting on deviation) and incident response workflows are organizational capabilities that require SIEM/UEBA integration and SOC processes. AEGIS expects both. The gateway provides the telemetry and the live traffic mirror that those processes require; the analytical and response layers sit above it.

Domain 6: Zero Trust Architecture

AEGIS's Zero Trust domain is the most direct validation of infrastructure-layer enforcement. The AEGIS framework requirements read as a description of a policy gateway architecture:

"All access to APIs or data [must] flow through Zero Trust enforcement layers such as policy gateways or conditional access controls."
Network access layer controls for agent workloads.
Continuous validation of agent runtime environment.
Agent-to-agent communication monitoring.
Ephemeral identities for autonomous systems.

Coverage: Strong. The Triple Gate architecture is a policy gateway. Every API call (Gate 1), every LLM interaction (Gate 2), and every MCP tool invocation (Gate 3) flows through the gateway with policy enforcement. There is no bypass path.

TBAC enforces least agency at the infrastructure layer. Agents receive only the tools, tasks, and transaction authority explicitly granted by their TBAC policy. No implicit permissions. No inheritance of elevated privileges from the agent runtime.

When agents communicate via API calls or MCP tool invocations, both sides of the interaction pass through the Triple Gate. The initiating agent's request is authenticated and authorized via TBAC. The receiving agent's response passes through the safety pipeline. The full interaction is logged with both agent identities.

JWT-based identity supports scoped tokens with configurable expiration. Agents can be issued tokens that are task-scoped, with a short time-to-live set at issuance time. Expired tokens and tokens with incorrect scopes are rejected.

For organizations in regulated industries where SaaS governance tools are not permitted, Traefik Hub is self-hosted on the customer's infrastructure. Air-gapped deployment is fully supported. Traefik Hub is FIPS 140-2 validated and FIPS 140-3 ready (native Go BoringCrypto). No governance data, telemetry, or policy configurations leave the customer's environment.

Putting It Together: One Request, Five Domains

Here's how AEGIS domains intersect in a single request through the Triple Gate:

Request arrives. Gate 1 validates the agent's JWT and enforces API rate limits. → Domain 2 (IAM), Domain 6 (ZTA)
Token budget check. If the estimated input tokens would exceed the agent's remaining budget, the request is blocked before any guard runs. → Domain 6 (ZTA: least agency applied to resource consumption)
Safety pipeline executes. Regex Guard masks SSNs and credit cards. Presidio catches context-dependent PII. NVIDIA NIMs scan for prompt injection and harmful content. All tiers run in parallel. → Domain 3 (Data Security), Domain 5 (Threat Management)
LLM responds. The response passes back through the pipeline. Granite Guardian checks for hallucination against the request context. Regex Guard masks any internal IP addresses in the output. → Domain 3 (Data Security), Domain 5 (Threat Management)
Agent invokes an MCP tool. TBAC validates the tool invocation against the agent's authorized tools, permitted parameters, and usage limits. → Domain 4 (Application Security), Domain 6 (ZTA: least agency)
Telemetry emitted. Every gate logs the agent identity, governance decisions, guard results, tool invocations, and token consumption. → Domain 1 (GRC: evidence for governance reporting), Domain 5 (Threat Management: SOC monitoring)

Five of six AEGIS domains are enforced in a single request, through one platform, with one identity context.

Coverage Summary

Not every AEGIS domain is a product-layer control. AEGIS is a comprehensive organizational framework, and some domains require processes, teams, and governance structures that no infrastructure product can provide. Here's our assessment of where Traefik Hub's Triple Gate provides strong coverage and where complementary solutions are required:

AEGIS Domain	Triple Gate Coverage	What's Covered	What Requires Complement
Domain 1: GRC	Partial (enforcement)	Machine-executable policy enforcement, governance evidence via telemetry	Governance oversight function, AI system inventory, risk classification
Domain 2: IAM	Strong	JWT identity continuity, per-agent tracking, TBAC-scoped authorization, continuous authentication	Agent onboarding/offboarding workflows (requires IdP integration)
Domain 3: Data Security	Strong	4-tier PII detection and masking, bidirectional enforcement, DLP, self-hosted data processing	Data classification taxonomy, agent memory controls
Domain 4: Application Security	Strong	TBAC runtime policy enforcement, MCP supply chain security, agent-aware error handling, input sanitization	SDLC integration, adversarial red teaming
Domain 5: Threat Management	Partial (detection + containment)	Hallucination detection, content threat detection, prompt injection defense, circuit breakers, comprehensive logging	Behavioral anomaly detection, forensic investigation workflows, incident response playbooks
Domain 6: Zero Trust	Strong	Policy gateway enforcement, least agency via TBAC, continuous validation, agent-to-agent monitoring, air-gapped deployment	Network segmentation (requires CNI integration)

Four of six domains with strong, product-native implementation. Two domains with partial coverage that requires organizational processes or complementary solutions. This is an expected distribution for an infrastructure product: GRC and SecOps are organizational capabilities, while IAM, Data Security, Application Security, and Zero Trust are enforceable at the infrastructure layer.

What's Next

AEGIS is gaining traction as the evaluation standard for agentic AI governance. As CISOs adopt it, the question shifts from "do we need agent governance?" to "how do we implement these 39 controls?" For the infrastructure-layer controls, the answer is a policy gateway that enforces identity, data security, application security, and zero trust on every request, every response, and every tool invocation.

For a complementary perspective on infrastructure-layer enforcement, see our mapping of the OWASP Top 10 for Agentic Applications to the gateway layer. And for a deep technical walkthrough of the composable safety pipeline referenced throughout this post, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Traefik Hub v3.20 is now available as an Early Access release, with general availability planned for late April 2026. To try the capabilities described in this post, sign up for Early Access.

Frequently Asked Questions

What is Forrester's AEGIS framework?

AEGIS (Agentic AI Enterprise Guardrails For Information Security) is a security framework published by Forrester Research that defines 39 controls across six domains for securing AI agents in the enterprise. The six domains are: Governance, Risk, and Compliance (GRC); Identity and Access Management (IAM); Data Security and Privacy; Application Security and DevSecOps; Threat Management and Security Operations; and Zero Trust Architecture. The framework includes regulatory cross-mapping to NIST AI RMF, ISO/IEC 42001, OWASP Top 10 for LLMs, the EU AI Act, and MITRE ATLAS.

What is "least agency" in the AEGIS framework?

Least agency is AEGIS's foundational security principle. It extends the concept of least privilege beyond access control to cover decisions and actions. Under least agency, AI agents should receive only the minimum permissions, capabilities, tools, and decision-making authority necessary for a specific task. It is the agentic equivalent of the principle of least privilege, applied to autonomous systems that make decisions, not just access resources.

How do you implement AEGIS controls at the infrastructure layer?

AEGIS controls can be implemented at the infrastructure layer through a policy gateway that sits between agent workloads and the services they consume. Traefik Hub's Triple Gate architecture (API Gateway, AI Gateway, MCP Gateway) enforces AEGIS controls on every API call, LLM interaction, and MCP tool invocation. Identity controls are enforced via JWT-based authentication. Data security controls are enforced via a composable safety pipeline. Application security controls are enforced via TBAC (Tools/Tasks/Transactions-Based Access Control). Zero trust controls are enforced through continuous per-request validation.

How many AEGIS domains can be enforced at the infrastructure layer?

Four of six AEGIS domains (IAM, Data Security, Application Security, Zero Trust) can be strongly enforced at the infrastructure layer through a policy gateway. The remaining two domains (GRC and Threat Management/SecOps) are primarily organizational capabilities that require governance processes, SIEM integration, and SOC operations, though infrastructure-layer telemetry provides the evidence these processes need.

What is the difference between AEGIS and the OWASP Top 10 for Agentic Applications?

AEGIS is a comprehensive organizational security framework with 39 controls across six governance domains, focused on how enterprises should structure their AI agent security programs. The OWASP Top 10 for Agentic Applications is a risk taxonomy that identifies the ten most critical security risks for autonomous AI agents (such as goal hijacking, tool misuse, and identity abuse). AEGIS tells you what controls to implement; OWASP tells you what risks to prioritize. Both are complementary and map to infrastructure-layer enforcement. See our OWASP Agentic Top 10 mapping for the risk-level view.

]]>

The Ingress NGINX Retirement Created Three Risks. Most Teams Are Only Managing One.

Sudeep Goswami — Thu, 26 Mar 2026 12:59:48 GMT

The retirement of Ingress NGINX is official. As of March 24, 2026, there are no more releases, no more bug fixes, and no more security patches. Roughly half of all cloud-native environments are running a controller that will never be updated again. The urgency of the situation is not in question.

What is in question is whether the response to that urgency is complete.

The Ingress NGINX retirement did not create one risk; it created three. The first is migration risk: the operational challenge of moving hundreds of clusters off a retired controller without breaking production. The second is security risk: the architectural question of whether the new controller eliminates the vulnerability class that made Ingress NGINX untenable, or inherits it. The third is stagnation risk: the strategic question of whether the foundation being laid today can support the governance layers every organization will need to build on top of it over the next several years.

The migration conversation happening across the Kubernetes community right now is almost entirely focused on the first risk. The other two deserve equal attention. The decisions being made now will compound, for better or worse, across all three.

Migration Risk

Migration risk is the risk most teams understand best, because it is the most immediately visible. Clusters running Ingress NGINX today are running unmaintained software. Every day that passes is a day closer to an unpatched CVE landing in a production environment with no fix available. The operational pressure to move is real and justified.

But migration risk is not just about urgency. It is about what the migration actually requires in practice, and that is where most of the market response has fallen short.

When the Ingress NGINX retirement was announced, vendors responded by positioning their controllers as the destination. Compatibility tables appeared. Blog posts explained why each vendor's product was the natural landing spot. The conversation became: which controller should we migrate to?

That framing misses the harder problem. Ingress NGINX users have built configurations over years using a specific annotation vocabulary. Some of those annotations have direct equivalents in other controllers. Some require behavioral substitutes. Some require rethinking routing logic entirely. For a team running a handful of clusters, that is a manageable audit. For a team running hundreds, it is a multi-quarter engineering project, unless someone has already built the bridge.

Traefik built the bridge. The open-source Ingress NGINX migration tool, available at github.com/traefik/ingress-nginx-migration, analyzes existing cluster configurations, maps annotations against Traefik's Ingress NGINX Provider, identifies gaps, and generates a migration report before any commitment is made. Traefik's Ingress NGINX Provider covers over 90% of Ingress NGINX annotations, meaning the vast majority of existing configurations migrate without manual rewriting.

That covers the immediate operational problem. But migration risk also has a longer dimension that most teams are not factoring in: the configuration model chosen today becomes the configuration model maintained for years.

The Kubernetes community has been clear on the direction. The Ingress spec is feature-frozen. Kubernetes Gateway API is the designated successor, now a CNCF-standard specification with broad ecosystem backing, and the routing model that future tooling, documentation, and community investment will be built around. Teams treating this migration purely as a like-for-like swap are solving the urgency problem without capturing the modernization opportunity.

Traefik supports both paths simultaneously. The same binary, the same Helm chart, the same Go-native data plane handles Ingress resources, Kubernetes Gateway API, and Traefik's native IngressRoute CRD concurrently. A team that needs to move fast can drop in Traefik against existing Ingress configurations today, validate the migration, and plan the Gateway API transition on its own schedule, without a future re-architecture. The migration decision and the modernization decision can be made independently. No second migration. No second vendor.

This reflects a long-view approach to winning in this market: solve the actual engineering problem teams are facing, and earn the adoption that follows. Not a destination to land on. A bridge to get there without operational disruption.

Migration risk is real and solvable. It is also the least consequential of the three risks if the other two go unaddressed.

Security Risk

Security risk is the risk the migration conversation has most conspicuously skipped. It is also the one with the most structural teeth.

Ingress controller migrations do not happen often. For most organizations, this is the first one in years and will likely be the last one for years to come. The operational cost of replacing a controller at scale is high enough that once the decision is made, it stays made. Which means the question being answered right now is not just "which controller do we migrate to." It is "which security architecture do we commit to for the next several years."

Most of the migration conversation has been focused on whether the destination controller is actively maintained. That is a necessary condition but not a sufficient one. A controller can be actively maintained and still carry a vulnerability class that no amount of maintenance can eliminate. The question worth asking is what the controller is built from, not just who is building it.

C and C++ put memory management responsibility on the developer. When that responsibility is handled imperfectly, the result can be a buffer overflow, a use-after-free, or a heap corruption bug. The critical word is "when," not "if." Two-thirds of reported vulnerabilities in memory-unsafe programming languages still relate to memory issues (CISA), despite decades of investment in tooling, training, and coding standards. Google Project Zero's review of real-world exploits found that 75% of CVEs used in active attacks were memory safety flaws (Industrial Cyber).

The pattern is consistent enough that the conclusion is unavoidable: this is not a discipline problem. It is a structural one. These vulnerabilities can be found and patched in any given version. They cannot be designed out of a C or C++ codebase.

The NSA, CISA, the FBI, and allied cybersecurity agencies across the UK, Australia, Canada, and New Zealand looked at that evidence and reached a formal position: the only durable fix is a language-level one. In October 2024, CISA and the FBI attached a deadline: publish a memory safety roadmap by the end of 2025 or be in a position described as "dangerous and significantly elevating risk to national security."

The deadline has since passed.

Among every C and C++ ingress controller on the official Kubernetes documentation page, not one published a roadmap.

Alarmingly, 77% of the self-hosted ingress controllers listed on the official Kubernetes documentation page currently carry C or C++ memory safety exposure with no published remediation roadmap.

Traefik Proxy is written in Go, which is on the NSA and CISA approved memory-safe language list. Go's garbage collector makes use-after-free structurally impossible. Go's bounds checking makes buffer overflow structurally impossible. These are not mitigations. They are language-level guarantees that remove the vulnerability class from the possibility space entirely. Traefik Proxy has no memory safety debt, and therefore no roadmap to publish.

A migration that lands on a Go-native controller addresses migration risk and security risk in a single motion. A migration that lands on a C or C++ controller defers the memory safety question to the next CVE cycle.

Stagnation Risk

Stagnation risk is the risk almost nobody is talking about. It is also the one with the longest tail.

Ingress NGINX retirement is forcing a migration decision today. But the ingress controller is not the destination. It is the first step.

Every organization running Kubernetes at production scale follows a predictable architectural evolution, and the controller chosen today will either support that evolution or constrain it.

Traffic enters through the ingress layer. As application complexity grows, an API gateway becomes necessary to centralize authentication, rate limiting, routing policy, and API lifecycle management. As security requirements mature, WAF capability provides protection against application-layer attacks. As AI workloads arrive, an AI gateway governs token costs, content policy, model routing, and inference guardrails. As agents and agentic systems move into production, an MCP gateway governs which tools agents can call, under what permissions, and according to what policy constraints.

Each of these is a layer of runtime governance. Each one needs to be deployed, configured, and maintained. And here is where stagnation risk becomes concrete: if each layer requires a separate product from a separate vendor on a separate control plane, the operational overhead compounds with every gateway added. If adding WAF capability means ripping out the ingress layer and replacing it with something else, the organization paid the migration cost twice. If the AI gateway requires a different data plane than the API gateway, the security posture of the two layers cannot be governed consistently.

Stagnation risk is the risk of building the governance architecture on a foundation that cannot grow with it.

Traefik addresses stagnation risk through an architecture built specifically for this evolution. Traefik Proxy is the ingress layer. Traefik Hub extends it, through Helm chart upgrades, to API gateway with integrated Coraza WAF, to AI gateway with NVIDIA safety guardrails and token governance, to MCP gateway with TBAC-based agent access control, and to full API management with GitOps-driven lifecycle management. A single binary. A unified control plane. The Triple Gate Pattern: API Gateway as Gate 1, AI Gateway as Gate 2, MCP Gateway as Gate 3, each adding a layer of runtime governance on the same Go-native, memory-safe foundation.

This means the organization that migrates to Traefik today does not pay a migration cost again when it adds WAF. It does not adopt a new vendor when it needs AI governance. It does not rearchitect the data plane when agents arrive. Each capability is a Helm upgrade on the same foundation. The stagnation risk collapses to near zero because the journey was planned for from the beginning.

The controller chosen today is not just an answer to the migration question. It is the foundation every subsequent decision gets built on. Choosing wrong means paying twice.

A Further Dimension for European Organizations

For organizations operating in European markets, all three risks carry an additional regulatory layer with hard dates. The EU Cyber Resilience Act, which entered into force in December 2024, requires that products with digital elements be designed with security from the outset, with secure practices as a design-phase requirement. Vulnerability reporting obligations become mandatory on September 11, 2026. Full compliance, including the requirement that non-compliant products cannot be sold in the EU market, takes effect on December 11, 2027. An ingress controller sits directly in the path of traffic to products subject to CRA compliance. The language-level security posture of that controller is inside the compliance perimeter. The end of 2025 CISA deadline has passed. The September 2026 CRA reporting deadline is six months away.

All Three Risks Are Addressable. Now Is the Time.

The Ingress NGINX retirement created a forcing function. Teams are doing the work regardless: auditing clusters, mapping configurations, planning cutovers. The operational investment is already in motion.

That investment is the reason now is exactly the right time to address all three risks, not just one. The incremental cost of choosing a Go-native, journey-ready controller over a C or C++ alternative is close to zero at the point where the migration is already running. The long-term cost of not making that choice accumulates with every security advisory, every compliance review, and every governance layer that has to be rebuilt on a foundation that was not designed for it.

Migration risk: addressed by the Traefik Ingress NGINX migration tool, the 90%+ annotation coverage of the Ingress NGINX Provider, and the step-by-step migration guide at ingressnginxmigration.org.

Security risk: addressed by Traefik Proxy's Go-native architecture, which eliminates the memory safety vulnerability class by design, with no debt and no roadmap required.

Stagnation risk: addressed by Traefik Hub's unified platform, which extends from ingress to API gateway, WAF, AI gateway, MCP governance, and API management through Helm upgrades on a single binary and a single control plane.

Three risks. One foundation. Now is the right time to address all of them.

Ready to start? Run the open-source migration tool at github.com/traefik/ingress-nginx-migration and generate your compatibility report in minutes. For enterprise migrations at scale, Traefik Labs provides dedicated migration support at traefik.io/solutions/oss-support.

]]>

OWASP Agentic Top 10: Where Should These Controls Be Enforced?

Zaid Albirawi, Immánuel Fodor — Wed, 25 Mar 2026 13:55:17 GMT

The OWASP Top 10 for Agentic Applications, published in December 2025, defines the ten most critical security risks for autonomous AI agents. It is quickly becoming the standard framework enterprise security teams use to evaluate agent governance solutions, and multiple vendors are publishing their coverage maps.

This is useful. The agentic AI space needed a shared vocabulary for risk, and OWASP provided one.

But the framework raises an architectural question that most coverage announcements skip over: where should these controls actually be enforced?

Key Takeaways

8 of 10 OWASP Agentic risks describe scenarios where the agent itself is the compromised component. Governance running inside that same process is operating in a compromised environment.
Infrastructure-layer enforcement (at the gateway) works even when the application layer is compromised, because it sits outside the blast radius.
Traefik Hub's Triple Gate covers 6 of 10 risks with strong gateway-layer enforcement, 2 with moderate coverage, and 2 with partial coverage. We'd rather publish an honest map than claim 10/10 coverage that doesn't survive architectural scrutiny.

The Enforcement Boundary Problem

Eight of the ten OWASP Agentic risks describe scenarios where the agent itself is the compromised or malfunctioning component. The agent's goals get hijacked (ASI-01). The agent misuses its tools (ASI-02). The agent escalates its own privileges (ASI-03). The agent's memory gets poisoned (ASI-06). The agent goes rogue (ASI-10).

This creates a problem for governance solutions that run inside the agent process. If the agent is hijacked, the governance middleware running in that same process is operating in a compromised environment. If the agent is rogue, application-layer kill switches depend on the rogue process honoring them. A policy engine that runs as an SDK import in the agent's code can be bypassed by the same exploit that compromised the agent.

The pattern is familiar. It's the same reason network firewalls exist alongside host-based security. It's also the same reason API gateways enforce authentication rather than trusting every microservice to check its own tokens. Enforcement at the infrastructure layer works even when the application layer is compromised, because it sits outside the blast radius.

Traefik Hub's Triple Gate Architecture (API Gateway, AI Gateway, MCP Gateway with TBAC) operates at the gateway layer. Every agent request, model interaction, and tool invocation passes through these gates regardless of the agent framework, language, or deployment model. The agent cannot modify, disable, or bypass gateway enforcement because the gateway runs in a separate process, in a separate trust boundary, with separate credentials.

This post maps Traefik Hub's capabilities against each of the ten OWASP Agentic risks, with an honest assessment of where gateway-layer enforcement is strong, where it contributes but doesn't fully own the risk, and where other layers need to fill the gap.

ASI-01: Agent Goal Hijack

Risk: Adversary manipulates agent objectives through prompt injection, context manipulation, or goal drift.

This is where the composable safety pipeline earns its keep. The AI Gateway inspects every prompt on the wire, both inbound (user requests) and outbound (model responses), through four guard tiers running in parallel.

Tier 1 (Regex Guard) catches known prompt injection signatures deterministically:

request:
  - jsonQueries:
      - ".messages[].content"
      - ".prompt"
    block: true
    reason: "prompt_injection_detected"
    entities:
      - "(?i)ignore.*(previous|above|prior).*(instructions|prompt)"
      - "(?i)you are now.*(new|different|unrestricted)"
      - "(?i)disregard.*(all|any).*(rules|guidelines|policies)"

Sub-millisecond, zero external dependencies, deterministic. This catches the common patterns.

Tier 3 (NVIDIA Safety NIMs) catches the uncommon ones. Jailbreak Detection NIM identifies prompt manipulation attempts that don't use obvious keywords. Topic Control NIM enforces conversation boundaries based on guideline prompts. These run on GPU and take 30-200ms per NIM, but with parallel execution, the total time equals the slowest guard, not the sum. For the full four-tier pipeline architecture, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Why gateway-layer matters here: A compromised agent cannot disable the gateway's prompt inspection. Application-layer policy checks run inside the same process the attacker is trying to hijack. The gateway inspects the actual payload on the wire, not a representation the agent chooses to expose.

Gap: No behavioral drift detection over time (detecting gradual goal shift across a multi-turn conversation). Each request is inspected independently.

ASI-02: Tool Misuse & Exploitation

Risk: Agent invokes tools beyond intended scope, with unauthorized parameters, or for unintended purposes.

This is where TBAC (Tools/Tasks/Transactions-Based Access Control) operates. The MCP Gateway enforces tool authorization before the request reaches the tool server.

TBAC provides three dimensions of control:

Tools: Which specific MCP tools each agent can invoke based on their identity.
Tasks: Which workflows or sequence of tool calls are allowed.
Transactions: Parameter-level constraints and validation per tool call.

Rate limiting per tool prevents abuse through rapid repeated invocations. Human-in-the-loop approval flows gate high-risk tool calls on explicit human confirmation before execution. Every tool invocation is logged with full identity, parameters, outcome, and timestamp.

Why gateway-layer matters here: TBAC enforces tool authorization at the infrastructure layer, not inside the agent's application code. A capability model defined in application code can be bypassed if the agent is compromised or if a developer misconfigures the SDK. TBAC cannot be bypassed because the MCP Gateway terminates the connection before the tool server receives it. The agent never talks directly to the tool; it talks to the gateway, which decides whether to forward the request.

This is the control we think matters most in the agentic era, and it only works at the infrastructure layer.

ASI-03: Identity & Privilege Abuse

Risk: Agents operate with excessive permissions, impersonate users, or escalate privileges.

JWT identity propagation flows through all three gates. The same identity context that governs API authentication (Gate 1) also governs token budgets (Gate 2) and tool permissions (Gate 3). There are no identity gaps between layers.

Identity can be managed per-user, per-team, and per-API-key enforcement via JWT claims. For fine-grained per-user rate/token/tool limiting, a JWT middleware extracts claims into headers via forwardHeaders, and Traefik middlewares will enforce limiting based on those headers.

TBAC restricts each agent identity to only the tools required for its task. No ambient authority. An agent authorized for read_database cannot invoke write_database even if the MCP server exposes both tools.

Why gateway-layer matters here: Identity verification at the gateway happens before the agent processes the request. Application-layer identity checks depend on the agent framework correctly extracting and honoring identity, which means the agent must be trusted to enforce its own permissions. The gateway enforces identity regardless of whether the agent is trustworthy.

ASI-04: Supply Chain Vulnerabilities

Risk: Compromised agent components, untrusted plugins, tampered models or tool definitions.

Honest assessment: partial coverage.

What the gateway provides today:

Air-gapped deployment. The full Traefik Hub stack runs without external dependencies, eliminating runtime supply chain exposure.
FIPS 140-2 validated, FIPS 140-3 ready. Cryptographic supply chain validated via native Go BoringCrypto module. No external crypto libraries.
Multi-vendor safety pipeline. Each guard tier comes from an independent vendor (custom regex, Microsoft Presidio, NVIDIA, IBM). Compromising one vendor's component doesn't compromise the pipeline.

What the gateway does not provide today: MCP server integrity verification (hash validation of tool definitions), plugin signing, or tamper detection at the gateway layer.

Architectural note: Supply chain integrity for agent internals (application modules, model weights) is inherently a build-pipeline concern, not a gateway concern. The gateway can verify what passes through it, but it cannot verify the internal integrity of components it doesn't serve. MCP server manifest verification (SHA-256 hashing of tool definitions at the gateway) is on our roadmap for Q2 2026. Build-pipeline integrity (Sigstore, SBOM generation, container image scanning) is complementary and should be addressed at the CI/CD layer.

ASI-05: Unexpected Code Execution

Risk: Agents generate and execute code without proper sandboxing or validation.

Assessment: partial coverage.

The gateway provides prevention, not containment. TBAC can prevent an agent from accessing code execution tools entirely. If an agent identity isn't authorized to invoke a run_code tool, the MCP Gateway blocks the request before it reaches the tool server. Parameter validation can constrain what the agent passes to tools that do execute code.

The gateway cannot sandbox execution that happens inside a tool's runtime. If an agent has authorized access to a code execution tool, the gateway doesn't inspect or contain what happens inside that tool after the request is forwarded.

Prevention is the stronger control for most deployments: a tool that cannot be invoked cannot produce unexpected execution. But containment matters too, and it's better addressed by container security contexts, language-level sandboxes (like LangChain's sandboxed code execution), or Kubernetes security policies.

ASI-06: Memory & Context Poisoning

Risk: Adversary corrupts agent memory, conversation history, or RAG context to influence behavior.

Tier 4 (IBM Granite Guardian) provides two capabilities that no other guard tier offers: hallucination detection and RAG quality assessment. The hallucination detection configuration inspects responses using the request history as context:

response:
  systemPrompt: "hallucination"
  useRequestHistory: true
  blockConditions:
    - reason: hallucination_detected
      condition: Contains("yes")

When useRequestHistory is set, the guard evaluates whether the model's response is grounded in the context that was provided. If the model generates claims not supported by the retrieved documents, the guard blocks the response before it enters the agent's memory.

Regex Guard (Tier 1: deterministic pattern matching against known injection signatures) catches known poisoning patterns in both requests and responses. NVIDIA NIMs (Tier 3: GPU-based semantic analysis) catch semantically poisoned content that doesn't match pattern rules. For a full walkthrough of all four tiers and how they compose, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Why gateway-layer matters here: The gateway inspects the model's actual response before the agent processes it. Application-layer memory integrity checks run after the potentially poisoned content has already entered the agent's context window. The gateway catches poisoning at the wire, not after ingestion.

ASI-07: Insecure Inter-Agent Communication

Risk: Agent-to-agent messages lack authentication, encryption, or trust verification.

The MCP Gateway is the communication channel. Every agent-to-tool call passes through it. The gateway terminates the inbound connection, verifies the agent's identity via JWT, applies TBAC policies, and opens a new authenticated connection to the tool server. TLS secures both legs.

You cannot have insecure inter-agent communication when every message passes through an authenticated, encrypted, audited gateway. Application-layer encrypted channels between agents require each agent to correctly implement and maintain encryption, which is a much larger attack surface.

The complete audit trail captures every inter-agent interaction with full identity, tool, parameter, and outcome detail. Traefik also supports request mirroring, which forwards live traffic to an external tool or analysis service in parallel with the primary flow. This enables real-time threat detection and behavioral monitoring alongside the forensic record. Together, these address both prevention (authentication, encryption) and detection (live analysis and forensic audit).

ASI-08: Cascading Failures

Risk: One agent's failure propagates through multi-agent systems causing systemic breakdown.

The gateway is the natural circuit breaker between systems. Three capabilities address this:

Failover Router builds circuit breaker chains across LLM providers; When a primary backend responds with a configured error (429, 500-504), the request replays to a fallback with independent model selection, API keys, and metrics. All safety policies continue to apply on every fallback. Governance doesn't degrade when the model does.

Graceful error handling via onDenyResponse returns structured HTTP 200 refusals instead of HTTP 403 errors. Agents parse refusals as valid conversation turns and continue operating. Without this, a single guardrail block crashes an entire multi-step workflow.

Token rate limiting with proactive estimation blocks over-budget requests before they reach the LLM. This prevents resource exhaustion cascades where one agent's runaway consumption starves others of capacity.

Why gateway-layer matters here: Application-layer circuit breakers fail when the application itself is failing. An agent that is crashing cannot reliably execute its own failure handling logic. The gateway operates outside the failing system.

ASI-09: Human-Agent Trust Exploitation

Risk: Agent manipulates human operators through social engineering or trust exploitation.

Content safety filtering through NVIDIA NIMs (22+ safety categories) filters manipulative, deceptive, or harmful content in model responses before they reach users. IBM Granite Guardian catches factually incorrect claims (hallucinations) the agent might use to mislead operators. Human-in-the-loop approval flows via TBAC require explicit human confirmation for sensitive operations.

Assessment: moderate coverage. The gateway filters manipulative content and gates sensitive actions, but it cannot fully prevent subtle social engineering within otherwise legitimate conversation. This risk also requires organizational controls (training, escalation procedures) and application-layer UX design (clear agent identity disclosure, confidence indicators).

ASI-10: Rogue Agents

Risk: Agents behave in unintended, uncontrollable, or malicious ways outside governance.

Token rate limiting and quota enforcement set hard ceilings on agent resource consumption. Proactive estimation blocks over-budget requests before GPU cost is incurred. TBAC restricts the blast radius to authorized tools only, so even a rogue agent can only reach the tools it was explicitly granted access to. Per-agent rate limiting prevents runaway request loops.

Assessment: moderate coverage. The gateway provides containment ceilings but not real-time behavioral detection. There is no behavioral anomaly detection today: no baseline modeling of normal invocation patterns, no deviation alerting, no agent kill switch. The data for anomaly detection is already being collected in the audit trail; the analysis layer is the gap. Whether that analysis belongs at the gateway layer or in a dedicated observability platform is an open architectural question. What the gateway provides today is the data collection infrastructure that either approach would require.

Coverage Breadth vs. Enforcement Depth

The OWASP Agentic Top 10 is a useful framework. But evaluating governance solutions purely on how many of the ten risks they "cover" misses the more important question: how deeply and reliably is that coverage enforced?

A governance system that checks all ten boxes from inside the agent process provides broad coverage in a narrow trust boundary. A governance system that enforces eight of ten risks from outside the agent's trust boundary provides deep, bypass-resistant coverage on the risks that matter most.

The right architecture uses both layers:

Layer	What It Governs	Examples
Infrastructure (Gateway)	Identity, tool authorization, content safety, traffic resilience, audit	Traefik Hub (API + AI + MCP Gateway)
Application (Agent Framework)	Agent logic, memory management, code sandboxing, behavioral policies	LangChain, CrewAI, Semantic Kernel
Platform (Kubernetes/OS)	Container isolation, network policies, resource limits	Security contexts, gVisor, network policies
Pipeline (CI/CD)	Supply chain integrity, model provenance, image scanning	Sigstore, SBOM, container scanning

No single layer covers all ten risks alone. But the infrastructure layer is the one that works even when the application layer is compromised. For organizations also adopting Forrester's AEGIS framework, we published a companion post mapping AEGIS's six domains to infrastructure-layer enforcement.

Our Coverage Map

OWASP Risk	Gateway Coverage	What Gateway Provides	What Other Layers Add
ASI-01: Goal Hijack	Strong	4-tier safety pipeline, parallel execution	Behavioral drift detection (app-layer)
ASI-02: Tool Misuse	Best-in-class	TBAC: tool auth, parameters, rate limits, human-in-loop	Framework-level tool selection (app-layer)
ASI-03: Identity Abuse	Strong	JWT propagation, per-user/team enforcement, least-privilege	IAM integration (platform-layer)
ASI-04: Supply Chain	Partial	Air-gapped deployment, FIPS crypto, multi-vendor guards	SBOM, image scanning, model provenance (pipeline-layer)
ASI-05: Code Execution	Partial	Tool access prevention via TBAC	Execution sandboxing (platform-layer)
ASI-06: Memory Poisoning	Strong	Hallucination detection, RAG quality, bidirectional inspection	Memory management policies (app-layer)
ASI-07: Inter-Agent Comms	Strong	MCP Gateway, mTLS, authenticated channels, audit	N/A (gateway is the channel)
ASI-08: Cascading Failures	Strong	Failover Router, circuit breakers, graceful errors, token limits	Application-layer retry policies
ASI-09: Trust Exploitation	Moderate	Content safety filtering, human-in-loop, audit	UX design, organizational controls
ASI-10: Rogue Agents	Moderate	Token ceilings, tool blast radius limits, rate limiting	Behavioral anomaly detection (roadmap)

Six strong, two moderate, two partial. We'd rather publish an honest map than claim ten out of ten coverage that doesn't survive architectural scrutiny.

What's Next

The OWASP Agentic Top 10 mapping detailed in this post reflects Traefik Hub v3.20 capabilities as of March 2026. The composable safety pipeline, TBAC, Failover Router, token rate limiting, and graceful error handling are all available in Early Access today, with general availability planned for late April 2026.

We encourage organizations evaluating agent governance to ask every vendor not just which risks they cover, but where their controls are enforced and what happens when the agent itself is the thing that's compromised.

For a complementary framework-level perspective, see our mapping of Forrester's AEGIS framework to infrastructure-layer enforcement. And for a deep technical walkthrough of the composable safety pipeline referenced throughout this post, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

Traefik Hub v3.20 is available as an Early Access release. To try the composable safety pipeline and the other features covered in this post, sign up for Early Access.

Frequently Asked Questions

What is the OWASP Top 10 for Agentic Applications?

The OWASP Top 10 for Agentic Applications is a risk taxonomy published in December 2025 that identifies the ten most critical security risks for autonomous AI agents. The risks range from agent goal hijacking (ASI-01) and tool misuse (ASI-02) to cascading failures (ASI-08) and rogue agents (ASI-10). It is maintained by the OWASP GenAI Security Project and is becoming the standard framework for evaluating AI agent governance solutions.

What is the difference between application-layer and infrastructure-layer AI agent governance?

Application-layer governance runs inside the agent process (as an SDK, middleware library, or framework plugin) and governs the agent's logic. Infrastructure-layer governance runs at the network gateway, outside the agent's trust boundary, and governs the traffic. The key distinction: if the agent is compromised through prompt injection, jailbreak, or privilege escalation, application-layer governance is compromised with it. Infrastructure-layer governance continues to enforce because it runs in a separate process with separate credentials.

What is TBAC (Tools/Tasks/Transactions-Based Access Control)?

TBAC is Traefik Hub's access control model for AI agents, enforced at the MCP Gateway. It provides three dimensions of control: which tools an agent can invoke, which tasks (workflows) it can perform, and what transaction parameters are permitted. TBAC operates at the infrastructure layer, meaning the agent runtime cannot modify, bypass, or disable these controls.

How does the OWASP Agentic Top 10 relate to Forrester's AEGIS framework?

The OWASP Agentic Top 10 is a risk taxonomy (what can go wrong). Forrester's AEGIS framework is a governance framework (what controls to implement). They are complementary: OWASP identifies the risks, AEGIS defines the organizational and technical controls to mitigate them. Both map to infrastructure-layer enforcement. See our AEGIS framework mapping for the governance-framework perspective.

Can any single product cover all 10 OWASP Agentic risks?

No. The ten risks span multiple enforcement layers: infrastructure (gateway), application (agent framework), platform (Kubernetes/OS), and pipeline (CI/CD). A governance system that claims 10/10 coverage from a single layer is either defining "coverage" loosely or operating within a narrow trust boundary. The right architecture uses defense-in-depth, with the infrastructure layer providing the controls that work even when the application layer is compromised.

]]>

When Every File is an Attack Vector: Why Agent Governance Must Live Outside the Runtime

Immánuel Fodor — Wed, 25 Mar 2026 12:25:09 GMT

The LiteLLM supply chain compromise on March 24 was a pip package that stole credentials, such as SSH keys, cloud provider credentials, Kubernetes configs, database passwords, and API keys. That is a serious incident, and the architectural lessons it exposed are significant. But the harder conversation is about what comes next.

AI agents in 2026 do not just call APIs. They read your filesystem. They process your documents, configs, and code. They maintain persistent memory across sessions. They spawn sub-agents. They act on your behalf at machine speed, often without a human reviewing each step.

The LiteLLM attack exploited the Python package supply chain. The next class of attacks will exploit something far larger: the entire input surface of the agent itself.

Your Filesystem Is the New Codebase

When a coding agent operates with filesystem access, every file it reads becomes a potential instruction. A poisoned PDF in a shared folder can carry hidden prompt injections. A modified .env file can redirect agent behavior. A malicious markdown document in a project directory can instruct an agent to exfiltrate data, modify code, or propagate itself to other files the agent touches.

This is not theoretical. Microsoft's security team has documented how indirect prompt injection works in practice: an attacker never talks to the agent directly. Instead, they poison the data sources the agent reads. When the agent retrieves that content through tool calls (emails, documents, support tickets, web pages, database entries), the malicious instructions ride along, invisible to human reviewers, fully legible to the model.

The structural problem is that LLMs process instructions and data as tokens in the same context window. The agent cannot reliably distinguish between a system prompt from its operator and an instruction embedded in a document it just ingested. As researchers in the "Agents of Chaos" study (MIT, Harvard, Stanford, CMU) put it: prompt injection is a structural feature, not a fixable bug.

In a vibe coding workflow, where developers describe intent in natural language and agents generate entire applications, this surface expands further. The agent reads project files, configs, dependencies, and documentation as context for its work. Every one of those files is now part of the instruction set. Every text file can carry a payload. The attack surface is not the chat interface. It is the entire data environment.

Vibe Coding Accelerates the Problem

The rise of vibe coding has made this structural vulnerability operationally urgent.

A CodeRabbit analysis of 470 open-source GitHub pull requests found that AI co-authored code contained 2.74x more security vulnerabilities than human-written code, along with 75% more misconfigurations. Veracode's GenAI Code Security Report found that 45% of AI-generated code introduces security vulnerabilities. These are not edge cases. They are base rates.

The Moltbook incident in February 2026 demonstrated the consequences. The AI-agent social network was built entirely through vibe coding, with no human-written code. Wiz discovered a misconfigured Supabase database exposing 1.5 million authentication tokens and 35,000 email addresses to the public internet. The root cause was not a sophisticated attack. It was the absence of a security review in a process optimized for speed.

There is a pattern here. Coding agents optimize for making code run, not making code safe. Research from Columbia University documented how agents routinely remove validation checks, relax database policies, and disable authentication flows to resolve runtime errors. The constraint causing the error is sometimes the security guard, and the agent removes it to satisfy the prompt.

When agents generate code at this velocity and with these failure modes, treating the agent's runtime as a trusted environment becomes untenable. The code it writes may be insecure. The files it reads may be poisoned. The dependencies it pulls may be compromised (as LiteLLM demonstrated). Every layer of the agent's input and output is potentially adversarial.

The Impersonation Problem

There is a dimension to this that goes beyond credential theft.

An AI agent with access to your filesystem, communication history, code repositories, and working documents has enough context to impersonate you. Not in the crude phishing sense of faking an email header, but in the deeper sense of replicating your communication patterns, decision-making style, and domain knowledge across every platform you use.

A compromised agent does not just steal your credentials. It becomes a distorted version of your digital identity, capable of acting on your accounts, writing in your voice, and making decisions that appear to be yours. The persistent memory that makes agents useful (remembering your preferences, your projects, your colleagues) is the same memory that makes compromise catastrophic. The agent knows enough to be you, and it operates faster than you can intervene.

This is why agent authorization cannot be implemented in the agent's runtime. If the runtime is compromised, every control embedded in it, including identity verification, access policies, and behavioral guardrails, is compromised with it.

Claws Need Shells

The security community is converging on a principle that infrastructure engineers have understood for decades: enforcement must be independent of the thing being enforced.

A firewall does not run inside the application it protects. A network policy does not depend on the container it restricts. In the same way, agent governance cannot live inside the agent runtime. The runtime is what gets compromised, whether through supply-chain attacks (LiteLLM), poisoned context (indirect prompt injection), or insecure generated code (vibe coding failures). Controls that depend on the integrity of that runtime are not controls. They are assumptions.

Defense in depth for AI agents requires multiple independent enforcement layers, each operating in its own trust boundary:

At the API layer: Rate limiting, authentication, and data loss prevention detect anomalous behavior at the network edge. An agent suddenly exfiltrating credential files to an unfamiliar domain triggers enforcement that the agent itself cannot disable.

At the AI interaction layer: Content safety, prompt defense, and PII filtering govern what goes into and comes out of the LLM. These controls run at the infrastructure layer, outside the application process, so they remain intact even when the runtime is compromised.

At the tool access layer: Task-scoped, tool-level authorization ensures that a compromised agent cannot pivot laterally to systems beyond its permitted scope. Authorization is cryptographically attested by an identity provider and enforced at the gateway, not granted by application code that the agent controls.

The critical property is independence. Each layer operates in its own trust boundary. Compromising the agent's runtime (through a poisoned file, a malicious dependency, or a prompt injection) does not disable enforcement at the other layers. This is what separates architecture from aspiration.

The Governance Gap Will Not Close Itself

The data on this is stark. According to Okta's AI at Work report, 91% of organizations are already deploying AI agents, yet only 10% have a well-developed strategy for managing non-human identities. The CrowdStrike 2026 Global Threat Report documented an 89% year-over-year increase in AI-enabled adversary attacks, with average breakout time falling to just 29 minutes and the fastest observed breakout occurring in 27 seconds. In one intrusion, data exfiltration began within four minutes of initial access.

A Dark Reading poll found that 48% of cybersecurity professionals now identify agentic AI as the top attack vector heading into 2026, outranking deepfakes, board-level cyber recognition, and passwordless adoption.

Application-level controls are necessary but insufficient. When the agent's runtime is the attack surface (not just the network or the endpoint, but the code, the context window, and the filesystem), governance must operate at a layer the agent cannot reach. That means infrastructure-layer enforcement: independent of the agent framework, below the application, governing every API call, LLM interaction, and tool invocation as it crosses the network.

Three Questions Every Team Should Ask

If a poisoned file entered your agent's context window, which controls would survive? If every guardrail runs inside the same runtime that processes the file, the answer may be none.
Can your agent disable its own governance? If access control, content safety, and tool authorization are implemented as libraries in the agent's process, a prompt injection or compromised dependency can override them. Infrastructure-layer enforcement operates in a separate trust boundary that the agent cannot modify.
How many independent enforcement layers exist between a compromised agent and your most sensitive systems? The LiteLLM breach showed what happens when the answer is zero. The vibe coding era, where agents generate code, read untrusted files, and spawn sub-agents at machine speed, makes this question existential.

The LiteLLM attack compromised a library. The next wave of attacks will compromise the files agents read, the code agents write, and the memory agents accumulate. The question is not whether your agent will encounter adversarial input. It is whether your architecture survives when it does.

Traefik Hub's Triple Gate Pattern (API Gateway, AI Gateway, MCP Gateway with TBAC) implements the independent, infrastructure-layer enforcement described in this post. For a technical walkthrough of how the composable safety pipeline works across these layers, see From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

]]>

The Agent Framework Wars Are Over. Everyone Won. That's the Problem.

Sudeep Goswami — Tue, 24 Mar 2026 13:34:20 GMT

Ask a platform architect how many cloud providers their organization runs. Watch their expressions. The answer is almost never one, and the reasons are always the same: different teams, different timing, different commitments made before the current strategy existed. The cloud was supposed to consolidate. It has been consolidating for 15 years, and most enterprises are still running more than one.

CI/CD was supposed to standardize. Jenkins still runs in the basement alongside GitHub Actions and whatever the platform team mandated two years ago. The service mesh question, Istio or Linkerd or Consul, was asked in 2019 and is still open at organizations that considered themselves decisive.

This is not a failure of enterprise decision-making. It is how enterprise infrastructure actually works. Commitments compound. Migration costs exceed the value of consistency. New teams make new bets. And the platforms themselves have no incentive to make switching easy, because lock-in is the business model.

Every major agent framework launched in the last twelve months is built on exactly this logic. Which means the governance problem coming at every CISO, Chief AI Officer, and platform architecture leader is not which framework to choose. It is what to do when the answer, inevitably, is all of them.

Twelve Months, Five Frameworks

The agent framework landscape consolidated from "many experiments" to "a few serious platforms" in roughly one year. The platforms that emerged are not startups or community projects. They are flagship products from the five largest technology companies in the world.

OpenAI Agents SDK, released March 2025, is a lightweight Python and TypeScript framework built around a deliberately minimal set of primitives: Agents, Handoffs, Guardrails, and Tracing. Provider-agnostic, supporting over 100 LLMs. The spiritual successor to Swarm, optimized for simplicity over configurability.

Google ADK, introduced at Google Cloud Next 2025, is an open-source framework designed for full-stack development of agents and multi-agent systems. Model-agnostic and deployment-agnostic in principle, optimized for Gemini and Vertex AI in practice. The same framework powering Google's own Agentspace and Customer Engagement Suite.

Microsoft Agent Framework, announced in October 2025, is the convergence of Semantic Kernel and AutoGen into a single, unified runtime that combines Semantic Kernel's enterprise foundations with AutoGen's multi-agent orchestration patterns. Open standards across MCP, A2A, and OpenAPI. Python and .NET. Routes toward Azure AI Foundry.

NVIDIA NemoClaw, unveiled at GTC 2026 last week, is the enterprise extension of OpenClaw, a full agent runtime with hardware-optimized inference, enterprise RBAC, signed skill registries, and the Nemotron model stack underneath. Jensen Huang called it the operating system of agentic computers.

LangChain ecosystem (LangGraph + LangSmith) holds a deep enterprise install base, highly recommended framework for production agents, with over 70 million monthly downloads across the LangChain ecosystem and deployments in production at LinkedIn, Uber, Cisco, BlackRock, and JPMorgan. LangSmith provides observability.

And behind these five: CrewAI, PydanticAI, LlamaIndex, Amazon Bedrock Agents, IBM Watsonx Orchestrate, and a dozen more in active enterprise deployment.

The Consolidation That Will Not Come

The reasonable expectation, looking at previous infrastructure categories, is that the market consolidates. Two or three frameworks win. Enterprises standardize. The governance story simplifies.

That is not what is happening here, and the reason is structural. These frameworks are not neutral tools. They are distribution mechanisms for model consumption, cloud compute, and inference revenue. No platform company has an incentive to make their framework interchangeable with a competitor's. The switching costs are a feature, not a bug.

Enterprises are not going to pick one. A financial services firm running on Azure will use Microsoft Agent Framework for its core workflows. Its data science team, hired from Google, will prototype with ADK. Its NVIDIA DGX cluster will run NemoClaw for latency-sensitive workloads. Its compliance team will insist on LangSmith observability because that is what they already have. Its newest engineering team will ship with OpenAI Agents SDK because it was the fastest to working prototype. The frameworks are too useful, the switching costs are too high, and the organizational inertia is too real. Multi-framework is the destination, not a temporary state.

The Governance Problem Nobody Has Solved

Every one of these frameworks ships with application-layer governance. Guardrails, tracing, access controls, audit logs, safety checks. Each vendor has invested seriously in making their governance story credible.

And every one of those governance implementations is specific to that framework.

OpenAI's guardrails govern OpenAI Agents SDK workflows. They do not govern NemoClaw agents. Google ADK's safety patterns apply inside ADK runtimes. They do not apply to Microsoft Agent Framework deployments. NemoClaw's signed skill registry and RBAC operate inside the NemoClaw runtime. They are unaware of the LangChain agents running alongside them on the same Kubernetes cluster.

As the framework landscape fragments, application-layer governance fragments with it. Each new framework an enterprise adopts adds a new governance silo: new policies to configure, new audit logs to aggregate, new access control models to reconcile with enterprise identity, and new safety configurations to maintain. The governance overhead scales with the number of frameworks, and the coverage is always incomplete at the edges where frameworks interact.

There is a more serious problem underneath this. Application-layer governance, by definition, lives inside the process being governed. When a NemoClaw agent calls an OpenAI endpoint through a LangChain routing layer inside a Microsoft Agent Framework orchestration workflow, a configuration that will exist in production before this year is out, and none of the application-layer governance from any of those frameworks can see the full picture. Each sees only its own slice of the execution. The inference call crosses network boundaries that each framework's internal controls cannot follow.

This is not an edge case. It is the normal operating condition of enterprise multi-agent deployments at scale.

The One Layer That Does Not Fragment

Infrastructure-layer governance does not know which framework built the agent. It governs the traffic.

An inference call from a NemoClaw agent and an inference call from a Google ADK agent are, at the network layer, identical: an HTTP request carrying a prompt to an LLM endpoint. The AI Gateway's safety pipeline, covering pattern matching, PII redaction, NVIDIA Safety NIMs, and hallucination detection, applies to both independently, without modifying either framework.

An MCP tool invocation from an OpenAI Agents SDK workflow and an MCP tool invocation from a Microsoft Agent Framework orchestration are, at the network layer, the same protocol: a tool call with an identity, a method, and parameters. Infrastructure-layer authorization validates both against the same policy, regardless of which runtime initiated the call.

A policy enforced at the network layer applies across the entire multi-framework agent estate. One set of policies. One audit log. One enforcement boundary. The governance overhead does not scale with the number of frameworks because it is not within any of them.

This is what Traefik Hub's Triple Gate architecture was designed to address. Three enforcement points, each targeting a distinct traffic pattern every agent generates regardless of runtime:

The AI Gateway intercepts every inference call: the safety pipeline, token rate limiting, semantic caching, and model failover, which apply universally to every LLM request, whether it originates from NemoClaw, ADK, OpenAI SDK, or LangGraph.
The MCP Gateway intercepts every tool invocation: infrastructure-layer authorization on the specific parameters of each tool call, independent of the role-based controls inside any particular framework's runtime.
The API Gateway governs the management plane: the configuration endpoints, monitoring interfaces, and orchestration APIs of the agent infrastructure itself, enforcing authentication, schema validation, and threat detection regardless of which framework generated the traffic.

Three patterns. Three enforcement points. Framework-agnostic, by design.

Why This Argument Gets Stronger Over Time

In a world with five competing frameworks running within the same enterprise, application-layer governance requires five separate configurations, five separate audit log streams, five separate safety policy maintenance cycles, and still provides no visibility at the boundaries where frameworks interact.

Infrastructure governance provides one. The multi-framework future makes that one more valuable, not less.

The pattern is already established in adjacent infrastructure categories. Enterprises do not maintain separate network security policies for each application stack. They do not run separate identity providers for each platform vendor's tooling. The governance lives at the layer that does not change when the application layer changes.

Agent infrastructure is arriving at the same conclusion. The application layer will continue to fragment: new frameworks, new runtimes, new vendor platforms, new enterprise bets. The infrastructure layer is where governance stabilizes.

The Strategic Question for Every Platform Leader

The question facing every CISO, Chief AI Officer, and platform architecture leader right now is not which agent framework to standardize on. That question has no good answer, because the organization is already running multiple frameworks and will continue to do so.

The question is: where does governance live in a multi-framework agent estate, and how does it scale as the number of frameworks grows?

If the answer is "inside each framework," the honest follow-up is: who maintains five separate governance configurations, how are they kept consistent, and what happens at the boundaries where frameworks interact?

If the answer is "at the infrastructure layer," the follow-up is: which traffic governance architecture covers all three patterns, inference calls, tool invocations, and management APIs, across every framework without requiring modification to any of them?

That second answer is the one that scales. It is also the answer that matches how enterprise security has worked in every infrastructure category that came before.

The agent framework wars are over. Everyone won. The governance story for a world where everyone won is an infrastructure-layer story, and that story is only beginning to be written.

]]>

Ingress NGINX is Done. Here are Your Three Migration Options.

Emile Vauge — Tue, 24 Mar 2026 07:40:59 GMT

Traefik v3.7 brings 80+ NGINX Ingress annotations, snippet support, and a zero-downtime migration path.

Last July, when we first introduced Ingress NGINX native support to Traefik, we knew something was happening, but we didn't yet know how soon the situation would escalate. Then came November: the Kubernetes SIG Network and Security Response Committee officially announced that the Ingress NGINX Controller would be retired in March 2026. No new releases. No bug fixes. No security patches.

As of this month, that deadline has arrived. If you're running Ingress NGINX in production—and millions of clusters still are—you're now running unmaintained software with known architectural vulnerabilities.

Over the past months, we've been on a singular mission: make the migration from Ingress NGINX a non-event. Not a multi-quarter project. Not a forced rearchitecture. A non-event. Today, I want to share where we've arrived, what's new, and why we believe Traefik is the only path that doesn't ask you to choose between safety and disruption.

And the path we've taken has been validated by the industry. This morning, we're proud to announce that IBM Cloud, Nutanix, OVHcloud, SUSE, TIBCO, and others have each independently chosen Traefik as their strategic ingress controller and Gateway API solution. These aren't experiments or evaluations. These are production commitments from organizations that evaluated every option on the table and concluded that Traefik's approach, native Ingress NGINX compatibility combined with full Gateway API support, is the right foundation for their Kubernetes networking stack going forward.

Laying Out Your Three Migration Options

Since the retirement announcement, the ecosystem has responded along three lines. The first two simply don't solve the fundamental issue.

Option 1: Migrate to a Different Configuration Format

Every major proxy vendor (e.g., Kong, HAProxy, NGINX F5, cloud-native providers) has published migration guides. We've read them all. They all share the same fundamental limitation: they require you to rewrite your configurations immediately.

Either you convert your NGINX annotations to a completely different set of vendor-specific annotations, or you jump straight to Gateway API resources. In both cases, every annotation, every routing rule, every customization must be manually converted to a different syntax, a different mental model, a different set of abstractions to land you on a new controller UX that you do not master yet. Under deadline pressure, this is a recipe for outages.

Option 2: Fork Ingress NGINX and Keep Running It

Following the retirement announcement, a maintenance fork emerged to keep the project alive by providing security patches. It's a tempting narrative; just switch the image and stay safe. But as we explained in detail in our analysis, a fork doesn't fix the fundamental architectural flaws. The configuration injection surface that enabled #IngressNightmare isn't a bug to patch. It's a consequence of how NGINX-based controllers work—template rendering, C/C++ memory safety risks, etc.

A fork inherits all of these structural vulnerabilities. And it anchors you to a legacy technology with no path forward. Finally, if the original maintainers gave up, how could a third-party vendor make this project safe to use?

Meanwhile, InGate, the proposed from-scratch successor, has also been announced as end-of-life in March as well. The official Kubernetes path forward for ingress is Gateway API, but Do It Yourself.

Option 3: Use Traefik, the Only Drop-In Replacement

We took a fundamentally different approach. Traefik's NGINX Ingress Provider reads your existing Ingress resources and their nginx.ingress.kubernetes.io annotations natively. No conversion scripts. No YAML rewriting. No dual-maintenance period where you're running two sets of configurations side by side.

This isn't a compatibility shim or a best-effort translation layer. It's a full provider that understands the Ingress NGINX annotation contract and translates it into Traefik's internal object model securely, without the configuration-injection vulnerabilities that plague template-based approaches.

This makes Traefik the only drop-in replacement for Ingress NGINX in the industry. And critically, Traefik is also the only solution that supports both Ingress NGINX annotations and Gateway API on the same cluster, on the same instance, at the same time. This is what makes a truly seamless migration possible, and this is exactly why IBM Cloud, Nutanix, OVHcloud, SUSE, TIBCO, and others chose Traefik as their path forward.

A Data-Driven Expansion of Annotation Support

From the beginning, we took a data-driven approach to deciding which annotations to support and in what order. Our open-source ingress-nginx-migration tool doesn't just analyze your cluster, when users opt in, it shares anonymized, aggregated statistics with us. No resource names, no namespaces, no configurations. Just annotation counts and usage patterns.

The result: we now have an excellent overview of real-world annotation usage across hundreds of diverse deployments, from small startups running a handful of Ingresses to large enterprises with thousands. This data has directly shaped our implementation priorities, ensuring that every annotation we add addresses actual production needs, not hypothetical edge cases.

Introducing Traefik v3.7: 80+ NGINX Ingress Annotations, 90% of Real-World Usage, Available Now

Today we are releasing Traefik v3.7 early access, and it's the biggest leap forward yet for Ingress NGINX migration. This release brings support for over 80 NGINX Ingress annotations natively, covering more than 90% of real-world usage patterns we've observed across hundreds of production clusters. Authentication, TLS, session affinity, CORS, routing, proxy settings, rate limiting, canary deployments, snippets, and more: they all work out of the box, with your existing Ingress manifests, unchanged.

This is the release that turns the Ingress NGINX sunset from a migration project into a configuration change.

Snippets: A Pragmatic Solution to the Long Tail

One of the most interesting findings from our migration tool data was the prevalence of configuration-snippet and server-snippet annotations. These are the escape hatches of the Ingress NGINX world (i.e., raw NGINX configuration directives injected directly into the generated config). They're also the primary vector behind the IngressNightmare vulnerabilities, and the reason a naive "just support all annotations" approach is a security non-starter.

But we can't ignore them either. Our data shows that snippets are heavily used in production, and a significant portion of "unsupported" Ingresses rely on them.

So we built something new: partial snippet support through a basic interpreter. Rather than blindly injecting raw NGINX configuration, which would recreate the exact security vulnerabilities we're trying to avoid, Traefik now parses snippet content and interprets a curated set of the most commonly used NGINX directives.

This covers the most common snippet use cases without enabling arbitrary configuration injection. It's a pragmatic middle ground: broad enough to unblock real migrations but strict enough to maintain Traefik's security-by-design guarantees.

Variable Interpolation

Alongside snippet support, Traefik now supports variable interpolation, initially focused on the headers manipulation use case. If your snippets use NGINX variables to set or modify request and response headers—referencing client IPs, URI components, or upstream information—Traefik maps them to their equivalent values in its own routing context.

This was another priority directly informed by our migration data. Header manipulation through snippets with variable interpolation is one of the most common patterns we observed, and without this support, even a well-parsed snippet would produce incorrect behavior. The architecture is designed to expand variable coverage to additional use cases based on community feedback.

Annotation Support: The Full Picture

Here's a snapshot of the 80+ NGINX Ingress annotations now supported by Traefik's NGINXIngress Provider, as available in the v3.7 early access.

For the full technical reference with behavioral notes and limitations, see the NGINX Ingress Provider documentation.

Authentication auth-type auth-secret auth-secret-type auth-realm auth-url auth-method auth-response-headers auth-signin auth-snippet auth-tls-secret auth-tls-verify-client auth-tls-verify-depth auth-tls-pass-certificate-to-upstream	SSL / TLS ssl-redirect force-ssl-redirect ssl-passthrough proxy-ssl-secret proxy-ssl-verify proxy-ssl-name proxy-ssl-server-name	Session Affinity affinity affinity-mode affinity-canary-behavior session-cookie-name session-cookie-path session-cookie-domain session-cookie-samesite session-cookie-secure session-cookie-max-age session-cookie-expires
Proxy Settings proxy-body-size client-body-buffer-size proxy-buffer-size proxy-buffers-number proxy-buffering proxy-max-temp-file-size proxy-connect-timeout proxy-read-timeout proxy-send-timeout proxy-request-buffering proxy-http-version proxy-redirect-from proxy-next-upstream proxy-next-upstream-timeout proxy-next-upstream-tries	CORS enable-cors cors-allow-credentials cors-allow-headers cors-allow-methods cors-allow-origin cors-max-age cors-expose-headers	Routing & Redirects use-regex rewrite-target app-root permanent-redirect permanent-redirect-code temporal-redirect temporal-redirect-code from-to-www-redirect server-alias
Load Balancing & Backend load-balance backend-protocol service-upstream upstream-vhost upstream-hash-by default-backend	Rate Limiting & Access Control whitelist-source-range allowlist-source-range limit-rps limit-rpm	Custom Headers custom-header x-forwarded-prefix
Canary Deployments canary canary-by-header canary-by-header-value canary-by-header-pattern canary-by-cookie canary-weight canary-weight-total	Error Handling custom-http-errors	Security / WAF enable-modsecurity modsecurity-snippet
Observability enable-access-log	Snippets configuration-snippet server-snippet

Many of these annotations were implemented by community contributors, and that momentum continues to grow. If you need support for annotations not listed here, or want to help expand coverage further, head to traefik/traefik#12631. Annotations are categorized by complexity (Basic, Medium, Complex), making it easy to find a contribution that matches your experience level.

More than half of Traefik's merged pull requests have historically come from external contributors. The Ingress NGINX Provider is proving to be no exception.

The Migration: Three Phases, Zero Downtime, Zero Risk

We designed the migration from Ingress NGINX to be a simple, three-phase process. All three phases uphold our core principles: zero downtime, progressive transition, and a smooth experience that lets you move at your own pace.

Phase 1: Assess Your Situation

Run the ingress-nginx-migration tool against your cluster. In minutes, you'll get a clear picture of your annotation usage and compatibility status: which Ingresses will migrate seamlessly, and which ones need attention. No manual audits are required.

Phase 2: Ingress NGINX Decommissioning

Deploy Traefik alongside your existing NGINX controller. Both will serve the same Ingress resources simultaneously. Progressively shift traffic to Traefik, one Ingress at a time, validating behavior with each step. Once you're confident, remove Ingress NGINX. There is no big-bang cutover, no downtime, and your production traffic remains secure while you eliminate the unmaintained software from your stack.

Phase 3: Progressive Transition to Gateway API

With Traefik handling your existing Ingress resources and their NGINX annotations, you're no longer under a deadline. When you're ready (not when a crisis forces you), you can begin converting individual Ingress resources to Gateway API. Traefik supports both simultaneously, allowing you to migrate one route at a time, validate behavior, and roll back if needed. This is a seamless, progressive modernization with zero downtime.

This three-phase approach is unique to Traefik. No other solution provides both a drop-in replacement for Ingress NGINX and a production-grade Gateway API implementation that runs side by side, enabling a fully progressive migration from legacy to modern at your own pace.

The retirement of Ingress NGINX doesn't have to be a crisis. We built a migration path for you because we believe no one should be forced to rearchitect under pressure. The Kubernetes ecosystem deserves better than that.

None of this would have been possible without the incredible contributions from the community. Developers from around the world have submitted pull requests adding new annotations, reporting edge cases, improving documentation, and testing the provider against their own production configurations. This effort is as much yours as it is ours, and we're deeply grateful for it.

If you haven't already, we invite you to try the Traefik v3.7 early access release against your own cluster. Run the migration tool, deploy Traefik alongside NGINX, and see for yourself how many of your Ingresses just work. Your feedback on this release will directly shape the final v3.7 GA.

The deadline is here. But for Traefik users, it's just another Tuesday.

Resources:

Migration tool: github.com/traefik/ingress-nginx-migration
Migration guide: doc.traefik.io/traefik/migrate/nginx-to-traefik
Early Access: Traefik v3.7 Early Access: v3.7.0-ea.2
Contribute annotations: traefik/traefik#12631
Full annotation reference: doc.traefik.io/.../ingress-nginx
Migration website: ingressnginxmigration.org
Why a fork won't save you: The Illusion of Safety

]]>

Hardware Has Never Cost More. Is Your Architecture Built for That Reality?

Sudeep Goswami — Thu, 19 Mar 2026 15:55:44 GMT

At NVIDIA GTC this week, the chairman of SK Group, which controls the world's largest supplier of high-bandwidth memory, told reporters that the global chip wafer shortage will likely persist until 2030. Not this year. Not next year. 2030.

The wafer supply deficit is running above 20% industry-wide. New fabrication capacity takes four to five years to come online. The root cause is not a temporary demand spike but a structural reallocation: AI infrastructure requires high-bandwidth memory at a scale that is pulling cleanroom capacity away from the standard DRAM and NAND that conventional enterprise servers depend on.

This is the environment in which enterprise IT leaders are operating for the foreseeable future. Hardware costs more. Hardware takes longer to arrive. And the old economics of traditional procurement, buying more capacity when you need more performance, no longer hold.

None of that changes the fact that compute, storage, and networking remain the foundation of every system. You cannot run workloads without them. But when that foundation costs more and takes longer to arrive, every layer of the stack that consumes hardware inefficiently becomes a liability.

Which raises the question every CIO should be asking: where in your architecture are you still consuming hardware for something software can do better?

The Case for Virtualization Has Never Been Stronger

Virtualization has been extending the life and efficiency of hardware for over two decades. What is new is the urgency. When DRAM prices surge nearly 95% in a single year and wafer supply is projected to stay constrained through the end of the decade, the cost of inefficiency at every layer of the stack compounds rapidly.

The question is no longer whether or not to virtualize. It is whether or not you have pushed virtualization far enough up the stack.

Most enterprises have virtualized compute. Many have virtualized storage. A growing number have virtualized networking at the infrastructure layer. But one layer consistently lags: application delivery.

Load balancers. Web application firewalls. API gateways. These functions still run on dedicated hardware appliances or on virtual appliances that behave like hardware and carry hardware-level cost, in the majority of enterprise environments. In a world where every rack unit needs to justify itself, that gap is no longer defensible.

What a Software-Defined Gateway Actually Delivers

A traditional application delivery architecture places purpose-built appliances at fixed points in the traffic path. One device handles external load balancing. Another manages internal routing. A third handles API authentication. A fourth inspects traffic for threats. Each is a capital expense, a configuration boundary, a failure domain, and a refresh cycle.

A software-defined gateway collapses all of those discrete points into a single programmable control plane. Whether running inside a VM on an existing virtualization platform or natively within a container environment, traffic is handled end-to-end by software: routing, TLS termination, authentication, rate limiting, circuit breaking, and observability. Policy is expressed as code. It is version-controlled, reproducible, and auditable. No firmware update or vendor support call required.

The performance argument for dedicated hardware has been closed for years. Modern software-defined gateways running on standard infrastructure handle enterprise traffic at scale with the observability and programmability that hardware appliances were never designed to provide.

The remaining argument for hardware appliances is inertia, not capability.

Portable vs Proprietary

Not all software-defined gateway solutions are equal, and the distinction matters more today than ever.

Some virtual appliances are software in name only. They run inside a specific hypervisor, require a particular vendor's control plane, or are licensed as part of a broader virtualization platform. These tools reduce hardware costs at the margin, but they substitute one form of lock-in for another. The procurement burden shifts; the architectural constraint does not.

The value of a truly software-defined gateway comes from its portability. A gateway that is environment-agnostic, hypervisor-agnostic, and location-agnostic runs identically whether deployed as a VM on an existing virtualization platform, as a workload inside a container environment, in a private data center, in a colocation facility, or in an air-gapped deployment with no external connectivity. Configuration is portable. Policy is portable. Observability is portable. The platform beneath you can change, and in 2026, platforms do change, without requiring you to re-architect the application delivery layer on top.

The right frame is not hardware versus software. It is portable versus proprietary.

Two Forcing Functions Arriving at the Same Time

The hardware economics argument is compelling on its own. But for much of the enterprise market, it is arriving alongside two platform-level disruptions that make the application delivery decision not just strategic but immediate.

VMware Migrations

The Broadcom acquisition of VMware has triggered one of the most consequential infrastructure re-evaluation cycles in a generation. Enterprises that built their virtualization strategy around vSphere and NSX are now reassessing licensing costs, platform dependencies, and long-term roadmaps. Many are moving workloads to alternative virtualization platforms. Some are accelerating their shift to containerized environments. All of them are asking the same question: which components of our stack do we carry forward, and which do we replace with something better?

VMware-bundled load balancing and gateway capabilities are, by definition, VMware-dependent. An enterprise migrating off VMware cannot lift those components and drop them into a new environment. It needs an application gateway that is platform-independent from the ground up. That is an architectural requirement, not a preference.

Ingress NGINX End-of-Life

For enterprises running containerized workloads, a parallel forcing function is already underway. Ingress NGINX, the most widely deployed Kubernetes ingress controller in production, announced end-of-life. This is not a minor dependency update, it is an architectural decision point.

The ingress layer is where external traffic enters, TLS terminates, routing decisions are made, and API governance begins. Replacing it isn’t a migration task, it means deciding what the next architecture looks like. Organizations that take the opportunity to rearchitect will come out with a gateway layer that is more capable, more portable, and substantially less expensive than what they are replacing.

Both forcing functions are active simultaneously. Each, on its own, is a sufficient reason to revisit the application delivery layer. Together, they represent a generational consolidation opportunity.

Migrate. Modernize. Transform.

Enterprises are currently in all three phases of infrastructure evolution simultaneously:

They are migrating: off VMware, off Ingress NGINX, off legacy appliances that cannot follow workloads to new environments.
They are modernizing: replacing hardware-dependent components with software-defined alternatives that are portable, programmable, and environment-agnostic.
They are transforming: deploying AI-powered applications, agent workflows, and model inference infrastructure that require governance capabilities hardware appliances were never designed to provide.

The risk in navigating all three individually is fragmentation. Each transition, approached in isolation, produces a new policy boundary, a new configuration silo, and a new governance gap. Audit risk compounds. Policy drift accumulates. Operational drag becomes structural.

The answer is a single software-defined gateway that follows workloads through migration, enforces consistent policy through modernization, and extends to govern AI traffic through transformation. One control plane. Consistent policy. No hardware in the critical path.

That control plane also serves as the enforcement point for AI governance: inspecting model requests before they reach an endpoint, enforcing authentication on agent calls, rate-limiting inference traffic by identity, and routing to different model versions based on policy. None of this requires a new appliance. It is a configuration change on the same gateway already managing the rest of your application traffic.

See It in Production

At KubeCon EU in Amsterdam next week, the Traefik Labs team will demonstrate exactly how this architecture operates in practice: software-defined ingress, API governance, and AI traffic management running on a single, environment-agnostic, hypervisor-agnostic, and location-agnostic control plane, with no hardware appliances in the critical path.

If you are working through a VMware migration, an ingress modernization, or the infrastructure layer for AI applications, we would welcome the conversation. Our team will be at Booth #981.

The hardware constraint is not going away before 2030. The enterprises that use this moment to unify their migration, modernization, and transformation onto a single portable gateway will carry a structural advantage that outlasts the current supply cycle.

]]>

OpenClaw is Having Its Enterprise Moment, But Application-Layer Governance with NemoClaw Isn't Enough

Sudeep Goswami — Wed, 18 Mar 2026 16:33:31 GMT

There is a specific kind of moment in enterprise technology that repeats itself with uncomfortable regularity. Something built for individuals gets handed to organizations, and nobody has figured out the governance story yet.

It happened with the cloud. The moment Amazon launched EC2, developers started spinning up servers on personal credit cards. It took years for the enterprise to catch up on IAM policies, cost governance, and audit trails. The capability arrived long before the controls did.

It happened again with SaaS. By the time IT leaders understood what Dropbox was doing on their networks, half the organization had already stored sensitive documents on personal accounts. Shadow IT emerged as a category because the tools moved faster than the institutions.

That moment is here again with AI agents. And the signal worth watching most closely right now is what just happened to OpenClaw.

From Developer Favorite to Enterprise Infrastructure

OpenClaw started as an open-source personal AI agent gateway, a tool that lets individual developers connect chat applications to AI models and local tools. It caught fire. 247k+ GitHub stars. 47k+ forks. The kind of organic adoption that does not happen because of marketing, it happens because engineers found something genuinely useful and started sharing it.

The creator, Peter Steinberger, joined OpenAI in early 2026. And then NVIDIA did something that tells you everything about where agentic AI is headed.

At GTC 2026, Jensen Huang unveiled NemoClaw, describing it as "30 years of NVIDIA computing, distilled into an agent platform." NemoClaw builds on what OpenClaw built for individuals and adds enterprise-grade architecture on top: multi-agent orchestration, hierarchical task delegation, signed skill registries, role-based access controls, audit logging, and deterministic logic for verifiable tool call sequences. It supports the Model Context Protocol natively, connects to the Agent-to-Agent protocol, runs on Nemotron models at 30B, 120B, and 500B parameter scale, and integrates with Salesforce, Cisco, Adobe, Google Cloud, and CrowdStrike out of the box.

This is not a library. Not a framework. It is the operating system of agentic computers — open-source under Apache 2.0, with a paid tier for managed infrastructure and compliance tooling.

This is the enterprise moment. And it changes what every CISO, Chief AI Officer, and platform architecture leader needs to be thinking about right now.

What NVIDIA Got Right

The governance capabilities built into NemoClaw are real and meaningful.

Enterprise role-based access controls on agent identities, so the finance agent and the customer support agent do not share credentials or capability sets. Signed skills, so the tools available to agents cannot be silently modified by a supply chain compromise. Step-level policy checks before every tool invocation, so an agent cannot take an action the platform has not explicitly authorized. Comprehensive audit logging at the orchestration layer.

These are not checkbox compliance features. They represent serious architectural thinking about what makes agentic AI deployable inside organizations with actual security programs.

NemoClaw's launch confirms what practitioners have been arguing for two years: enterprise agent governance is a mainstream requirement, not a niche concern. NVIDIA, with all its platform credibility and hardware distribution power, is now building it into a flagship product. That is a meaningful signal.

But the existence of application-layer governance does not imply complete governance. That distinction is where the real architectural question begins.

One Governance Layer Is Never Enough

The foundational principle of enterprise security is Defense-in-depth: no single control is trusted to stand alone. Every meaningful security architecture deploys independent enforcement at multiple layers, so that a failure, bypass, or compromise at one layer does not collapse the entire posture.

This principle predates AI agents by decades. It is why organizations run both endpoint detection and network monitoring. It is why zero-trust architectures do not simply trust identity at login and then grant full access. It is why the firewall analogy holds so cleanly: application firewalls understand business logic and enforce policy with context, but network firewalls enforce at a layer the application cannot reach. The answer has never been one or the other. It has always been both.

NemoClaw's governance is application-layer governance. Its guards operate inside the agent runtime, coupled with the agent itself. The access control policies, the signed skill checks, the deterministic logic engine, the audit logs, all of these run within the same process that is being governed. A compromise to the agent also compromises the guards that are governing it. This is not a design flaw. It is the nature and limitation of tightly coupled application-layer controls, and it is exactly what Defense-in-depth says is necessary but not sufficient.

Consider what happens when a NemoClaw agent makes an inference call to a Nemotron model. The request leaves the agent runtime and travels across the network to an endpoint. At that moment, one cannot be sure whether NemoClaw's governance allowed it because it's a legitimate request or because its governance was also compromised along with the agent. Its governance might allow the payload to contain PII from a customer database. It might be a prompt injection payload embedded in a tool response. Also, application-layer controls cannot enforce a token budget at the network layer before GPU cycles are consumed. It cannot verify that the specific parameters in an MCP tool call, not just the tool class, but the actual values, conform to the organization's security policy.

This is not a gap unique to NemoClaw. It is inherent to any system that governs from the inside out. And it is precisely the gap that the defense-in-depth principle exists to close.

The Triple Gate: Defense-in-Depth Made Concrete

Applying Defense-in-depth to NemoClaw deployments means identifying the distinct traffic vectors generated by agents and placing independent enforcement at each.

NemoClaw agents generate three kinds. Every inference call to a language model. Every MCP tool invocation. Every request to the agent management and orchestration APIs. Three vectors, each carrying different risks, each requiring its own enforcement point.

This is the architecture Traefik Hub's Triple Gate Pattern addresses directly.

The AI Gateway sits in front of every inference call a NemoClaw agent makes, to Nemotron, to OpenAI, to Anthropic, to any model endpoint. It runs a four-tier safety pipeline: pattern-matching for injection signatures and PII formats, real-time PII redaction, NVIDIA Safety NIMs for topic control and jailbreak detection, and response-side hallucination scoring. This pipeline runs at the traffic layer, independently of NemoClaw's internal guardrails. If the agent runtime is compromised, the AI Gateway continues to run. It also enforces token rate limits per agent, per team, and per API key, a hard network-layer stop that operates before GPU resources are allocated.

The MCP Gateway sits in front of every tool invocation. NemoClaw's role-based controls determine which tools an agent role is permitted to use. The MCP Gateway enforces at the parameter level, the specific values being passed, the specific data being accessed, the specific operations being requested, against infrastructure-layer policy that exists independently of the agent runtime. This is the enforcement layer that catches tool-call escalations even when the tool call itself is legitimately authorized. Refusals are returned as structured responses that NemoClaw agents handle gracefully, without crashing workflows.

The API Gateway governs the NemoClaw management plane itself, the agent configuration endpoints, monitoring interfaces, and orchestration APIs that constitute the control surface of any NemoClaw deployment. These endpoints are attack surfaces in their own right. The API Gateway enforces authentication, rate limiting, schema validation, and WAF rules against the OWASP threat model, independent of anything running inside the NemoClaw runtime.

Three gateways. Three enforcement points. Each layer is independent. Each layer covers attack surfaces that the others do not reach. This is Defense-in-depth applied to the specific topology of enterprise agent deployments.

Four Scenarios That Illustrate the Need for Defense-in-Depth

Defense-in-depth only has value if it addresses real attack vectors. These four scenarios are representative of what enterprise deployments will face.

1. Prompt Injection via Tool Response

An agent calls an external tool and receives a response. Embedded in that response is a carefully crafted instruction designed to redirect the agent's next action. NemoClaw processes this at the application layer after it has already been ingested by the runtime. The AI Gateway intercepts the tool response before the agent sees it, running it through the safety pipeline at the traffic layer. The injection never reaches the runtime.

2. Token Drain via Recursive Calls

A misbehaving agent triggers recursive inference loops. NemoClaw's resource budgets are soft limits enforced inside the runtime. If the runtime is in a problematic state, those budgets may not hold. The AI Gateway's token rate limiting is a hard stop at the network layer, applied before inference requests reach the GPU cluster. No compute is consumed, regardless of the runtime's state.

3. MCP Tool Escalation via Parameter Crafting

An agent constructs a tool call in which the tool itself is authorized under its role, but the parameters are crafted to exceed the intended scope. NemoClaw's role-based controls authorize the tool class. The MCP Gateway validates the specific parameters against infrastructure-layer policy at the network layer, independently of the runtime. The escalation attempt fails at the enforcement point closest to the tool.

4. PII Leakage in Inference Responses

A model response contains customer personally identifiable information that was present in the context window but was not intentionally requested. NemoClaw logs the interaction after the fact. The AI Gateway operates bidirectionally on the traffic itself, redacting PII in both requests and responses before they move further in the chain. Whether the agent requested the data or received it incidentally, anything that crosses the network gets inspected and sanitized before it travels further.

In each scenario, the enforcement point that stops the incident is independent of the agent runtime. That independence is exactly what Defense-in-depth requires.

The Architecture Choice That Only Gets Made Once

When enterprise leaders evaluate NemoClaw deployments, the real decision is not which agent platform to use. NemoClaw is genuinely compelling and will see serious adoption. The decision is whether to deploy it with only application-layer governance or with the full defense-in-depth posture demanded by the threat landscape.

That choice compounds over time. Teams that build governed agent infrastructure now, with the agent runtime owning application-layer governance and the Triple Gate owning traffic governance, will move faster as the agent landscape expands. Their security posture does not depend on any single component being uncompromised. Their compliance story holds up to auditors who understand Defense-in-depth. Their cost controls operate at the network layer, where they cannot be bypassed by a misbehaving agent.

Teams that deploy with only application-layer governance will eventually encounter an incident that exposes the gap. Retrofitting infrastructure controls into a production system under pressure is the most expensive way to build them.

What This Means for Platform Decisions

For security leaders evaluating agent deployments, the question to put to architecture teams is direct: does our governance posture reflect Defense-in-depth, or does it rely on a single enforcement layer inside the agent runtime? If the runtime is degraded or compromised, what remains?

For Chief AI Officers enabling agentic workflows at scale, the infrastructure layer is what enables scalable governance without becoming a bottleneck. A policy enforced at the network layer applies to every agent, regardless of which framework built it, which runtime runs it, or which vendor's model it calls. That universality is the difference between governance that scales and governance that requires a new exception for every new deployment.

For platform architecture leaders, the Triple Gate model maps cleanly to a separation of concerns that infrastructure teams already understand. The agent platform team owns the runtime governance. The platform infrastructure team owns the traffic governance. Neither depends on the other to do its job, and a compromise of one does not take down both.

The Broader Signal

NVIDIA building enterprise governance into NemoClaw confirms that the demand for governed agents is now mainstream enough to justify a platform-level investment from one of the most capital-efficient organizations in technology history. Jensen Huang does not build products for markets that do not exist.

What NemoClaw does not include, the API gateway in front of agent endpoints, the safety pipeline on inference traffic, the infrastructure-level MCP authorization, the token rate limiting at the network layer, is not an oversight. It is a scope decision. NVIDIA's business is silicon and model infrastructure. Agent governance at the traffic layer is a different problem, one that requires deep integration with Kubernetes ingress, API gateway protocols, and network-layer traffic management.

That scope decision is not a gap NVIDIA needs to fill. It is the natural boundary where the agent runtime ends and the infrastructure governance layer begins — the boundary where defense-in-depth becomes operational.

The organizations that understand this, that see NemoClaw's application-layer governance and Traefik's Triple Gate as two halves of a complete defense-in-depth posture, are the ones that will deploy AI agents at enterprise scale without discovering their assumptions were wrong after the fact.

OpenClaw had its enterprise moment at GTC 2026. Defense-in-depth is what turns that moment into a durable foundation.

To see how Traefik Hub's Triple Gate architecture provides infrastructure-level governance for NemoClaw deployments, see this blog post.

]]>

The McKinsey Breach Was SQL Injection. The Real Threat Was 95 Writable System Prompts.

Zaid Albirawi, Immánuel Fodor — Tue, 17 Mar 2026 17:47:53 GMT

An autonomous AI agent breached one of the world's most prestigious consulting firms in two hours. The vulnerability was 20 years old. The implications are brand new.

The Attack Chain

On March 9, 2026, security startup CodeWall disclosed that its autonomous offensive AI agent had achieved full read-write access to McKinsey's internal AI platform, Lilli, in under two hours. The agent operated with no credentials, no insider access, and no human in the loop.

The attack chain was startlingly simple. Of Lilli's 200+ API endpoints, 22 required no authentication whatsoever. The agent discovered that while user input values were properly parameterized in SQL queries (the standard defense against injection), the JSON field names were concatenated directly into SQL without sanitization. When database error messages began reflecting live production data, the agent recognized a classic error-based SQL injection vector and began extracting data iteratively.

The result: access to 46.5 million plaintext chat messages covering strategy, M&A, and client engagements. 728,000 files, including PDFs, Excel spreadsheets, and PowerPoint decks. 57,000 user accounts. 266,000+ OpenAI vector stores. 3.68 million RAG document chunks. And 95 system prompts across 12 model types.

All writable.

McKinsey patched the vulnerabilities within hours of disclosure, engaged a third-party forensics firm, and stated that no evidence of unauthorized client data access was found. CodeWall operated under McKinsey's public HackerOne responsible disclosure program. The response was swift and professional.

But the structural lessons are too important to let pass quietly.

This Is Not a Data Breach Story

Every headline focused on the data: 46.5 million messages, 728,000 files, the sheer volume of sensitive information exposed. Those numbers are staggering, and they deserve attention. But they are not the most important part of this story.

The most important detail is that 95 system prompts were writable.

Lilli is used by over 40,000 McKinsey consultants, and processes more than 500,000 prompts per month. Consultants rely on it for strategy research, competitive analysis, M&A evaluation, and client recommendations. The system prompts define how Lilli thinks: what it recommends, what it refuses, how it cites sources, and what guardrails it follows.

As CodeWall put it in their report: "No deployment needed. No code change. Just a single UPDATE statement wrapped in a single HTTP call."

A threat actor with write access to those prompts could have silently rewritten how Lilli frames competitive landscapes, evaluates acquisition targets, or assesses risk. The poisoned output would flow directly into deliverables for Fortune 500 clients. No one receiving the advice would know it had been tampered with.

This is not a data breach. It is a supply chain attack vector for corporate decision-making itself.

And it gets worse. The same vulnerability exposed 266,000+ OpenAI vector stores and 3.68 million RAG document chunks. These are the knowledge bases that Lilli retrieves from and synthesizes when answering questions. An attacker with write access to both system prompts and RAG stores could manipulate not just how the AI reasons, but also the source material it draws from. The poisoning would be nearly undetectable.

The Root Cause Is Architectural

It is tempting to reduce this breach to a single attack vector: SQL injection, one of the oldest vulnerability classes in web security. And yes, it is remarkable that a production AI platform serving 40,000 users in 2026 shipped with an OWASP Top 10 vulnerability. But blaming the developers misses the point.

McKinsey's team did the standard thing. They parameterized user input values in their SQL queries. They followed the textbook. What they missed was that JSON field names were also being concatenated into SQL, an unusual injection vector that standard scanners like OWASP ZAP do not typically test for.

The real failure is not that a developer missed an edge case. The real failure is that the architecture had no independent layers of defense between the internet and the production database. No request inspection at the gateway. No authentication on 22 endpoints. No content safety checks on the AI pipeline. No access governance for agents or automated callers.

The application was the only line of defense. When it failed, everything behind it was exposed.

This is the architectural lesson: application-level security is necessary but never sufficient. You cannot rely on every developer getting every edge case right across every endpoint in every release. You need independent enforcement at the infrastructure layer, inspecting and governing traffic before it ever reaches application code.

Defense in Depth for the AI Era: The Triple Gate Pattern

At Traefik Labs, we have been building toward this exact threat model. The Triple Gate Pattern is an architecture for defense in depth across the full AI execution path: from HTTP requests to LLM interactions to agent tool calls. Three independent gates, each operating on different principles, each enforcing policy at the infrastructure layer, independent of the application runtime.

Diagram: The Triple Gate Pattern

Let's walk through the McKinsey attack chain and show how each gate addresses a specific class of failure.

Gate 1: API Gateway—Authentication, WAF, and Runtime API Governance

The first gate governs all HTTP traffic entering your infrastructure. It enforces authentication, authorization, rate limiting, schema validation, and web application firewall (WAF) protection.

This is where the McKinsey breach would have been stopped cold.

Traefik Hub's API Gateway natively integrates the Coraza WAF with OWASP Core Rule Set (CRS) support. The CRS is a community-maintained, battle-tested collection of attack detection rules that has closed over 500 rule bypasses through its own bug bounty program. SQL injection detection is one of its most mature capabilities.

CRS inspects the entire HTTP request payload for SQL injection patterns. It does not care whether the injection point is in a query parameter, a JSON value, or a JSON key. It scans the full request. McKinsey's developers missed the JSON key vector because they were thinking about parameterization at the application layer. A WAF at the gateway layer does not make that distinction. It would have flagged the injection pattern and blocked the request before a single byte reached the application.

And this is not a theoretical claim. SQL injection accounts for roughly 65% of all web application attacks. It is the single most common attack class on the internet. The OWASP CRS exists precisely because you cannot trust application code alone to catch every variant.

But WAF is only one capability within Gate 1. Even if an exotic injection variant somehow bypassed the WAF rules, the API Gateway enforces authentication and authorization on every endpoint. McKinsey's 22 unauthenticated endpoints would never have existed in this architecture. No valid identity token, no access. Zero trust at the API layer means every request is scoped, every call is auditable, and every endpoint is protected regardless of what the application code does or does not validate.

Traefik Hub's native Coraza integration runs at high performance because it is compiled directly into the gateway binary. No sidecar container. No separate appliance. No additional network hop. For organizations currently running ModSecurity as a separate layer in front of their ingress controller, this is a consolidation story: the WAF, the API gateway, and the ingress controller converge into a single platform with a single control plane.

Gate 1 alone would have prevented the McKinsey breach entirely. But defense in depth means you never rely on a single gate.

Gate 2: AI Gateway—Content Safety, Guardrails, and Cost Controls

The second gate governs the AI-specific traffic: LLM prompts, model responses, and the content flowing through the inference pipeline.

Even if an attacker bypasses Gate 1 and reaches the AI layer, Gate 2 provides independent enforcement. The AI Gateway sits between callers and the LLM backend, inspecting both inbound prompts and outbound responses against configurable safety policies.

In the McKinsey scenario, the most dangerous outcome was not data exfiltration. It was the ability to rewrite system prompts. A poisoned system prompt would change how Lilli responds to every query from every user. But if an AI Gateway with content-safety guardrails is inspecting every response, the poisoned outputs would need to survive those checks on every single inference call. A prompt rewritten to inject biased analysis, suppress certain topics, or leak confidential data would trigger content safety, topic control, or jailbreak detection guardrails on the response path.

With Traefik Hub v3.20, announced yesterday at NVIDIA GTC, the AI Gateway introduces a composable, multi-vendor safety pipeline with parallel guard execution. This means organizations can chain multiple high-latency guardrail providers (NVIDIA Safety NIMs, IBM Granite Guardian) and execute them in parallel rather than sequentially. The architectural insight is that safety layers must be composable and multi-vendor. No single guardrail provider catches everything. Parallel execution means you get multi-vendor coverage without the latency penalty.

Token-level cost controls, also new in v3.20, address another dimension of the McKinsey exposure. An attacker with write access to system prompts could inflate token usage massively: longer prompts, more verbose responses, and chain-of-thought reasoning injected into every query. At 500,000+ prompts per month, this becomes a denial-of-wallet attack. Infrastructure-layer token rate limiting and quota management prevent this class of abuse before requests reach the model.

Multi-provider failover routing adds operational resilience. McKinsey was deeply coupled to OpenAI (evidenced by the 266,000+ vector stores). If your primary model provider's integration is compromised, multi-provider failover lets you route traffic to a secondary provider while the primary is under investigation.

Gate 2 ensures that even if the perimeter is breached, the AI pipeline itself has independent safety enforcement that an attacker cannot bypass by modifying application-layer configuration.

Gate 3: MCP Gateway—Agent Governance and Access Control

The third gate governs what AI agents can do: which tools they can invoke, which tasks they are authorized for, which data they can read or write, and which operations they can perform.

This is the gate that addresses the most forward-looking dimension of the McKinsey breach. CodeWall's agent autonomously selected McKinsey as a target, mapped its attack surface, and executed a multi-step attack chain at machine speed. This is precisely the kind of autonomous agent behavior that enterprises are now deploying internally for legitimate purposes and that threat actors will weaponize externally.

The MCP Gateway enforces Task-Based Access Control (TBAC), an authorization model designed specifically for AI agents. Traditional API security (RBAC, OAuth scopes) was not built for autonomous callers that probe, adapt, and chain actions iteratively. TBAC scopes permissions to the actual work being done: which business tasks the agent is authorized to perform, which MCP tools it can access, and which exact operations and data resources it can touch.

In the McKinsey context, even if an autonomous agent somehow bypassed Gate 1 (WAF + API authentication) and Gate 2 (AI content safety), Gate 3 would constrain what the agent could actually do with any access it obtained. A TBAC policy can enforce that no caller, human or autonomous, has write access to system prompt tables through the MCP layer. Tool invocations are scoped, transactions are auditable, and permissions are enforced at the infrastructure layer, below the application runtime.

This last point is critical. Agent platforms are increasingly adding application-level governance: RBAC, audit logging, and signed skills. These controls are valuable, but they operate inside the agent runtime. If the runtime is compromised, as McKinsey's effectively was, so are the guardrails. Infrastructure-layer enforcement through the MCP Gateway operates independently of the agent platform, so even a compromised runtime cannot override the access policies.

Traefik Hub v3.20 also introduces graceful error handling for agent-aware enforcement. When a guardrail blocks a request, traditional gateways return an HTTP 403 response, breaking agent control flow and crashing multi-step workflows. Traefik Hub can now return structured, schema-compliant refusal responses that agents and applications process gracefully. The agent continues operating within its authorized scope. Middleware chains stay intact. Users see conversational refusals instead of technical errors. This is what makes the runtime governance agent-aware: enforcement that works with autonomous workflows rather than breaking them.

The Uncomfortable Question

McKinsey will strengthen its security posture. They responded quickly, they engaged forensics, and they patched within hours. That deserves acknowledgement.

The uncomfortable question is not about McKinsey. It is about the thousands of enterprises that are deploying internal AI platforms right now with the same architectural gaps. No WAF on the AI endpoints. No authentication on development or staging APIs that quietly became production. No content safety guardrails independent of the model provider. No agent governance at the infrastructure layer.

Gartner estimates that 40% of enterprise applications will incorporate AI agents by the end of 2026, up from less than 5% in 2025. The MCP ecosystem has grown to over 10,000 published servers. NVIDIA just unveiled NemoClaw at GTC 2026, bringing enterprise-grade agent orchestration to the NVIDIA stack.

The agents are coming. The attack surface is expanding at machine speed. And the defenses, in most organizations, are still designed for a world where humans typed queries into web forms and waited for responses.

Defense in depth is not a new concept. But the AI era demands a new implementation: one that governs not just HTTP traffic, but LLM content, model interactions, and autonomous agent behavior, all at the infrastructure layer, all enforced independently of the application code that will inevitably have bugs.

That is what the Triple Gate Pattern is for. And it is what we built Traefik Hub to deliver. Organizations already running Traefik Proxy for ingress can add the full API Gateway (with native WAF), AI Gateway, and MCP Gateway capabilities through a single in-place upgrade. No re-architecture. No traffic migration. No additional proxies in the data path. Three gates, one platform, one control plane.

The McKinsey breach is the proof point that this architecture is no longer optional.

Traefik Hub v3.20, including the composable safety pipeline, multi-vendor guardrails with parallel execution, token-level cost controls, graceful agent-aware error handling, IBM Granite Guardian integration, and custom Regex Guards. Sign up for Early Access or read the technical deep dive: From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline.

For a detailed overview of the Triple Gate Pattern and Traefik Hub's AI and MCP Gateway capabilities, visit traefik.io/solutions/ai-gateway.

]]>

From Regex to GPU: Building a Multi-Vendor AI Safety Pipeline with NVIDIA, IBM, Microsoft, and Custom Pattern Matching

Immánuel Fodor, Zaid Albirawi — Mon, 16 Mar 2026 15:19:57 GMT

AI runtime safety has always been about layers. No single guard catches everything. Deterministic pattern matching is fast, but it can't understand context. NLP-based entity detection catches structured PII but misses semantic threats. GPU-accelerated models understand nuance but add latency and cost. The right approach has always been defense-in-depth: multiple guards, different detection methods, complementary coverage.

The problem is that defense-in-depth has historically meant stacking latency. Three guards running in series can add several seconds of enforcement overhead per request. In production, that's a non-starter. Organizations end up choosing between thorough protection and acceptable performance, deploying one or two guards instead of the four or five they'd actually want.

Traefik Hub v3.20 changes this. The AI Gateway now supports a composable, multi-vendor safety pipeline, where organizations can choose from multiple guardrail providers, combine them based on their requirements, and run them all in parallel. Total enforcement time equals the slowest guard, not the sum. And when a guard blocks a request, the response is structured so agents and middleware chains can handle it gracefully instead of crashing.

This post walks through the architecture, configuration, and practical considerations for building a multi-vendor safety pipeline with Traefik Hub.

The Pipeline: Four Tiers from Deterministic Speed to Semantic Intelligence

The composable safety pipeline spans four tiers. Each tier serves a different purpose, uses a different detection approach, and has different performance and infrastructure characteristics. Organizations deploy the tiers they need based on their threat model, latency budget, and infrastructure.

Tier 1: Regex Guard (Custom Pattern Matching)

Regex Guard is a framework for organizations to write their own content guards. It's not a pre-built guard with fixed rules. Teams define the patterns they want to catch, the fields they want to scan, and whether to block or mask matched content.

Here's how it works: the Regex Guard implements a rule-based pattern matching engine integrated into the Content Guard middleware. Each rule specifies one or more JSONQuery expressions that locate target fields in structured payloads, one or more regex patterns to match against, and an action (block or mask) to take on match.

Here's a practical configuration that blocks prompt injection patterns in chat messages while masking credit card numbers in customer data:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: regex-guard
  namespace: apps
spec:
  plugin:
    content-guard:
      engine:
        regex: {}

      request:
        # Block prompt injection attempts in chat messages
          - jsonQueries:
              - ".messages[].content"
              - ".prompt"
            block: true
            reason: "Potential prompt injection detected"
            entities:
              - "(?i)ignore.*(previous|above|prior).*(instructions|prompt)"
              - "(?i)you are now.*(new|different|unrestricted)"

        # Mask credit card numbers in customer data
        - jsonQueries:
            - ".customer.payment"
            - ".billing.card_number"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}"

        # Mask US Social Security numbers
        - jsonQueries:
            - ".user.ssn"
            - ".applicant.social_security"
          mask:
            char: "*"
            unmaskFromLeft: 0
            unmaskFromRight: 4
          entities:
            - "\\d{3}-\\d{2}-\\d{4}"

      response:
        # Mask internal IP addresses in error responses
        - jsonQueries:
            - ".error_detail"
            - ".debug_info"
          mask:
            char: "X"
            unmaskFromLeft: 0
            unmaskFromRight: 0
          entities:
            - "192\\.168\\.[0-9]{1,3}\\.[0-9]{1,3}"
            - "10\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}"

A few things to note about the configuration:

JSONQuery field targeting lets you scope rules to specific fields rather than scanning entire payloads. .messages[].content targets the content field in a chat completion request. .customer.payment targets a specific nested field.

Block vs. mask serves different purposes. Blocking rejects the request entirely. Masking redacts the matched content in-place and lets the request continue. Use blocking for threats (prompt injection, system prompt leakage). Use masking for PII (credit cards, SSNs, phone numbers) where you want the request to proceed with sensitive data redacted.

Bidirectional rules apply to both requests and responses independently. Request-side rules catch threats before they reach the LLM. Response-side rules prevent information leakage (internal IPs, database connection strings, infrastructure identifiers) in the LLM's output.

Performance: Regex Guard runs entirely in-process with zero external dependencies. Latency is sub-millisecond for typical payloads. There are no network round-trips, no GPU requirements, and no probabilistic variance. The same input always produces the same output.

The cost argument is straightforward. Catching a credit card number or Social Security number doesn't require semantic understanding. These are well-defined patterns with well-known formats. Running them through a GPU-accelerated AI model adds latency and cost with no improvement in detection accuracy for these patterns. Regex Guard handles them at a fraction of the cost and an order of magnitude faster.

Tier 2: Content Guard (Microsoft Presidio)

Content Guard provides PII detection and masking powered by Microsoft's Presidio analyzer. Where Regex Guard relies on patterns you define, Presidio uses statistical NLP-based entity recognition with a library of built-in entity types: email addresses, phone numbers, medical record IDs, credit card numbers, and more.

Presidio requires an external analyzer instance. You deploy the Presidio service and configure Content Guard to point at it. The middleware sends the content to Presidio for analysis, receives entity detection results, and applies blocking or masking based on the matches.

Content Guard supports custom entity patterns through Presidio's custom analyzer endpoints, allowing organizations to extend beyond the built-in entity types for formats specific to their business.

The tradeoff vs. Regex Guard: Presidio's NLP-based detection can catch entities that don't follow strict patterns (misspelled names, partial phone numbers, context-dependent PII). But it requires an external service, adds network latency, and introduces a dependency. For well-defined patterns, Regex Guard is faster. For fuzzy or context-dependent PII, Presidio is more thorough. In a composable pipeline, you can run both.

Tier 3: LLM Guard with NVIDIA NIMs

NVIDIA NIM (NVIDIA Inference Microservices) integration provides GPU-accelerated content safety through three specialized microservices:

Topic Control NIM enforces conversation boundaries. It takes guideline-based prompts that define allowed and prohibited topics, and blocks requests that fall outside the defined scope.

Content Safety NIM detects harmful content across 22+ safety categories, including violence, hate speech, and privacy violations. Analyzes both user requests and AI responses in real-time.

Jailbreak Detection NIM identifies attempts to bypass AI system restrictions through prompt manipulation.

Each NIM requires a dedicated GPU with 24GB+ memory. Minimum processing latency per NIM ranges from 30ms (jailbreak detection) to 200ms (content safety). When running all three sequentially, total minimum latency is 180-350ms. With parallel execution, total latency equals only the slowest NIM.

This is where GPU-accelerated semantic intelligence earns its latency cost. NVIDIA NIMs catch threats that pattern matching and NLP-based detection cannot: nuanced prompt injection attempts that don't use obvious keywords, subtle harmful content that requires contextual understanding, and off-topic drift that a regex rule would need hundreds of patterns to approximate.

Tier 4: LLM Guard with IBM Granite Guardian

IBM's Granite Guardian 3.3 (8B parameter) is an open-source safety model that provides capabilities other guard providers don't yet offer: hallucination detection and RAG quality assessment.

Granite Guardian uses a prompt-template approach. Each safety task (harm, jailbreak, topic control, hallucination) is triggered by a specific system prompt. The model responds with a structured <score> yes </score> or <score> no </score> evaluation, which the LLM Guard middleware evaluates with block conditions.

Here's the hallucination detection configuration, which is response-only and uses request history for context:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: granite-hallucination-detection
  namespace: apps
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      response:
        systemPrompt: "hallucination"
        useRequestHistory: true
        blockConditions:
          - reason: hallucination_detected
            condition: Contains("yes")

And here's harm detection, which scans both requests and responses:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: granite-harm-detection
  namespace: apps
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      request:
        systemPrompt: "harm"
        blockConditions:
          - reason: harmful_content_detected
            condition: Contains("yes")
      response:
        systemPrompt: "harm"
        useRequestHistory: true
        blockConditions:
          - reason: harmful_response
            condition: Contains("yes")

Granite Guardian is deployed via vLLM and requires one GPU with approximately 16GB memory (FP16) or 8GB (8-bit quantization). The temperature: 0 parameter is required for reliable classification.

The unique value of Granite Guardian in the composable pipeline is hallucination detection and RAG quality assessment. NVIDIA NIMs don't yet offer hallucination detection. Presidio and Regex Guard are structurally unable to detect hallucinations (they match patterns, not factual accuracy). If your application uses retrieval-augmented generation and you need to verify that the model's response is grounded in the retrieved context, Granite Guardian is currently the only guard tier in the pipeline that addresses this.

Parallel Guard Execution: Defense-in-Depth Without the Latency Tax

With four tiers available, the question becomes: can you run them all without making every request unacceptably slow?

The answer is parallel guard execution. Instead of running guards in series (where total latency equals the sum), Traefik Hub runs all guards simultaneously. Total enforcement time equals only the slowest guard.

The parallel-llm-guard middleware orchestrates this. It spawns all configured guards as concurrent Go routines, collects results as they complete, and makes a pass/fail decision. Every guard blocks equally; there is no critical/optional distinction. If any guard blocks, the request is rejected with a 403, and all other guards are immediately cancelled. If any guard errors, the request fails with a 500, and all others are cancelled.

Each guard in the guards array wraps one of four guard types:

chat-completion-llm-guard
chat-completion-llm-guard-custom
llm-guard
llm-guard-custom

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: parallel-safety-pipeline
  namespace: ai-services
spec:
  plugin:
    parallel-llm-guard:
      guards:
        - chat-completion-llm-guard:
            endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
            model: ibm-granite/granite-guardian-3.3-8b
            clientConfig:
              timeoutSeconds: 5
            params:
              temperature: 0
              maxTokens: 50
            request:
              systemPrompt: "harm"
              blockConditions:
                - reason: harmful_content
                  condition: Contains("yes")

        - chat-completion-llm-guard:
            endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
            model: ibm-granite/granite-guardian-3.3-8b
            clientConfig:
              timeoutSeconds: 5
            params:
              temperature: 0
              maxTokens: 50
            request:
              systemPrompt: "jailbreak"
              blockConditions:
                - reason: jailbreak_attempt
                  condition: Contains("yes")

        - llm-guard:
            endpoint: http://nvidia-content-safety.apps.svc.cluster.local:8000
            clientConfig:
              timeoutSeconds: 5
            request:
              blockConditions:
                - reason: unsafe_content
                  condition: Contains("UNSAFE")

In this configuration, all guards run concurrently. Total enforcement time equals the slowest guard, not the sum. If one guard completes and blocks the request, all other guards are immediately cancelled, as there is no need to spend tokens (and GPU cycles) on finishing processing guard executions on a request that's already been rejected by a faster check.

Agent-Aware Enforcement: Graceful Error Handling

A composable safety pipeline that catches threats is only half the story. The other half is what happens when a guard blocks a request.

Traditional gateways return an HTTP 403 Forbidden with a plain-text error message. This works for simple request-response APIs. It doesn't work for autonomous agents or agentic workflows.

When an agent executes a multi-step workflow and one step triggers a guardrail, an HTTP 403 breaks the agent's control flow. The agent can't distinguish between "the guardrail blocked this specific request" and "something is fundamentally wrong with the system." Most agent frameworks treat non-2xx responses as errors that require human intervention or task failure. A single guardrail block can crash an entire multi-step workflow.

Traefik Hub v3.20 introduces the onDenyResponse feature. When configured, guardrail blocks return a custom HTTP status code and message instead of the default 403. This is configured per block condition, so different denial reasons can return different responses:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: llm-guard-with-graceful-errors
spec:
  plugin:
    chat-completion-llm-guard:
      endpoint: http://granite-guardian.apps.svc.cluster.local:8000/v1/chat/completions
      model: ibm-granite/granite-guardian-3.3-8b
      params:
        temperature: 0
        maxTokens: 50
      request:
        systemPrompt: "harm"
        blockConditions:
          - reason: harmful_content
            condition: Contains("yes")
            onDenyResponse:
              statusCode: 200
              message: 'This request was blocked by security policies. Please reformulate your request.'

An agent receiving an HTTP 200 with a refusal message can parse it as a valid turn in the conversation, understand that the request was refused (not that the system failed), and decide what to do next: retry with modified input, skip this step, or escalate to a human.

This also keeps middleware chains intact. Without onDenyResponse, a 403 from an upstream guardrail could potentially break downstream custom plugins ( eg., custom response handlers, loggers) that expect structured JSON and were not created with graceful error handling in mind. With it, the entire chain processes normally.

Existing deployments continue returning HTTP 403 by default. Each block condition can independently define its own onDenyResponse with a custom status code and message template.

Operational Controls: Resilience and Cost

A safety pipeline, even one that's composable and parallel, isn't sufficient for production on its own. Organizations also need their AI to stay up when providers go down and stay within budget. Traefik Hub v3.20 adds both.

Failover Router

The Failover Router uses nested TraefikService resources to build a failover chain across LLM providers. When a primary backend responds with a configured error status code (e.g., 429 rate limit, 500-504 server errors), the request is automatically replayed to the fallback service. Each backend gets its own chat-completion middleware for independent model selection, API keys, and metrics.

# Primary backend: OpenAI GPT-4o
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: openai-primary
  namespace: ai-services
spec:
  failover:
    service:
      name: openai-gpt4o
      middlewares:
        - name: openai-chat-completion
    fallback:
      name: anthropic-fallback
    errors:
      status:
        - "429"
        - "500-504"
---
# Secondary backend: Anthropic Claude
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: anthropic-fallback
  namespace: ai-services
spec:
  failover:
    service:
      name: anthropic-claude
      middlewares:
        - name: anthropic-chat-completion
    fallback:
      name: openai-mini-budget
    errors:
      status:
        - "429"
        - "500-504"

Each chat-completion middleware configures the provider-specific details (model name, API key secret, parameters) independently. This means each backend in the failover chain can use a different provider, model, and set of credentials. The failover mechanism is status-code-based: when the primary responds with a status matching errors.status, the request is replayed to the fallback. This enables cost-optimized degradation: when the premium model is rate-limited or down, fall back to a budget model from a different provider.

Because the Failover Router operates within the Triple Gate architecture, all safety pipeline guards and cost controls continue to apply regardless of which provider is serving the request. The governance doesn't degrade when the model does.

Token Rate Limiting and Quota Management

LLM providers charge by token, not by request. A single request with a 100K-token prompt costs orders of magnitude more than one with a 100-token prompt. Request-count-based rate limiting doesn't capture this.

Token governance tracks input, output, and total tokens independently. It operates in two modes:

Rate limiting handles traffic spikes with short windows (hourly, daily). Prevents burst overages without hard-capping long-term consumption.

Quota enforcement sets hard budget caps over longer periods (monthly). When the quota is reached, requests are rejected entirely.

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ai-token-rate-limit
spec:
  plugin:
    ai-rate-limit:  # or ai-quota
      store:
        redis:
          endpoints:
            - redis.default.svc.cluster.local:6379
      inputTokenLimit:
        limit: 5000
        period: 1h
        jsonQuery: ".usage.prompt_tokens"
      outputTokenLimit:
        limit: 10000
        period: 1h
        jsonQuery: ".usage.completion_tokens"
      totalTokenLimit:
        limit: 15000
        period: 1h
        jsonQuery: ".usage.total_tokens"
      sourceCriterion:
        requestHeaderName: "X-User-ID"
      estimateStrategy:
        simple: {}

The sourceCriterion determines how requests are grouped for rate limiting. It supports three options: requestHeaderName (a simple string identifying which header to use, e.g., "X-User-ID"), ipStrategy (with depth, excludedIPs, and ipv6Subnet for IP-based tracking), and requestHost (a boolean for host-based tracking). For JWT-based per-user limiting, the recommended pattern is to deploy a separate JWT middleware that extracts claims into headers via forwardHeaders, then reference that header with requestHeaderName. Composability enables many use cases, such as group/team-based buckets (using a Group header with the group claim extracted) or department/location-based bucketing.

The key differentiator is proactive token estimation. Most solutions enforce limits reactively: they let the request through, wait for the LLM response, read the token count, and then enforce limits on the next request. By that point, the cost has already been incurred.

Traefik Hub's token rate limiting estimates input tokens before the request reaches the LLM. If the estimated usage would exceed remaining capacity, the request is blocked immediately, before consuming any LLM resources or incurring any cost.

Token counts are stored in a distributed Redis backend, ensuring accurate tracking across multiple gateway instances.

The Full AI Workflow: End-to-End

Here's how all of these pieces can work together in a single request:

Request arrives at the API Gateway (Gate 1). JWT is validated, identity is extracted, and rate limit headers are checked.
Token estimation runs. If the estimated input tokens would exceed the user's remaining rate limit or quota, the request is blocked before anything else runs. No guard processing, no LLM invocation, no cost incurred.
Deterministic guards execute (Gate 2, first pass). Regex Guard scans for pattern-based threats (prompt injection, credential leaks) and masks PII with known formats (SSNs, credit cards). Content Guard (Presidio) performs NLP-based entity detection for fuzzy PII that regex patterns might miss. These are fast (sub-millisecond for regex, ~200ms for Presidio) and block early before heavier guards run.
Parallel LLM guard pipeline executes (Gate 2, second pass). NVIDIA NIMs and IBM Granite Guardian run simultaneously for semantic threat detection. If any guard blocks, all others are cancelled, and a structured refusal is returned (if onDenyResponse is configured) or an HTTP 403 (if not).
Request reaches the LLM via the Failover Router. If the primary provider responds with a configured error status (429, 500-504), the request is replayed to the fallback service. All safety policies continue to apply.
Response passes back through the guards. Response-side rules in Regex Guard check for internal IP addresses and infrastructure identifiers. Granite Guardian checks for hallucinations using the request history as context.
Actual token counts are captured from the LLM response and stored in Redis, updating the user's rate limit and quota tracking.
If the LLM triggers an agent action, the request hits the MCP Gateway (Gate 3). Tool-Based Access Control (TBAC) enforces what the agent is allowed to do: which tasks, which tools, which parameters. The same JWT identity that governed token budgets in Gate 2 now governs agent permissions in Gate 3.

Three gates, one identity context, and a composable safety pipeline that runs in parallel across the entire AI workflow.

Choosing Your Guard Tiers: A Decision Framework

Not every deployment needs all four tiers. Here's a practical framework for choosing:

Start with Regex Guard if you have well-defined patterns to catch. PII with known formats (SSNs, credit cards, phone numbers), prompt injection signatures, credential patterns, infrastructure identifiers. Zero infrastructure cost, sub-millisecond latency, 100% deterministic. This should be the baseline for any deployment.
Add Content Guard (Presidio) if you need NLP-based entity detection that goes beyond fixed patterns. Presidio catches entity variations that a regex might miss (misspelled names, partial phone numbers, context-dependent PII). Requires deploying a Presidio analyzer instance.
Add NVIDIA NIMs if you need semantic threat detection. Jailbreak attempts, subtle harmful content, topic drift, and prompt injection that doesn't use obvious keywords. Requires GPU infrastructure. This is where you catch threats that pattern matching fundamentally cannot.
Add IBM Granite Guardian if you need hallucination detection or RAG quality assessment. No other tier in the pipeline offers this. It’s also valuable for organizations in the IBM ecosystem. Keep in mind, it requires a GPU.
Enable parallel execution whenever you deploy LLM Guards. There's no reason to stack latency in series when the guards are independent.
Enable graceful error handling if your consumers include autonomous agents or if you use middleware chains that expect structured JSON at every stage.

What's Next

The composable pipeline is designed for extension. Traefik has integrated guardrails from NVIDIA, IBM, and Microsoft, and will continue integrating third-party providers as the ecosystem matures. Organizations can combine these vendor integrations with their own custom Regex Guard rules, building a safety pipeline tailored to their specific threat model, compliance requirements, and infrastructure constraints.

Traefik Hub v3.20 is now available as an Early Access release, with general availability planned for late April 2026. To try the composable safety pipeline and the other features covered in this post, sign up for Early Access.

]]>

How to Choose the Right Load Balancing Strategy for Your Use Case

Simon Delicata — Thu, 05 Feb 2026 17:11:06 GMT

In a previous article on advanced load balancing, we explored features like sticky sessions, health checks, and traffic mirroring. Those are the tools. This article covers the strategy.

Your application runs smoothly with round-robin load balancing. Traffic distributes evenly, everything works. Then you notice some requests take 50ms, others 500ms. Monitoring shows all servers are healthy, but user experience is inconsistent. The problem isn't if you're load balancing—it's how.

Load balancing strategies determine which server handles which request, and this decision has a direct impact on performance, cost, and user experience. Choosing the wrong strategy is like taking the highway when you need to navigate city streets—you might eventually get there, but you're not adapting to the terrain.

Traefik's load balancing strategies map to specific technical requirements. Weighted Round Robin (WRR) works for uniform backends. Power of Two Choices (P2C) handles variable connection lifetimes. Highest Random Weight (HRW) optimizes caching. Least-Time adapts to heterogeneous performance.

Understanding the Landscape: Features vs. Strategies

First, an important distinction:

Load balancing features (sticky sessions, health checks, mirroring) define capabilities—what your load balancer can do.

Load balancing strategies define decision-making—which server gets the next request.

In other words, features are your car's options: heated seats, navigation, backup camera. Strategies are your driving style: highway cruising, city maneuvering, off-road driving.

Both matter, but serve different purposes. This article focuses on strategies—the algorithms that route your traffic.

The Four Core Strategies: Overview

Traefik supports four primary load balancing strategies at the server level:

Strategy	How It Works	Why Choose It
WRR (Weighted Round Robin)	Fair rotation with optional weights	General-purpose, predictable workloads
P2C (Power of Two Choices)	Pick 2 random servers, choose the one with fewer connections	Dynamic scaling, variable connection lifetimes
HRW (Highest Random Weight)	Consistent hashing per client	Cache optimization, client affinity
Least-Time	Lowest response time + active connections	Heterogeneous backends, performance optimization

Each strategy maps to specific business scenarios.

Matching Strategy to Your Use Case

These scenarios are presented in order of increasing complexity, but choose based on your technical requirements, not your company size or stage. A caching-heavy product should use HRW from day one, while a large-scale service with uniform workloads might use WRR forever.

Scenario 1: Simple and Predictable—WRR

Great for: Internal tools, stable production services

Use When

Homogeneous backends (all servers are similar)
Predictable traffic patterns
Simplicity and reliability are priorities
You don't need advanced optimization

The Problem You're Solving
You've deployed three identical application servers and need to distribute traffic. Your focus is on uptime and simplicity—not premature optimization.

The Solution: Weighted Round Robin (WRR)
WRR is Traefik's default strategy. It distributes requests in a predictable, fair rotation:

http:
 services:
   my-api:
     loadBalancer:
       servers:
         - url: "http://server-1:8080"
         - url: "http://server-2:8080"
         - url: "http://server-3:8080"

With WRR, if you send 300 requests, each server handles approximately 100 over time. Simple, predictable, reliable.

When to Add Weights
When you have servers with different capabilities, WRR's weight system lets you send proportionally more traffic to more powerful machines. The higher the weight, the more traffic the server gets.

http:
 services:
   my-api:
     loadBalancer:
       servers:
         - url: "http://small-server:8080"
           weight: 1
         - url: "http://medium-server:8080"
           weight: 2
         - url: "http://large-server:8080"
           weight: 4  # Gets 4x traffic over time of small server

Benefits and Trade-Offs
WRR delivers predictable costs and straightforward capacity planning. Troubleshooting is simple since you always know which server handled a request.

The trade-off: WRR doesn't adapt to real-time conditions. All requests are treated equally, regardless of complexity or current server load.

Consider Other Strategies When
Your infrastructure becomes heterogeneous, connection lifetimes vary significantly, or you need caching/performance optimization.

Scenario 2: Variable Connection Lifetimes—P2C

Great for: Real-time applications, auto-scaling environments

Use When

Variable connection lifetimes (WebSockets, long-polling, streaming mixed with short requests)
Auto-scaling infrastructure (servers frequently added/removed)
Connection count imbalance is causing performance issues
You need automatic load distribution without manual tuning

The Problem You're Solving
You've implemented auto-scaling, but you notice uneven load distribution. Some servers are overloaded while others are idle. Your monitoring shows:

Server A: 150 active connections, CPU at 80%
Server B: 30 active connections, CPU at 20%
Server C: 200 active connections, CPU at 95%

Your Weighted Round Robin (WRR) strategy keeps sending traffic to Server C even though it's struggling.

The Solution: Power of Two Choices (P2C)
P2C uses a simple algorithm (hence the name): randomly pick two servers, then choose the one with fewer active connections.

http:
 services:
   my-api:
     loadBalancer:
       strategy: "p2c"
       servers:
         - url: "http://server-1:8080"
         - url: "http://server-2:8080"
         - url: "http://server-3:8080"

How It Works

Request arrives
Traefik randomly selects two servers (say, A and C)
Compares their active connection counts
Routes to the less loaded server (A has 150, C has 200 → choose A)

This simple algorithm balances connections effectively with minimal overhead.

Real-World Scenario: Real-Time Collaboration Platform
You run a SaaS collaboration platform (think Figma, or Google Docs) that combines real-time editing with standard API operations.

Your infrastructure handles:

Quick API calls: Loading documents, saving comments (50-200ms, connection closes immediately)
Live editing sessions: WebSocket connections for real-time collaboration (users stay connected for 30-60 minutes while editing)
File operations: Document exports, image uploads (30-90 second connections)

With WRR, all servers initially get equal requests, but connection lifetimes vary dramatically. Server A has 8 active editing sessions and 8 WebSockets open for the next hour. Server B has 150 quick API calls that all completed, and the server is now idle. Server C has 5 file uploads and the connections are held for 60 seconds.

WRR keeps distributing requests evenly, regardless of active connections. Once a new editing session starts, it's sent to the already-loaded Server A.

Now Server A is overloaded, Server B is underutilized, and users wonder why their editing sessions is so laggy.

With P2C, however, Traefik randomly picks two servers, compares active connection counts. It recognizes that Server A has 8 connections and Server B has 0, so Traefik routes to Server B.

Long-lived editing sessions naturally spread across available capacity, while quick API calls fill gaps on less-loaded servers. The result is smooth real-time collaboration for all users with optimal resource usage.

Benefits and Trade-Offs
P2C automatically distributes load without manual tuning, handling mixed workloads gracefully. It reduces hot spots and improves resource utilization, which translates to lower cloud costs.

The trade-off: P2C only considers connection count, not processing time. A server handling 10 lightweight requests looks the same as one handling 10 heavy requests. The random selection also makes debugging slightly less predictable than WRR.

Consider Other Strategies When
You need client affinity for caching (HRW), or backend performance varies significantly (Least-Time).

Scenario 3: Cache Optimization—Client Affinity with HRW

Great for: Any stage with caching requirements (CDNs, personalization platforms, session-heavy apps)

Use When

Stateful backends or caching layers
Users benefit from hitting the same server repeatedly
Cache hit rate is more important than perfect load distribution
You have session data, user-specific caches, or personalized content

The Problem You're Solving
You have multiple backend servers, each building up its own cache (Redis, in-memory, CDN edge). With P2C or WRR, a single user's requests bounce between different servers, resulting in:

Cache misses (user's data isn't on the selected server)
Repeated cache warming for the same user across servers
Wasted memory and processing power

The Solution: Highest Random Weight (HRW)
HRW (also called Rendezvous hashing) uses consistent hashing to map clients to servers deterministically.

http:
 services:
   cached-api:
     loadBalancer:
       strategy: "hrw"
       servers:
         - url: "http://cache-server-1:8080"
         - url: "http://cache-server-2:8080"
         - url: "http://cache-server-3:8080"

How it works:

Traefik hashes the client's IP address
Calculates a score for each server using the hash
Consistently routes that client to the highest-scoring server
Same client routes to same server unless servers are added/removed
When servers change, only ~1/N clients get remapped (vs. 100% with simple hash % server_count)

Real-World Scenario: User-Specific Content Caching
You run a personalized recommendation platform. Each server builds in-memory caches of user preferences, browsing history, and ML model outputs.

With WRR/P2C, a user requests a recommendation, which is routed to Server A→ cache miss → compute (100ms). The same user makes a second request, which is then routed to Server B → cache miss → compute (100ms). The result is frequent cache misses (users randomly distributed across servers) and high CPU usage across all servers.

With HRW, a user requests a recommendation, which is routed to Server A (based on client hash) → cache miss → compute and cache (100ms). The user's second request, however, is also routed to Server A (same hash) → cache hit (5ms). With HRW, cache hit rates climb since users consistently hit the same server, which significantly reduces CPU usage from cache efficiency.

Benefits and Trade-offs
HRW maximizes cache efficiency by ensuring the same client consistently hits the same server. This reduces origin load and makes debugging straightforward since routing is deterministic.

The trade-off is that HRW prioritizes consistency over load distribution. If one client generates disproportionate traffic, its assigned server becomes a hotspot. Adding or removing servers also requires careful planning since it triggers client remapping.

Consider Other Strategies When
Load imbalance becomes problematic, or backend performance varies significantly and you need adaptive routing (Least-Time).

Scenario 4: Performance Optimization—Adaptive Routing with Least-Time

Great for: High-traffic applications, performance-critical services, and heterogeneous infrastructure

Use When

Backend performance varies (mixed instance types, different hardware)
Every millisecond of latency matters
You want automatic adaptation to performance changes
Backends have different processing speeds

The Problem You're Solving
Your infrastructure uses mixed instance types:

Server A: High-performance dedicated server (5ms average response)
Server B: Standard cloud instance (15ms average response)
Server C: Cheaper instance with variable performance (10-50ms)

With previous strategies, you face impossible trade-offs:

WRR: Treats all servers equally, users randomly get slow responses
P2C: Balances connections but ignores that Server A is 3x faster
HRW: Sticks users to potentially slow servers

The Solution: Least-Time Strategy
Least-Time combines response time measurement with active connection tracking to route intelligently

http:
 services:
   backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://fast-server:8080"
         - url: "http://standard-server:8080"
         - url: "http://variable-server:8080"

How It Works

For each server, Traefik calculates a score based on recent response times (Time To First Byte) and active connections. Requests are routed to the server with the lowest score. Fast servers with few active connections automatically get more traffic, while degrading servers are deprioritized.

Real-World Scenario: Mixed Instance Types with Weighted Least-Time
Your application runs on heterogeneous infrastructure:

Server A: High-performance dedicated instance (5ms average API response)
Server B: Standard cloud instance (15ms average API response)
Server C: Burstable instance with variable performance (10-50ms depending on CPU credits)

With WRR, all servers receive equal traffic. Users randomly experience 5ms, 15ms, or 50ms responses, which creates an inconsistent user experience. Fast Server A is underutilized while slow Server C gets equal load.

With Least-Time, however, Traefik measures each backend's actual response time (TTFB). Server A (fastest) automatically receives more traffic, while Server C receives less traffic when its CPU credits are depleted. Then, when Server C performance improves, traffic naturally increases. The user experience is consistent, and resource utilization is optimal.

Using weights for capacity-aware routing:

http:
 services:
   backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://high-perf-server:8080"
           weight: 3  # Premium instance, can handle more
         - url: "http://standard-server-1:8080"
           weight: 1
         - url: "http://burstable-server:8080"
           weight: 1

Weights indicate capacity. A server with weight=3 can handle 3x the traffic before its score becomes unfavorable, allowing you to maximize value from premium infrastructure.

Benefits and Trade-Offs
Least-Time delivers near-optimal performance by routing to the fastest available backend. It adapts automatically when servers degrade—no manual intervention required. This leads to better resource utilization and graceful degradation without hard failovers. It's particularly effective for mixed infrastructure where backend performance varies.

The trade-off is: accurate routing depends on stable network conditions for reliable measurements. There's also slight computational overhead from tracking response times, though this is negligible in practice.

Beyond Server Strategies: Service-Level Load Balancing

So far, we've discussed server-level strategies—how to distribute traffic among backend instances. Traefik also offers service-level strategies for advanced patterns:

Weighted Round Robin (Service Level)

Distribute traffic between different services (not just servers). This is perfect for:

Canary deployments: 95% to stable version, 5% to new version
Blue-green deployments: Gradual traffic shifting
A/B testing: Route percentage of users to experiment variants

http:
 services:
   app:
     weighted:
       services:
         - name: stable-v1
           weight: 95
         - name: canary-v2
           weight: 5

Mirroring

Duplicate traffic to multiple services, which is essential for:

Testing new versions with real traffic (without risking production)
Performance comparisons between implementations
Data pipeline validation

http:
 services:
   api-with-mirror:
     mirroring:
       service: production-api
       mirrors:
         - name: new-api-version
           percent: 10  # Mirror 10% of traffic

Failover

Automatic fallback when primary service fails:

http:
 services:
   resilient-api:
     failover:
       service: primary-cluster
       fallback: backup-cluster

Combining Server and Service-Level Strategies

You can combine strategies at multiple levels. For example, you can use Least-Time at the server level for each service, and weighted service-level balancing for canary deployments:

http:
 services:
   # Service-level: Weighted between versions
   app:
     weighted:
       services:
         - name: v1-backend
           weight: 90
         - name: v2-backend
           weight: 10

   # Server-level: Least-Time within each version
   v1-backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://v1-server-1:8080"
         - url: "http://v1-server-2:8080"

   v2-backend:
     loadBalancer:
       strategy: "leasttime"
       servers:
         - url: "http://v2-server-1:8080"
         - url: "http://v2-server-2:8080"

Conclusion: Choosing Your Strategy

Use this decision tree to select the best strategy for your use case:

Quick Reference Table

Your Scenario	Recommended Strategy	Why It's Best
Identical servers, uniform requests	WRR	Simple, predictable, no overhead
Auto-scaling, variable connection lifetimes	P2C	Balances connection count dynamically
Client affinity for caching, stateful backends	HRW	Consistent client→server mapping
Different server types/performance	WRR with weights	Proportional to capacity
Heterogeneous backends, latency-sensitive	Least-Time	Optimizes for actual performance
Canary/blue-green deployments	Weighted (service-level)	Control traffic percentage
Testing with production traffic	Mirroring	Safe real-world validation

Load balancing isn't one-size-fits-all—but it doesn't have to be complicated either. Start with WRR, measure your system's behavior, and evolve deliberately.

Traefik lets you change strategies without downtime, so you can adapt as your requirements change.

The Infrastructure Reality Behind AI Hype: What the 2026 CNCF Survey Reveals (And What It Doesn't)

Sudeep Goswami — Mon, 02 Feb 2026 23:41:47 GMT

The January 2026 CNCF Annual Cloud Native Survey landed this month. Buried beneath the expected headlines about Kubernetes dominance is something far more interesting: a comprehensive map of where enterprise infrastructure investment is actually going, and where it's not.

The survey, based on 628 respondents across multiple industries and geographies, confirms what many of us building infrastructure have observed. The AI conversation has fundamentally shifted from "who can train the best model" to "who can operationalize inference at scale." This shift has profound implications for where value will be created and captured over the next several years.

The Inference Economy Is Here

Let's start with the most striking finding: 52% of organizations don't build or train their own AI models. They're consumers. The remaining organizations that do "train" are predominantly fine-tuning existing models rather than building from scratch.

This shouldn't surprise anyone who's been paying attention, but it should reframe how we think about AI infrastructure investment. The report's authors put it well: "If we cut through the hype of chatbots and agents, we can clearly see that we will need to greatly decrease the difficulty of serving AI workloads while massively increasing the amount of inference capacity available across the industry."

Take a moment and let that sink in. The infrastructure challenge isn't about building bigger GPU clusters for training. It's about the unglamorous work of routing, caching, rate limiting, and managing inference traffic at production scale.

Consider the deployment frequency data: 47% of organizations deploy AI models only occasionally (a few times per year), and just 7% manage daily deployments. The gap between "we have a model" and "we have a production AI system" remains enormous. That gap is an infrastructure problem, not an algorithm problem.

There's also a question the survey doesn't ask. 37% of organizations using managed generative AI APIs are sending inference traffic somewhere. Where? Under what data residency constraints? With what model governance guarantees? As the EU AI Act takes effect and regulated industries face increasing scrutiny over AI decision-making, the sovereignty dimension of inference infrastructure will become harder to ignore.

Kubernetes Won. Now What?

The survey confirms what we've observed in our customer base: 82% of container users now run Kubernetes in production, up from 66% in 2023. The report aptly describes Kubernetes as "boring," and correctly notes that boring is the highest compliment in infrastructure.

But here's the more interesting number: 66% of organizations are using Kubernetes specifically to host their generative AI workloads. Kubernetes has evolved from "container orchestrator" to "AI infrastructure platform." The report notes that projects like Kubeflow provide end-to-end ML workflows while KServe handles model serving at scale.

This convergence creates a clear market reality: whatever you build for AI workloads needs to be Kubernetes-native. Not Kubernetes-compatible. Not Kubernetes-adjacent. Native. The organizations that have achieved "true MLOps maturity" (the 23% running all inference workloads on Kubernetes) have done so by integrating AI into their existing CI/CD pipelines, GitOps workflows, and observability stacks.

Security Hasn't Gone Away. Complexity Has Caught Up.

The survey asked respondents about their top challenges in deploying containers. "Cultural changes with the development team" came in at #1 with 47%, followed by "lack of training" (36%) and "security" (36%).

That security figure deserves attention. In 2023, security was the top challenge. It hasn't become less important. If anything, it's become more critical as AI workloads introduce new attack surfaces. What's changed is that platform engineering complexity has risen to match it as an operational bottleneck.

The report's authors frame the cultural challenge as evidence for Platform Engineering: building standardized platforms with "paved roads, sensible defaults, and clear guardrails." That framing is correct, but incomplete. The cultural resistance the survey identifies isn't irrational. Developers express skepticism that containers add unnecessary complexity for simple applications. Operations teams worry about troubleshooting containerized systems. Management questions whether investments are distracting from feature delivery.

Now, layer AI agents on top of this. The MCP revolution is enabling AI systems to call external tools, access databases, and take autonomous actions. Each of those capabilities is a new permission surface. Each is a new governance challenge. Organizations struggling with basic container deployment complexity aren't ready for agent-to-agent communication patterns where security and access control become exponentially harder to reason about.

The security challenge hasn't diminished. It's compounding. And it's compounding on top of platform engineering foundations that many organizations haven't solidified.

GitOps as the Maturity Marker

The survey segments organizations into four maturity profiles: explorers (8%), adopters (32%), practitioners (34%), and innovators (25%). The progression between these stages correlates strongly with specific technology and practice adoption.

The most telling correlation is GitOps adoption: 0% of explorers have implemented GitOps, while 58% of innovators run GitOps-compliant deployments. CI/CD shows a similar pattern. 42% of explorers use it versus 91% of innovators.

Figure 17, 2026 CNCF Annual Cloud Native Survey

Figure 18, 2026 CNCF Annual Cloud Native Survey

This isn't just correlation. GitOps requires, and forces, a level of operational discipline that defines mature cloud native organizations: declarative configurations, version control for infrastructure, automated synchronization, and clear audit trails. That last point matters enormously for security. You can't secure what you can't audit. You can't audit what isn't versioned. Organizations that can't achieve this discipline will struggle with AI workloads, which require even more rigorous deployment practices and access controls.

The survey also reveals that innovators operate at a fundamentally different velocity. 74% check in code multiple times daily (versus 35% of explorers). 41% run daily releases (versus 12% of explorers). And 59% automate more than 60% of their deployments (versus 6% of explorers).

If your infrastructure layer requires ClickOps, you're building for yesterday's market. And if your security model depends on manual reviews that can't keep pace with automated deployments, you're building a compliance gap that will only widen.

The Sustainability Warning

Buried in the report is a warning that deserves more attention. A September 2025 open letter from open source infrastructure stewards warned that critical systems operate under "a dangerously fragile premise," relying on goodwill rather than sustainable funding models.

The letter specifically calls out AI/ML workloads as drivers of "machine-driven, often wasteful automated usage" that strains open source infrastructure. The report notes that "commercial-scale workloads often run without caching, throttling, or even awareness of the strain they impose."

This should concern anyone building AI infrastructure. The naive approach (more API calls, more requests, more brute-force retry logic) is technically simple and operationally catastrophic. Organizations that build efficiency into their AI infrastructure from day one will have both cost and sustainability advantages.

What This Means for Infrastructure Strategy

The CNCF survey paints a clear picture of where infrastructure investment should flow:

Inference over training. The majority of organizations are consumers of models, not producers. Infrastructure that optimizes inference through intelligent caching, efficient routing, and scale-to-zero capabilities will capture more value than training-focused solutions.
Kubernetes-native is table stakes. With 66% of organizations running AI workloads on Kubernetes, solutions that require separate orchestration or bolt-on integrations will face adoption friction.
Security and simplicity aren't trade-offs. When "cultural changes" and "security" are both top-three challenges, infrastructure that reduces cognitive complexity while strengthening access controls has a real moat. This becomes even more critical as MCP-enabled agents expand the permission surface.
GitOps compatibility is non-negotiable. The correlation between GitOps adoption and organizational maturity is too strong to ignore. Any infrastructure that requires manual configuration or UI-driven workflows is building for a shrinking market segment. More importantly, GitOps provides the audit trail that security and compliance teams increasingly demand.
Efficiency matters more than we admit. The sustainability concerns raised in the report aren't just ethical considerations. They're operational realities. Inefficient AI infrastructure is expensive AI infrastructure.

Questions for the Road Ahead

The survey opens as many questions as it answers.

As organizations move from occasional AI deployments to production-scale inference, how are they governing agentic workflows? The MCP ecosystem is evolving rapidly, but we have no baseline data on adoption patterns or access control approaches.
Similarly, the Kubernetes-centric view is appropriate for CNCF, but enterprises increasingly manage traffic across VMs, containers, and serverless simultaneously. How are organizations handling that complexity?
And at the most fundamental level: how is traffic actually flowing through these systems? The survey covers orchestration and observability well, but the routing and governance layer between services remains unexplored.

These aren't criticisms. They're signs of a maturing ecosystem where the interesting questions keep evolving.

The Bottom Line

The CNCF survey confirms that we're past the "should we do AI" phase and deep into the "how do we operationalize AI" phase. The winners won't be determined by who has the best model. Foundation model providers have commoditized that race. The winners will be determined by who can move inference workloads from demo to production at scale, with the governance, security, and reliability that enterprises require.

That's an infrastructure problem. And infrastructure problems are what we solve.

]]>

From Zero to Production-Grade WAF (Without the Appliance Pain)

Carlos Villanúa Fernández, Immánuel Fodor — Thu, 29 Jan 2026 20:55:46 GMT

Traefik Hub + Coraza + OWASP CRS give developers a practical, high-powered WAF that fits cloud-native workflows.

Let us start with the uncomfortable truth: your app is getting hammered long before it hits prod.

As of January 2026, Imperva reports that almost 51% of internet traffic is non-human, and bad bots alone make up nearly one-third of all traffic. Data Breach Investigations Report 2025 adds a second punch: a 34% increase in attackers exploiting vulnerabilities to gain initial access. These are the turbulent waters your APIs are shipping into.

And it gets worse. SQL injection alone accounts for 65% of all web application attacks. A 20+ year-old vulnerability class that still dominates. When breaches happen, IBM's 2025 report shows U.S. costs average $10.22 million, while global costs average at $4.44 million.

This is why Web Application Firewalls (WAFs) still matter. Not because they replace secure coding, but because they buy you time and block obvious exploitation while you patch, tune, and ship.

The other truth: WAFs used to mean expensive appliances and slow integrations. That is no longer the case. Traefik Hub API Gateway integrates Coraza WAF as a native feature, providing high-performance protection with an implementation flow that fits developer workflows. Coraza and OWASP CRS are both well-maintained, standalone open-source projects, and now Traefik Hub packages them into a production-ready gateway.

Before we dive into how to deploy and test Traefik Hub's native WAF integration, let's quickly explore why they matter and the technologies behind them.

Why WAFs Matter (Right Now)

A WAF is a shield in front of your app. It inspects HTTP traffic and blocks known malicious patterns before requests hit your code. You still need secure coding, auth, and rate limiting. But WAFs are your safety net when:

You cannot patch instantly
Attackers are automated and relentless
A single vulnerable endpoint could open the door to [insert your most valuable information sources here]

The dataset behind the OWASP Top 10:2025 document includes over 2.8 million applications. In that data, common risk categories remain pervasive: Broken Access Control, Security Misconfiguration, and Cryptographic Failures show up in production apps at scale. Those are not edge cases. They are everyday issues.

Here's how WAF coverage maps to the OWASP Top 10:

Rank	Category	WAF Coverage
A01	Broken Access Control	Custom rules
A03	Software Supply Chain Failures	Partial
A04	Cryptographic Failures	Limited
A05	Injection (SQLi, XSS)	Full CRS coverage
A06	Insecure Design	Limited
A07	Authentication Failures	Custom rules
A10	Mishandling of Exceptions	Partial

Notice that injection attacks (the category that includes SQL injection and XSS) get full CRS coverage out of the box. Given that SQLi represents 65% of web app attacks, that is significant protection for minimal effort.

Your Defense Stack... In Plain English

Here is what each piece does:

ModSecurity: the original open-source WAF engine. The project calls it the "Swiss Army Knife" of WAFs. It defined the standard for rule-based web application firewalls.
SecLang: the ModSecurity rules language. Rules look like SecRule VARIABLES "@OPERATOR ..." "ACTIONS".
Coraza: a modern, high-performance WAF engine that speaks the same language as ModSecurity (SecLang). Written in Go, memory-safe, actively maintained. Think of it as the next-generation engine that runs the same rules.
OWASP CRS: the rule set. A community-maintained collection of generic attack detection rules designed to cover the OWASP Top Ten with minimal false alerts. Over 500 rule bypasses closed through their bug bounty program.
Traefik Hub API Gateway: a commercial gateway that integrates Coraza WAF and makes deployment and tuning practical for teams.

Coraza gives you the WAF engine, CRS gives you the rules, and Traefik Hub turns it into a clean, deployable, version-controlled middleware.

The WAF Landscape: Why Cloud-Native Wins

The Problem with Legacy Enterprise WAFs

Traditional enterprise WAFs come with enterprise pricing: $50,000 to $100,000+ annually. They require dedicated security teams, complex deployment processes, and were not designed for Kubernetes or microservices architectures. Hidden costs abound: throughput limits, overage charges, and mandatory professional services.

Cloud Provider WAFs: Death by a Thousand Cuts

AWS WAF looks affordable at first: $5/month plus $1/rule plus $0.60/million requests. But add Shield Advanced for DDoS protection, and you are looking at ~$3,000/month. Costs scale unpredictably with traffic, and you are locked into a single cloud provider.

The real problem is not just cost. It is opacity. Cloud provider WAFs are black boxes. You cannot see exactly what rules are doing, you cannot customize the detection logic, and you cannot audit the security decisions being made. When a false positive blocks legitimate traffic, debugging is guesswork.

The Traefik Hub Gateway-Backed Approach

Traefik Hub takes a different path:

Affordable pricing: Enterprise-grade WAF without the enterprise “big-vendor” pricing
Cloud-native architecture: Built for Kubernetes and microservices from day one
Native Coraza integration: No sidecars, no WASM overhead, no complex setup
Open-source foundations: Battle-tested Coraza engine & OWASP CRS rules
Transparent rules: You can see exactly what is being blocked, no black box

You get the best of both worlds: an open-source WAF engine with community-hardened rules, commercial support, and native integration through Traefik Hub.

Architecture at a Glance

WAF protection is applied as middleware. That design matters because you can:

Enable it per route or per service
Start in detection-only mode
Layer it with OIDC, JWT, or rate limiting
Tune rules without touching application code

This is the developer workflow we want: security becomes a YAML configuration change, reviewed in Git, shipped with the same pipeline as your app.

Hands-On: From Zero to Protected

Here is a minimal middleware that enables CRS and turns the engine on:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: waf
  namespace: apps
spec:
  plugin:
    coraza:
      crsEnabled: true
      directives:
        - Include @coraza.conf-recommended
        - Include @crs-setup.conf.example
        - Include @owasp_crs/*.conf
        - SecRuleEngine On
        - SecRequestBodyAccess On

Attach that middleware to an IngressRoute, and your service is protected.

Quick Start: A Local Demo You Can Actually Run

This kit uses k3d + Traefik Hub + Coraza + CRS, then runs Bruno (or Postman) tests to validate 200 (OK) vs 403 (Forbidden) behavior.

# 0) Unzip the the Workshop Repository
cd traefik-waf-test-kit

# 1) Create a local cluster
k3d cluster create traefik-waf \
  --port 80:80@loadbalancer \
  --port 443:443@loadbalancer \
  --port 8000:8000@loadbalancer \
  --k3s-arg "--disable=traefik@server:0"

# 2) Add Traefik Helm Repository
helm repo add --force-update traefik https://traefik.github.io/charts

# 3) Create Traefik Namespace and License
kubectl create namespace traefik

# Note: Generate your Traefik Hub token from https://hub.traefik.io  
# Replace <TRAEFIK_HUB_TOKEN> with the token from hub.traefik.io
kubectl create secret generic traefik-hub-license --namespace traefik --from-literal=token=<TRAEFIK_HUB_TOKEN>

# 4) Install Traefik Hub
helm upgrade --install --namespace traefik traefik traefik/traefik \
  --set hub.token=traefik-hub-license \
  --set ingressRoute.dashboard.enabled=true \
  --set ingressRoute.dashboard.matchRule='Host(`dashboard.docker.localhost`)' \
  --set ingressRoute.dashboard.entryPoints={web} \
  --set image.registry=ghcr.io --set image.repository=traefik/traefik-hub --set image.tag=v3.19.0

# 5) Create Application Namespace
kubectl create namespace apps

# 6) Deploy the httpbin app + WAF middlewares
kubectl apply -f httpbin-waf/

Now validate with a normal request and a SQL injection attempt:

# normal request -> 200 OK
curl -i "http://httpbin.traefik.localhost/waf/anything?test=hello"

# SQLi attempt -> 403 Forbidden
curl -i "http://httpbin.traefik.localhost/waf/anything?user=admin&password=test%27%20OR%20%271%27%3D%271"

You can also run the Bruno collection in bruno-waf-tests/ for full coverage across SQLi, XSS, bots, traversal, command injection, and sensitive files examples.

Want to see the full test suite in action? Try Traefik Hub with a complete WAF test kit, which includes 19 pre-built attack scenarios across 8 categories, plus the full middleware config with 50+ custom rules. Request your trial here.

How the Rules Work (So You Can Tune Them)

CRS is not just regex. It is a structured ruleset that targets specific attack classes and uses anomaly scoring to decide when to block. That is why tuning is possible without turning protection into guesswork.

SecLang rules are readable and direct. The syntax is:

SecRule VARIABLES "@OPERATOR OPERATOR_ARGUMENTS" "ACTIONS"

For example:

SecRule ARGS "@detectSQLi" "id:302,phase:2,deny,msg:'SQL Injection detected'"

This rule checks all request arguments (ARGS) for SQL injection patterns (@detectSQLi), runs during phase 2 of request processing, denies matching requests, and logs a message. The syntax is powerful enough to express complex conditions, yet readable enough for security teams to audit.

You can use CRS for baseline protection and then add tight custom rules for your app's specific needs:

# Admin path protection
- SecRule REQUEST_URI "@rx /admin" "id:102,phase:1,log,deny,status:403,msg:'Admin path blocked'"

# Bot and scanner detection
- SecRule REQUEST_HEADERS:User-Agent "@pm sqlmap nmap nikto" "id:501,phase:1,deny,msg:'Scanner detected'"

# Sensitive file protection
- SecRule REQUEST_URI "@pm .git .env .conf" "id:505,phase:1,deny,msg:'Sensitive file access'"

Rollout Strategy That Won't Break Prod

A safe rollout looks like this:

1. Detection-only: Deploy with SecRuleEngine DetectionOnly to see what would be blocked without impacting production. Let it run for at least a week to capture normal traffic patterns.

- SecRuleEngine DetectionOnly

2. Review: Analyze the logs. Group blocked requests by rule ID. Look for false positives (legitimate requests that triggered rules). Common culprits include rich text editors, JSON payloads with code snippets, and query strings that resemble attacks.

3. Tune: Create exclusion rules for legitimate patterns. Be surgical. Do not disable entire rule categories when a narrow exclusion will suffice. CRS uses anomaly scoring, so you can also tune the tx.inbound_anomaly_score_threshold variable.

4. Enforce: Switch to blocking mode once you are confident.

- SecRuleEngine On

5. Expand progressively: Start with your lowest-risk endpoints, verify everything works, then expand coverage. This reduces the blast radius if you missed something during testing.

This is the difference between "WAF as a checkbox" and "WAF that actually helps." It should block real attacks, not your legitimate users.

Production Considerations

Defense in Depth

WAF is one layer among many you can add. Combine it with:

Rate limiting: Prevent brute force and volumetric attacks
Authentication: OIDC, JWT, or OAuth2 for identity verification
Security headers: CSP, HSTS, X-Frame-Options for browser-side protection
mTLS: Certificate-based service-to-service authentication

Traefik Hub supports all of these as composable middleware. Your security configuration lives in Kubernetes manifests: version-controlled, reviewable in pull requests, and deployed alongside your applications through the same GitOps pipelines.

Monitoring and Observability

A WAF that silently blocks attacks is not giving you the security posture improvement you need. Enable structured JSON audit logging and forward to your centralized logging platform (ELK, Loki, Splunk). Set up alerts for:

Blocked request spikes: Often indicates an active attack. Correlate with source IPs.
New attack patterns: Watch for rule IDs you have not seen before.
False positive patterns: Track when legitimate users report issues.
Anomaly score distributions: Monitor how close legitimate traffic gets to your blocking threshold.

Treat WAF logs as a security data source, not just application logs. They tell you what attackers are trying to do, which can inform broader security improvements.

Enterprise Protection with Developer Workflows

The payoff is not just security. It is developer time and operational sanity.

No heavyweight appliance or weeks-long integration
Proven rules mapped to the OWASP Top 10 risks
Local demo and validation in minutes
Configuration lives in Git and deploys with your app
Transparent, auditable rules you can read and customize
Crowdsourced intelligence with constant ruleset updates

WAFs no longer require massive security budgets. With Traefik Hub API Gateway integrating Coraza & OWASP CRS, you deploy a modern WAF in a way that fits developer workflows while still delivering production-grade performance.

Ready to experience cloud-native WAF protection? Click below to request your free trial of Traefik Hub and get our complete WAF Test Kit. It includes the full middleware config with 50+ security rules, Bruno test suite with 19 attack scenarios across 8 categories, Kubernetes deployment manifests, and a step-by-step deployment guide.

Sources

Imperva 2025 Bad Bot Report - 51% non-human traffic
Data Breach Investigation Report 2025 - 34% increase in vulnerability exploitation
SQL Injection = 65% of Web App Attacks - Dark Reading
IBM Cost of Data Breach Report 2025 - $4.44M global avg, $10.22M U.S.
OWASP Top 10:2025
OWASP Core Rule Set
Coraza WAF
Coraza SecLang Syntax
Traefik WAF Solution

]]>

The Illusion of Safety: Why the Ingress NGINX Fork is Not a Security Strategy

Emile Vauge — Tue, 27 Jan 2026 19:57:22 GMT

When a building’s foundation cracks, you don’t repaint the walls. You move.

On November 11, 2025, the Kubernetes SIG Network and Security Response Committee announced the upcoming retirement of the Ingress NGINX Controller. The project is now in a "best-effort" maintenance phase until March 2026. After this date, there will be no further releases, bug fixes, or security updates.

On December 22, 2025, Chainguard announced a maintenance fork of ingress-nginx via its "EmeritOSS" program.

For platform engineers and CTOs, this creates a tempting narrative: "We don't have to migrate. We can just switch the image and stay safe."

Here is the problem: that conclusion confuses Extended Support with Artificial Life Support.

In software, extended support is standard practice for stable, mature platforms (like Ubuntu ESM or Traefik’s own LTS versions). But ingress-nginx isn’t retiring because it is old. It is retiring because its creators deemed its architecture fundamentally unsafe to maintain.

Supporting a solid architecture is stability. Supporting a broken architecture is a liability.

Even Chainguard is explicit about the intent. They are not continuing development. They are maintaining a working version to buy users time to move to a different solution, with CVEs addressed on a "best-effort" basis.

The Hard Truth: Why the Experts Walked Away

To understand why the fork is a strategic error, we have to look at why the project ended. The Kubernetes SIG Network and Security Response Committee didn't retire ingress-nginx solely due to a lack of resources. They retired it because the project suffered from what they officially termed "insurmountable technical debt."

According to the official retirement announcement, the project’s flexibility—once its greatest asset—had become its greatest vulnerability. The maintainers were explicit:

"What were once considered helpful options have sometimes come to be considered serious security flaws, such as the ability to add arbitrary NGINX configuration directives via the 'snippets' annotations."

If the original maintainers—the world's foremost experts on this codebase—could not secure this architecture without a total rewrite, it is improbable that a third-party vendor can do so with a "maintenance-only" approach.

The Fork Fallacy: You Might Patch CVEs, But You Can’t Patch a Fundamentally Broken Architecture

Chainguard is a leader in supply chain security, and their ability to package software is undisputed. However, the corruption in ingress-nginx isn't in the packaging; it's in the fundamental logic.

The fork promises to "keep the lights on" by updating dependencies and patching known CVEs. But this ignores the structural reality that makes the project insecure:

The Templating Trap (The Root Cause)

The core flaw of ingress-nginx is its reliance on text-based templating. It functions by reading Kubernetes resources and mashing them into a single, massive nginx.conf file using templates.

The Risk: Features like "configuration snippets" allow users to inject raw NGINX directives and arbitrary commands directly into this template. This architecture effectively means the controller is continuously compiling untrusted user input into its root configuration.
The Reality: Securing this model against injection attacks without removing the flexibility users rely on is nearly impossible. A maintenance fork cannot fix this without rewriting the controller entirely.

The Memory Safety Gap

ingress-nginx is built on NGINX, which is written in C. It is inherently vulnerable to memory-safety issues like buffer overflows. Modern infrastructure demands memory-safe languages. Traefik is built in Go, which eliminates entire classes of memory vulnerabilities by default. A fork cannot "patch" C into Go.

The “Cold Restart” Model

Because of the templating model, many configuration changes require regenerating the file and reloading the worker processes. At scale, this can cause latency spikes and dropped connections. Traefik was born in the cloud-native era; it is purely dynamic, updating routing tables instantly via the Kubernetes API without ever restarting.

The "Many Eyes" Myth

Open-source security relies on community scrutiny—the concept that "given enough eyeballs, all bugs are shallow." Since the fork, the project has seen minimal traction. Crucially, issues and PRs are turned off in the repository. This is not a thriving community revival; it is a quiet, vendor-specific repository. By switching to the fork, you trade a community-supported standard for a niche, proprietary patch set with significantly fewer eyes on the code.

The Strategic Pivot: Evolution, Not Stagnation

Sticking with ingress-nginx is a decision to invest in the past. The industry is moving toward the Gateway API, a standard that solves many of the original Ingress limitations.

Your strategy for 2026 should focus on adopting these modern standards, not propping up a 2015 architecture. However, we understand that "rewriting everything" is a non-starter for most teams.

The Solution: An On-Ramp, Not a Cliff

The reality is that the new Gateway API standard, though excellent, doesn't yet offer perfect feature parity with the complex annotations many users rely on in Ingress NGINX. Furthermore, moving dozens or even hundreds of existing ingress resources to a completely new configuration format presents a significant and complex re-platforming challenge for any team.

That is why we built the Ultimate Ingress NGINX Migration Kit. It is a comprehensive tool designed to assist you in planning a progressive migration strategy, bridging the gap between where you are (Legacy Ingress) and where you need to be (Traefik & Gateway API).

Our approach includes three primary steps:

1. The Automated Analysis (Audit Your Risk)

Don't guess how complex your migration will be; know for sure. Use our open-source Ingress NGINX Migration Tool.

What it does: Connects to your cluster, analyzes every Ingress resource, and generates a detailed report.
The value: It tells you exactly which annotations you are using and maps them to Traefik features, flagging any edge cases before you change a single line of code.

2. The Drop-In Replacement (The "Buy Time" Feature)

The biggest blocker to migration is rewriting thousands of lines of YAML. We solved this with the Traefik Ingress NGINX Provider.

What it does: Traefik natively reads, understands, and translates your existing ingress-nginx annotations into Traefik configuration.
The value: You can swap the controller binary today without rewriting your application manifests. This eliminates the "Big Bang" migration risk, lands you on a well-maintained controller, and allows you to modernize your config at your own pace.

3. The Bridge to Gateway API (The “Future-Proofed” Path)

Traefik offers full, production-ready support for the Gateway API today. By moving to Traefik via the Migration Kit, you gain the power of coexistence.

What it does: You can run your legacy Ingress resources (using NGINX annotations) side-by-side with modern Gateway API routes on the same controller.
The value: This allows for a truly gradual migration. You don't need to modernize everything at once—you can reconfigure your services to use the new standard piece-by-piece, while Traefik handles traffic for both simultaneously.

Don't Wait for the Next "IngressNightmare"

The retirement of ingress-nginx marks the end of an era. The current Chainguard fork, or any future one, may keep the lights on for a few more months, but it cannot fix the crumbling foundation that led the Kubernetes team to walk away.

Your Action Plan

1. Assess your exposure: Visit ingressnginxmigration.org to understand the timeline and risks.

2. Audit now and help shape the future: Run the migration tool to get your custom report:

$ curl -sSL https://raw.githubusercontent.com/traefik/ingress-nginx-migration/main/scripts/install.sh | bash
$ ingress-nginx-migration

💡 Pro Tip: Share your migration report with us (anonymized by design). We are actively adding support for more annotations based on community feedback. Your report helps us prioritize the features that matter most to your setup.

3. Decommission NGINX Securely: Use the Traefik Ingress NGINX Provider to switch controllers and decommission Ingress NGINX without the headaches.

4. Migrate to Gateway API Progressively: Take control of your Gateway API adoption by planning it on your own schedule. Use regular sprint cycles to assess readiness, and deploy only when you are ready.

Don't let your infrastructure rely on life support. Move to a platform that is active, modern, and secure by design.

]]>

Ingress NGINX Retirement: The Ultimate Migration Kit

Emile Vauge — Tue, 09 Dec 2025 01:54:39 GMT

The ticking clock for Kubernetes networking has a date: March 2026.

For years, Ingress NGINX has been the default choice for routing traffic in Kubernetes clusters. It has served the community well, but recent announcements from the Kubernetes project confirm what we anticipated: Ingress NGINX is retiring, in less than 120 days.

We previously covered the urgency of this move in our articles on migrating from Ingress NGINX to Traefik now and strategies for the transition. Today, we are taking the next step to support the community with dedicated tools and resources.

The Hard Deadline

The official deprecation timeline is set. By March 2026, the project will cease all support. No patches. No updates. No security fixes.

If you are running Kubernetes in production, this is a critical risk. Running an unmaintained ingress controller exposes your infrastructure to future vulnerabilities that will simply go unpatched.

You Need a Plan

Panic is not a strategy, but preparation is. The move to the new Gateway API standard is the ultimate destination for the ecosystem, but rewriting every ingress route and reconfiguration every load balancer overnight is rarely feasible.

You need a strategy that covers both immediate security and future modernization:

Short Term (The Safety Net): Traefik with its NGINX provider is currently the only available solution on the market to provide native support for Ingress NGINX custom annotations. This makes Traefik the only true drop-in replacement, allowing you to migrate from Ingress NGINX with no changes required to your existing Ingress resources. You stay secure and supported without the need for a massive rewrite.
Long Term (The Modernization): Once you are on a stable, supported platform, you can gradually migrate to the Gateway API at your own pace, future-proofing your stack.

We Built The Ultimate Migration Kit

To help the community navigate this transition, we have consolidated all relevant information into a comprehensive project. We launched a dedicated website to guide you through every step of the process.

👉 The Migration Guide: ingressnginxmigration.org This website is your central resource. It breaks down the timeline, the risks, and provides a clear decision tree for your migration strategy.

Screenshot of ingressnginxmigration.org

👉 The Migration Tool: traefik/ingress-nginx-migration We didn't just write a guide; we built a powerful CLI tool to automate the pain away.

Screenshot of Migration Tool

Right now, the tool connects to your cluster, analyzes all your existing Ingress resources, and generates a comprehensive migration report. It identifies exactly which annotations you are using and maps out your path to Traefik’s NGINX-compatible provider.

We are actively developing this tool, with upcoming updates focused on Gateway API. Soon, it will not only help you migrate to a secure Ingress controller but also assist in migrating to the new Gateway API standard.

Making Migration a Non-Event

At Traefik Labs, we are dedicated to helping the community navigate this shift smoothly. Our goal is to turn this deprecation into a non-event for your infrastructure.

If you want to dive deeper into the migration process, join us at the webinar 100 Days to Migrate: Why Traefik is the Only Drop-in Replacement for Ingress NGINX.

Both the migration tool and the Traefik NGINX provider are fully Open Source. We built them for the community, and we believe in the power of collaboration. We invite you to try them, examine the code, and contribute. Whether it’s adding support for a specific annotation or improving the analysis logic, your contributions help make the path easier for everyone.

Don't wait for the deadline. Start planning your migration today.

Essential Resources

Check out the Guide
Ingress NGINX Migration Tool
Traefik NGINX Provider Documentation
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The Traefik Labs Manifesto: The Case for a Unified Runtime Layer

Sudeep Goswami — Thu, 04 Dec 2025 22:47:19 GMT

Modern infrastructure has built itself into a corner.

Over two decades of rational technology choices have produced an irrational result: a fractured architecture that slows development, complicates security, and prevents organizations from operating at the speed their business demands. This is not a problem we can optimize our way out of. It requires a fundamental shift in how we think about application infrastructure.

We believe the enterprise needs a unified runtime layer, not as a future luxury, but as an immediate necessity.

This is Traefik Labs' position. We have built our entire platform around this belief, and the industry is starting to recognize why it matters.

The Problem: Fragmentation Has Become the Default

Modern organizations run on a mosaic of execution environments: virtual machines, Kubernetes clusters, serverless platforms, public clouds, private clouds, sovereign regions, and edge locations. Each environment comes with its own governance model, networking assumptions, security policies, and operational culture. Each choice made sense in isolation. Collectively, they created chaos.

Every environment has its own ingress pattern, every system its own access rules, every team its own API conventions. Every workflow must navigate inconsistent connectivity, and every security model must be re-implemented. We have normalized dysfunction.

Developers spend more time wiring systems together than building features. Security teams struggle to enforce policies across incompatible platforms, while operations teams manage overlapping tools that don't talk to each other. We tell ourselves this is the cost of innovation, but in reality, it is the cost of fragmentation. It is holding the enterprise back.

The Tipping Point: AI Cannot Tolerate What Humans Have Learned to Ignore

Humans have learned to socially engineer solutions to these roadblocks; we open tickets, manually bridge gaps, and route around broken systems. AI cannot do this.

While an LLM can instantly adapt to a changing API schema or logic flow, it cannot bypass the hard physics of the network. When an AI agent hits a firewall, a missing route, or an ambiguous identity policy, it cannot "negotiate" access. It simply fails.

Agentic systems require stable APIs, trustworthy identity, predictable governance, end-to-end visibility, context-aware access, and consistency across environments. They need what the enterprise has historically struggled to provide: a coherent runtime.

AI is no longer confined to specialized projects; inference is becoming ubiquitous. Every application will soon make calls to language models, vision models, embedding models, and specialized AI services. These inference workloads will sit alongside traditional API traffic, demanding the same reliability, security, and observability as mission-critical services require today.

Agentic infrastructure is not creating a new problem. It is exposing the problem we have been tolerating for years, serving as the stress test that proves the current model is broken and forcing the conversation we should have had long ago.

The infrastructure that served the last era cannot serve the next one.

The Solution: A Unified Runtime Layer for All Workloads

A unified runtime layer sits above the execution environments but below the application logic. It standardizes how traffic flows, how identity is enforced, how APIs are exposed, how workloads are protected, and how intelligence is applied. Rather than replacing infrastructure, this approach brings coherence to it.

Unified Ingress Diagram

Our Principles

Consistent governance everywhere. Policies must be defined once and applied universally across Kubernetes, VMs, and edge nodes. By enforcing standards like OAS (OpenAPI Specification) at the definition level, we transform loose integrations into strict, governable contracts.
Universal connectivity. VMs, containers, functions, and legacy systems must behave like a single addressable estate. Connectivity is not negotiable. It is the baseline requirement for everything that follows.
Intelligent routing by default. Modern workloads need more than Layer 7 routing. They need context-aware decisions, semantic caching, and policy enforcement that understands intent rather than just packets.
Security as a standard, not an add-on. Security policies are agnostic to the underlying infrastructure. No exceptions, no workarounds, no environment-specific compromises.
Complete observability across all traffic. Every request, response, and failure must be traceable. Without observability, there is no understanding. And without understanding, there is no control.
Sovereignty and edge readiness. The runtime must function wherever the enterprise operates: global cloud regions, regulated environments, private clusters, and air-gapped infrastructures. Geography cannot break the model.
Developer experience is not cosmetic. If it is hard to use, it will not be used. Simplicity is a requirement for adoption, never a compromise on capability.

These principles define the application fabric enterprises need. They make applications more reliable, security more robust, and create the foundation AI requires to function.

The Traefik Vision: Application Intelligence as the Connective Layer

At Traefik Labs, we’ve spent years building the Application Intelligence Platform: a unified runtime solution that acts as the connective, governed, and intelligent layer for all application and AI traffic across an enterprise.

Our product suite embodies this vision:

Traefik Proxy unifies ingress, routing, and traffic management across containers (Kubernetes, ECS, Swarm, Nomad, etc.), VMs, and hybrid architectures.
Traefik API Gateway brings consistent identity, policy enforcement, and visibility to every environment.
Traefik AI Gateway manages AI inference traffic as a first-class workload, handling guardrails, cost controls, and semantic caching.
Traefik MCP Gateway ensures agentic AI systems interact with enterprise infrastructure safely and predictably.
Traefik API Management provides comprehensive governance, from developer portals to GitOps-driven operations at scale.

Together, they form a coherent architecture. Organizations can start with any Traefik Labs product and add capabilities seamlessly, in seconds, as requirements change. This makes our entire suite less a collection of tools and more a platform with a clear point of view.

Why This Matters

The organization that unifies its runtime layer will move faster, operate more securely, and adopt AI more confidently than those still managing fragmented infrastructure. This is a present requirement, not a distant future state.

Organizations are realizing that the runtime is the control plane for modern applications. Governance is the foundation, not a feature. Connectivity must be consistent everywhere, and APIs have become the enterprise's most critical interface. AI inference workloads will flow through the same infrastructure as every other service, requiring unified observability.

The question is no longer whether to unify, but when.

Our Commitment

Traefik Labs has always believed that connectivity should never be the limiting factor for innovation. As the enterprise enters the age of AI, this belief matters more than ever.

We have built the unified runtime layer that addresses the fragmentation enterprises face today while enabling the intelligent workloads they are deploying now. This architecture spans clouds, data centers, VMs, and edge, with a foundation in open standards and enterprise-grade reliability.

This is the foundation the industry needs. This is the architecture we have built. This is the future we are committed to advancing.

The unified runtime layer is no longer optional; it is the foundation that makes everything else possible.

]]>

AI Sovereignty: Why Control Is Your Ultimate Operating Leverage

Sudeep Goswami — Tue, 02 Dec 2025 23:43:28 GMT

Six months ago, the conversations happening in boardrooms across regulated industries were remarkably consistent. CIOs and CTOs were all fielding the same urgent question: "How do we adopt AI?"

Today, the conversation has flipped. The pressure to adopt hasn't gone away, but a new, more critical question has taken over: "How do we control AI?".

We are living through one of the most significant inflection points in enterprise technology. Artificial intelligence has moved from "science project" status to full-scale production at a rate faster than any technology in history. But this speed has come at a cost. It has created a crisis of AI Sprawl.

Developers started with personal ChatGPT accounts, then moved to corporate accounts, and finally to embedded APIs. Now, API keys are scattered across repositories, "shadow agents" are running in production, and billing statements are arriving with numbers that no one can fully explain.

For most companies, this is a governance headache. For regulated industries—Defense, Healthcare, Financial Services, and the Public Sector—it is an existential risk. The answer to this crisis lies in a concept that is frequently discussed but rarely defined correctly: AI Sovereignty.

The Illusion of Control: What Sovereignty Isn't

The term "sovereignty" is often thrown around loosely in technology discussions. Before we can understand what it requires, we need to be precise about what it is not. There are three common myths that lull organizations into a false sense of security.

1. The Data Residency Myth

Many organizations believe that if they store their data in a specific geography—say, a data center in Frankfurt or a specific US region—they are sovereign. But data residency is not sovereignty. If your data sits in Germany but the control plane managing it lives in Virginia, or if the AI model processes that data via an API call to a US-based hyperscaler, you do not have control. If the "brain" of the operation is elsewhere, the "body" is not sovereign.

2. The Hybrid Cloud Trap

"Hybrid cloud" is often sold as the solution to sovereignty, but it frequently masks deep dependencies. If your on-premise AI workload relies on public cloud services for authentication, billing, or management, you have created a fragility point. If your internet connection goes down, or if the cloud provider suffers an outage, your "sovereign" AI stops working. That isn't sovereignty; it is dependency.

3. Vendor-Managed Sovereignty

Perhaps the most dangerous myth is that you can buy sovereignty as a managed service. If you are operating on someone else’s infrastructure, under their terms, with their ability to change the rules, pricing, or availability at any moment, you are not in control. You are simply a tenant. True sovereignty means you are not operating on someone else's terms.

Defining True Sovereignty

If those are the illusions, what is the reality? In our new eBook, The Sovereign AI Infrastructure Imperative, we define true sovereignty through three non-negotiable capabilities.

First, You Need Architectural Control.

This means you can run your entire AI stack—gateway, models, safety systems, and governance—in your own environment. There are no required connections to external services. There is no "phone home" telemetry. If you pull the network cable, the system keeps working. For defense and intelligence sectors, this isn't a "nice to have"—it is an operational requirement.

Second, You Need Portable Governance.

Your policies, security controls, and audit trails must follow your workload. The same governance rules should apply whether you are running in the public cloud, on-premises, or in an air-gapped bunker. Your governance should be defined as code, not by clicking buttons in a proprietary cloud console that you can't export.

Third, You Need Escape Velocity.

This is the litmus test for sovereignty: Can you leave? True sovereignty means you are not locked into proprietary APIs or opaque platforms. You own the architecture. If a vendor changes their terms or a geopolitical event makes your current setup risky, you can migrate your entire stack without rewriting it.

Simply put: You own it, you control it, and you can move it.

The New Threat: The Agent Governance Crisis

Why is this definition becoming so critical right now? Because the nature of AI is changing. We are moving from "Chat" to "Action."

Six months ago, the risk was mostly about a user pasting sensitive data into a chatbot. Today, we are seeing the rise of Agentic AI—systems that use tools like the Model Context Protocol (MCP) to take actions. These agents read databases, call internal APIs, modify records, send emails, and execute code.

This breaks the traditional security model. In the past, you gave a human employee, let's call her Sarah, access to the database. If Sarah did something wrong, she was accountable. But agents are different. You cannot simply "authenticate once and trust forever." An agent might be doing exactly what it was prompted to do, but that prompt might be adversarial. Or the agent might be hallucinating. Or it might be exploring a solution path you didn't anticipate.

If you rely on a cloud provider for your governance, you are asking an external entity to police your internal agents. That introduces latency, metadata leakage, and security risks that regulated industries cannot accept. You need a security model that assumes agents are unprivileged actors requiring continuous authorization—and that authorization must happen on your infrastructure.

The Architecture of Control

To solve this, organizations need to move beyond simple "perimeter security." You cannot just put a firewall around your AI and hope for the best. You need defense in depth.

We have previously detailed the technical implementation of this approach, which we call the Triple Gate Pattern. It involves three distinct layers of defense:

The AI Gateway to secure the conversation (handling topic control and jailbreak detection).
The MCP Gateway to govern the tools (controlling what agents can actually do).
The API Gateway to protect the backend systems.

For a technical deep dive on how to implement these defensive layers, read our post: The Triple AI Security Gap.

The connection to sovereignty here is vital: You cannot effectively implement the Triple Gate Pattern if you are dependent on the cloud.

Traditional AI safety architectures rely on calling an external API to check if a prompt is safe. This creates three problems. First, Network Dependency: if you are in a defense installation without internet, you can't make that call. Second, Fragility: if the safety API goes down, your operations halt. Third, Metadata Leakage: even if you aren't sending the full data, you are revealing operational patterns to a third party.

True sovereignty requires that these safety checks—the NVIDIA NIMs, the policy enforcement, the agent governance—run locally on your hardware. The architecture must be offline-capable by default.

Sovereignty as Operating Leverage

We often hear sovereignty described as an insurance policy—something you buy to avoid fines from the EU AI Act or to comply with HIPAA. While that is true, it misses the bigger picture.

True sovereignty is offensive operating leverage.

When you build on truly portable, self-hosted infrastructure from day one, you gain strategic advantages that cloud-dependent competitors lack.

1. Negotiating Power

When you have "escape velocity"—the proven ability to move your stack without rewriting it—your relationship with vendors changes. You are no longer a captive customer who has to accept every price hike or term change. You are a partner with options. That optionality has real monetary value.

2. Deployment Agility

This is particularly valuable for Financial Services. These organizations often want to develop in the cloud to move fast but must deploy on-premises for compliance. If your governance is tied to a specific cloud, you have to rewrite everything when you move to production. With sovereign infrastructure, you develop once and deploy anywhere. Your governance code looks the same in AWS as it does in your private data center.

3. Uncompromised Trust

In industries like healthcare and defense, trust is the product. Patients and citizens are asking, "Where does the AI run? Who can see my data?" If your answer is, "It's in the cloud and we trust the vendor," that is no longer acceptable. Being able to prove—architecturally—that no telemetry leaves your environment is a massive competitive differentiator.

The Strategic Choice

We are early in this market. Most enterprises are still in the "Cloud-First" mindset, prioritizing convenience over control. But the organizations that are thinking ahead are asking a different question: "Are we building on infrastructure that gives us options, or are we making decisions now that eliminate sovereignty as a possibility later?".

The narrative has long been that you have to choose. You can have cutting-edge AI in the cloud, or you can have control on-premises. That is a false choice. With the right infrastructure—portable, declarative, and self-hosted—you can have both.

You can innovate fast and maintain control. But you have to architect for it now.

Go Deeper

How do you build an AI stack that creates this kind of leverage? We explore the architectural principles, the agent governance crisis, and the future of regulated AI in our latest guide.

]]>

Cilium + Traefik: The Superhighway and Traffic Cop Pattern for Multi-Cloud Networking

Immánuel Fodor — Tue, 02 Dec 2025 22:57:48 GMT

It's 3 AM. You're the on-call engineer, and your globally distributed payment API is timing out on AWS, but running fine on Azure. The API is spread across multiple clouds for high availability and low latency. You pull up three different dashboards—AWS CloudWatch, Azure Monitor, and your homegrown Grafana—trying to correlate logs across environments.

After an hour of intense archaeology, bug hunting, documentation review, and support forum reading, you discover the culprit: AWS ALB's default timeout of 60 seconds versus Azure App Gateway's 90-second default is causing long-running transactions to time out. It's the same application code, different cloud plumbing, and production is still down.

Different provider settings are the hidden tax of multi-cloud architecture.

We tell ourselves, "Kubernetes abstracts the infrastructure." That's true for compute. But for networking? Kubernetes gives you portability at Layer 7 while leaving you stranded in the quicksand of Layers 3 and 4. The networking rules in EKS differ from those in AKS. Load balancers speak different dialects. At scale, your networking configuration compounds with every new service until troubleshooting feels like defusing a bomb in the dark.

It's like driving across Europe in the 1970s. Every border crossing means new road signs, new traffic laws, and a new currency to fumble with.

But there's a better way. A new architectural pattern is emerging that actually delivers on Kubernetes' portability promise. It combines the kernel-level performance of Cilium (the Superhighway) with the application-layer intelligence of Traefik (the Traffic Cop).

For platform engineers tired of maintaining three different networking stacks, here's why this combo is becoming the new standard.

The Legacy Bottleneck: When Every Packet Hits a Red Light

Let's be honest about what we're replacing.

Traditional Kubernetes networking routes traffic through Linux firewall tools like iptables. This technology, designed in the 1990s, was built for firewalls, not for orchestrating thousands of microservices. Every packet that enters a node triggers a linear scan through potentially thousands of rules. The more services and rules you add, the slower every packet gets processed, and the more difficult it becomes to debug.

Picture a city where every intersection has a stop sign and a traffic cop manually checking every car's paperwork. That's iptables. It works fine for 10 cars (services). It becomes complex at 100 rules, and a maintenance headache at 1,000+.

The symptoms are familiar:

CPU spikes during deployment rollouts as firewall chains rebuild due to the rapid creation and destruction of containers.
Unpredictable latency as rule chains grow longer.
Debugging that requires deep Linux kernel knowledge.
Connection tracking tables that overflow under load.

The Superhighway: How Cilium Rewrites the Rules

The breakthrough came from eBPF (Extended Berkeley Packet Filter). Cilium is its best implementation for Kubernetes networking.

Instead of sequential rule checking, Cilium uses hash maps for instant lookups. Crucially, instead of context-switching between user space and kernel space, eBPF programs run directly in the kernel. It's the difference between stopping at every intersection versus taking a grade-separated freeway at constant speed.

The Real Shift: Cloud Providers Are Standardizing on Cilium

This is what changes the game for multi-cloud architecture: for the first time, the networking data plane is becoming identical across clouds.

Azure: "Azure CNI Powered by Cilium" is now the recommended option for AKS
Google Cloud: "GKE Dataplane V2" is Cilium under the hood
AWS: Cilium is the de facto choice for performance-conscious EKS deployments

The road surface is the same whether you're running in Virginia, Dublin, or Singapore. This means:

Predictable latency: Sub-millisecond service mesh overhead regardless of cloud
Consistent observability: Hubble flow logs look the same everywhere
Portable NetworkPolicies: Your security rules work identically across environments
No more cloud-specific tuning: The kernel-level plumbing just works

The Traffic Cop: Why Speed Alone Isn't Enough

A frictionless highway is powerful, but speed without direction is just a fast way to crash. You need intelligent control at the application layer. You need someone at the exit ramp deciding which requests go where, enforcing speed limits, and blocking bad actors.

You need Traefik.

The Multi-Cloud Load Balancer Problem

Most teams hit multi-cloud and immediately face this choice:

Use each cloud's native load balancer (AWS ALB, Azure App Gateway, GCP Load Balancer, etc.)
Deploy and maintain your own Ingress Controller

Option 1 seems easier until you realize you're hiring a different traffic cop for every city ... and they don't speak the same language or follow the same rules:

AWS ALB:

annotations:
  alb.ingress.kubernetes.io/scheme: internet-facing
  alb.ingress.kubernetes.io/target-type: ip
  alb.ingress.kubernetes.io/healthcheck-path: /health

Azure App Gateway:

annotations:
  appgw.ingress.kubernetes.io/health-probe-path: /health
  appgw.ingress.kubernetes.io/request-timeout: "90"

GCP Load Balancer:

annotations:
  cloud.google.com/neg: '{"ingress": true}'
  cloud.google.com/backend-config: '{"default": "backend-config"}'

Same intention. Three different languages. Three different configuration systems. Three different failure modes to debug at 3 AM.

Traefik: Bring Your Own Traffic Cop

When you deploy Traefik as your Ingress Controller across all clouds, you establish universal laws:

One Configuration Language

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: payment-api
spec:
  routes:
  - match: Host(`api.example.com`) && PathPrefix(`/payments`)
    kind: Rule
    services:
    - name: payment-svc
      port: 8080
    middlewares:
    - name: rate-limit
    - name: auth-oidc

This YAML works identically on AWS, Azure, and GCP. Change it once, deploy everywhere.

Portable Intelligence

Rate-limiting rules that follow your app across clouds
OIDC/OAuth flows that don't require cloud-specific IAM translation when using a central Identity Provider (IdP)
Circuit breakers that understand your application's failure modes
A/B testing and canary deployments with consistent behavior

Developer Velocity
Your developers define routing via standard Kubernetes resources. They don't need to know whether they're deploying to EKS or AKS. The platform team maintains a single Traefik configuration, rather than three cloud-specific load balancer setups.

Why Not Cilium Ingress?

Cilium does offer a basic Ingress controller built on the eBPF data plane. However, ingress is not its primary focus, and it typically offers the bare minimum necessary for Layer 7 entry.

For complex, multi-cloud environments, Traefik is better suited as the Traffic Cop because it provides an application-centric control plane. Traefik delivers advanced Layer 7 features—like portable middleware, integrated OIDC/OAuth, A/B testing, and comprehensive circuit breakers—that are central to managing a complex application portfolio at scale.

The "Better Together" Synergy

The architecture gets elegant when you look at how these two layers interact.

Traefik operates at Layer 7 (HTTP/gRPC), making intelligent routing decisions based on headers, paths, and application-level logic. Cilium operates at Layer 3 and 4, handling the actual packet forwarding with kernel-level efficiency.

The handoff is seamless:

External request hits your LoadBalancer Service (backed by a simple cloud LB)
Traefik receives the request, processes middleware, terminates TLS, and selects a backend
Traefik hands off to a Kubernetes Service (just an IP:port)
Cilium bypasses iptables entirely, using eBPF to route directly to the pod
Response follows the same optimized path back

The result: You get Layer 7 intelligence without Layer 3/4 bottlenecks. No tradeoffs.

Full-Stack Observability: The "X-Ray" Effect

This multi-layer observability is the key to platform team sanity.

If you've ever debugged a 502 Bad Gateway error, you know the frustration:

Traefik logs: "Request sent to backend, connection refused".
Application logs: "I never received a request".
Network monitoring: "I see packets moving, but no idea what they contain".

Three different monitoring systems, three different stories, zero root cause. This is the observability air gap that kills productivity.

The breakthrough: Distributed tracing stitched across network layers. Traefik and Cilium solve multi-layer observability together through a deceptively simple mechanism:

Traefik mints (or reuses) the trace ID: When a request hits your edge, Traefik generates a unique TraceParent header (OpenTelemetry W3C standard), or reuses one if it already exists. Think of this as stamping a license plate on the car before it enters the highway.
Cilium carries the context: As the request flows through the network, Cilium's Hubble observability engine reads HTTP headers at the kernel level, without slowing the packet down. It sees the same trace ID and logs the network path, latency, and any TCP-level failures.
One timeline, complete picture: In your observability backend (Grafana Tempo, Jaeger, Datadog), you search for that trace ID and get:

Edge ingress at Traefik (Layer 7)
Network path through Cilium with exact latency and packet drops (Layer 3/4)
Application processing time (Layer 7)

Real-World Example
A developer reports "users in Tokyo are seeing 2-second response times". You pull up the trace ID:

Traefik reports: Request took 2100ms total, 2000ms waiting for backend.
Hubble reports: TCP connection from Tokyo pod exists to both Tokyo database AND us-east database.
Application reports: Query took 50ms locally, 1950ms on cross-region query.
Root cause: Misconfigured service selector routing 10% of Tokyo traffic across continents.

Same debugging workflow whether you're troubleshooting EKS, AKS, or GKE.

The Technical Detail (Read This Before Deploying)
By default, Cilium prioritizes maximum speed, so it acts like a courier who reads the address on the box (IP/Port) without opening it. This makes it incredibly fast, but it means it can't see the specific "Trace ID" stamped inside the package headers.

To connect the dots between Traefik and the network, you must explicitly enable "Deep Packet Inspection" for your specific services. This is done using a CiliumNetworkPolicy.

Think of this as flipping a switch that says, "For this specific application, it's okay to peek inside the envelope to read the tracking number." It is a simple, one-time configuration per namespace, but without it, your observability layers won't communicate.

The Honest Complexity Tradeoff

Let's be clear: This isn't the right stack for everyone.

You probably don't need this if:

You're running on a single cloud with no plans to expand.
You have 10 microservices and 3 engineers.
Your cloud provider's defaults work fine for your scale.

This stack pays off when:

You're running production workloads across 2+ clouds.
You're managing 50+ microservices with complex routing needs.
You need consistent security policies and observability across environments.
Your platform team is tired of maintaining cloud-specific configurations.
Developer velocity matters more than "using cloud-native defaults".

The upfront complexity is real. You're deploying and managing Cilium and Traefik instead of clicking "enable" in a cloud console. But the payoff is a platform that your team masters once and runs everywhere.

The Bottom Line: Platform Team Sanity as a Service

Multi-cloud isn't going away. Whether you're pursuing vendor negotiation leverage, data sovereignty requirements, or genuinely distributed architecture, you need networking that works the same everywhere.

The old approach (maintaining separate networking configurations per cloud) doesn't scale. Your platform team becomes translators, converting concepts between AWS-speak, Azure-speak, and GCP-speak. Your developers slow down, waiting for platform teams to implement the same feature three different ways.

The Cilium + Traefik stack inverts this:

Cilium: Universal, high-performance data plane (the cloud providers are standardizing on it anyway)
Traefik: Portable, developer-friendly control plane (you own it, not the cloud vendor)

You get predictable performance, consistent debugging, and configurations that travel with your applications. Your platform team maintains one networking stack, not three. Your developers deploy with confidence, knowing the rules are the same everywhere.

The roads are open. The traffic is moving. And for the first time, the highway and the traffic cop speak the same language. No matter which cloud you're driving through.

Ready to build a truly portable Kubernetes platform?
Start with Cilium for your CNI and Traefik for your Ingress Controller. You'll thank yourself the next time production spikes.

]]>

The Hub-and-Spoke Ingress Pattern: Unifying EKS, ECS, and EC2 at Scale

Zaid Albirawi — Sun, 30 Nov 2025 18:27:52 GMT

Large-scale AWS architectures rarely maintain homogeneity over time. They evolve through eras.

A service might begin life on Amazon EC2 because the team needed full control over the OS kernel. Years later, a new initiative embraces Amazon EKS for microservices. Meanwhile, other teams deploy onto Amazon ECS or AWS Fargate because the bursty, event-driven nature of their workloads demands it.

Individually, these are sound technical decisions. EC2 offers stability; EKS offers orchestration; ECS offers simplicity. The problem does not lie with the platforms themselves, but with how they are exposed to the outside world.

As these compute "islands" grow, they create a fractured edge. EKS uses Ingress controllers; EC2 relies on ALBs/NLBs with custom proxy logic; ECS uses ALBs mapped 1-to-1 with services. The result is operational incoherence: inconsistent authentication, fragmented routing rules, and a nightmare for anyone trying to debug a request that spans multiple environments.

In our latest technical AWS builders-style ebook, Unifying Ingress Across Distributed AWS Compute Environments, we detail an architectural pattern that solves this: the Hub-and-Spoke Ingress Model.

Here is a deep dive into how it works and how to implement it using the Traefik AWS Elastic Provider.

The Anatomy of Fragmentation

Organizations do not set out to build fragmented systems; fragmentation is a side effect of growth.

When you view these platforms from the inside, they make sense. But from the application edge—where users and clients interact with your system—the cracks begin to show.

Identity Fragmentation: EKS might validate JWTs via middleware, while EC2 relies on a totally different auth proxy.
Routing Complexity: Migrating a service from EC2 to ECS often requires DNS cutovers because there is no single routing plane that spans both.
Security Risk: To let a central ingress talk to backend services across VPCs, teams often widen Security Group rules or create permissive IAM roles, expanding the blast radius.

To solve this, we need an architecture that can see the entire system from the outside while interacting with each environment from the inside—without breaking isolation boundaries.

The Hub-and-Spoke Architecture

The solution is to decouple the global entry point from the local service discovery. We call this the Hub-and-Spoke model.

The Unified Ingress as the Hub

For the hub, you need a "Unified Ingress," a Traefik instance running on your primary compute platform (often EKS). It is responsible for high-level concerns: terminating TLS, validating identity (OIDC/JWT), applying global rate limits, and handling the routing logic.

The Compute Spokes

The spokes in our Hub-and-Spoke Model are lightweight Traefik instances deployed in remote environments—on EC2 instances, in ECS clusters, or in other EKS clusters.

Crucially, the Unified Ingress does not query the AWS API to find services. Instead, it polls the spokes. Each spoke inspects its local environment (via a Docker socket, the ECS API, or local tags) and exposes a sanitized list of available services via an internal API endpoint.

This separation of concerns yields a modular ingress fabric:

The spokes are authoritative for their own environments.
The Unified Ingress hub is authoritative for routing.
Neither requires deep knowledge of the other's internal workings.

Why Polling? (The Security Argument)

In modern cloud-native development, "polling" is often considered a dirty word compared to event-driven architectures. However, in a multi-account, multi-VPC AWS environment, polling is actually the superior architectural choice for three reasons.

1. Eliminating Cross-Account IAM Complexity

If you use a centralized ingress controller that discovers services by querying the AWS API directly, that controller needs "God Mode" permissions. It needs to AssumeRole into every other AWS account to list instances and tasks. This creates a massive security risk—if the Ingress is compromised, the attacker can map your entire cloud estate.

With the Hub-and-Spoke Model, the Unified Ingress never touches AWS APIs outside its own account. It simply sends an HTTP request to the Spoke. The Spoke needs local permissions, but the Unified Ingress needs none. This aligns strictly with least-privilege principles.

2. Predictable Failure Modes

In distributed systems, simple failure modes are valuable. If a spoke goes down, the Unified Ingress stops polling it, times out, and removes those routes. When the spoke recovers, the routes return on the next interval. There is no "stale state" hidden in an event queue or a stream processor that stopped silently.

3. Tight Network Boundaries

Event-driven push models often require bidirectional connectivity. Polling is unidirectional. The Unified Ingress calls the spoke. The spoke requires no inbound rules beyond a single HTTPS listener on a specific port, accessible only from the Unified Ingress’s Security Group.

Solving the Identity Crisis

One of the most painful aspects of heterogeneous infrastructure is unifying authentication.

In a fragmented system, EKS might handle auth via an Ingress Controller, while legacy EC2 apps handle it in code. This makes it nearly impossible to enforce policies like "Only users with the admin claim in their JWT can access the /admin path on any service."

Because all external traffic flows through the Ingress, authentication becomes centralized by design.

The Unified Ingress receives the request.
The Identity Middleware validates the JWT against your OIDC provider (Cognito, Okta, etc.).
The Unified Ingress extracts claims (Group, Tenant, Role).
The Unified Ingress uses those claims to make routing decisions before forwarding the packet to the EKS, EC2, or ECS instances.

This transforms identity from a platform-specific headache into a consistent network primitive.

Cross-VPC Isolation

Many AWS environments segment workloads across multiple VPCs for compliance or to reduce the blast radius. Without a unified ingress, teams often resort to complex VPC Peering meshes or Transit Gateways with overly permissive security groups to allow east-west traffic.

The Hub-and-Spoke Model respects AWS’ "segmentation first" philosophy.

The Unified Ingress acts as the only cross-VPC touchpoint. Workloads in VPC A (ECS) never talk directly to Workloads in VPC B (EC2). They don't even need to know that the others exist. Security groups open only a narrow set of ports from the Unified Ingress’ CIDR range to the spoke’s Traefik instance.

This allows you to maintain strict network isolation while presenting a unified API surface to the internet.

The Traefik AWS Elastic Provider

While the Hub-and-Spoke is an architectural pattern, implementing it from scratch requires significant engineering. You need to build the spokes, standardize the metadata format, and build the reconciliation logic in the Unified Ingress.

This is why we built the Traefik AWS Elastic Provider.

It acts as the concrete realization of this architecture. It operationalizes the model by providing:

Unified Discovery Fabric: Spokes automatically enumerate local services (EC2 tags, ECS tasks, Kubernetes services) and normalize the data.
Resilience Logic: The provider handles the polling intervals, timeouts, and route eviction logic automatically.
Incremental Adoption: You can add a spoke to one ECS cluster today without touching your EKS or EC2 environments. The architecture evolves as you do.

When to Use This Pattern

This approach is not for everyone. If you are running a single EKS cluster in a single VPC, the AWS Load Balancer Controller is likely sufficient.

However, the Hub-and-Spoke Model offers substantial operational leverage for environments that exhibit:

Multiple Compute Platforms: A mix of EC2, ECS, and EKS.
Multi-VPC/Multi-Account: Strict segmentation requirements.
Migration Projects: Moving monoliths to microservices where traffic must be shifted gradually between platforms based on identity or weights.
High Governance: Environments requiring centralized audit trails for ingress traffic.

Conclusion: Consistency at the Edge

AWS will continue to evolve. New compute services will launch; existing ones will change. If your ingress strategy is tightly coupled to the underlying compute platform (e.g., "We use ALBs because we use ECS"), you will always be playing catch-up.

The Hub-and-Spoke architecture offers a principled path forward: a consistent edge atop an inherently diverse landscape. It allows you to choose the right compute tool for the job—EC2 for stateful legacy apps, ECS for batch jobs, EKS for microservices—without fragmenting the user experience.

It enables organizations to retain the advantages of heterogeneity while presenting a unified, reliable, and secure face to the world.

Read the Builders Guide

We have compiled a detailed AWS builders-style architectural guide including sequence diagrams for authentication flows, cross-VPC security group configurations, and failure recovery scenarios. Click the promo below to get your free copy.

]]>

120 Days Until Ingress NGINX Dies: Traefik is the Only True Drop-in Replacement

Emile Vauge — Wed, 19 Nov 2025 00:35:24 GMT

The Kubernetes community was shaken on November 12, 2025, when maintainers of the Ingress NGINX project announced: Ingress NGINX Controller will be retired in March 2026. After March 2026, there will be no releases, no bug fixes, and critically, no security updates.

Organizations now have just 4 months to migrate away from one of the most widely deployed ingress controllers in the ecosystem. This isn't theoretical—if you're running production workloads on Ingress NGINX after March 2026, you're running unmaintained software with potential security vulnerabilities. Historically, there have been critical vulnerabilities discovered in this project, including the #IngressNightmare vulnerabilities that demonstrated how attackers could compromise entire clusters through malicious configuration injection.

The timeline is brutal and many organizations are already behind schedule before they've started.

Why Traefik is Your Only Realistic Option

Earlier this year, we published a deep-dive into the #IngressNightmare situation and how to transition from Ingress NGINX to Traefik. In short, we added an Ingress NGINX compatibility layer to Traefik, introducing native support for many custom annotations and creating the only drop-in replacement for Ingress NGINX in the industry.

Here's the reality: every other ingress controller requires you to rewrite your configurations. F5 NGINX Ingress, HAProxy Ingress, cloud provider solutions—e.g., AKS, AWS, etc.—they all provide migration guides, annotation mappings, or promote a Gateway API transition, not migration compatibility. You'll need to manually convert every custom annotation, test every configuration, and essentially rebuild your ingress layer from scratch.

Traefik's NGINX Provider works differently. Most used nginx.ingress.kubernetes.io annotations are handled natively by Traefik. Your Ingress objects work unchanged. This is a huge differentiator compared to other solutions.

But compatibility alone isn't enough in this crisis. Traefik offers three critical advantages that make it the only realistic choice:

Security by Design: The #IngressNightmare vulnerabilities revealed fundamental architectural flaws in NGINX-based solutions. Traefik's architectural choices—Go over C/C++, static linking, structured parsing over templating—prevent the memory safety and configuration injection attacks that plague NGINX controllers.

Broad Ecosystem Adoption: With 3.4+ billion downloads, 58,000+ GitHub stars, and 900+ contributors, Traefik isn't experimental—it's battle-tested across startups and enterprises. Unlike solutions adapted for Kubernetes, Traefik was designed specifically for dynamic, cloud-native environments.

Gateway API Leadership: As a key contributor to Gateway API specification with full v1.4 support in Traefik v3.6, you're positioning for the future of Kubernetes networking, not just solving today's crisis. While others treat Gateway API as experimental, Traefik has made it core.

When you're facing a 120-day deadline with no margin for error, Traefik emerges as the obvious choice—seamlessly compatible with your existing deployments while positioning you for the future of cloud-native networking.

Stop the Bleeding First, Then Modernize

The retirement of Ingress NGINX is an urgent, time-sensitive problem demanding a structured migration strategy. It can be tempting to directly think of Gateway API as the obvious solution. Indeed, this new Kubernetes specification handles much more than the Ingress specification, and we are not only a big fan at Traefik Labs, but we are leading this standard as the future of Kubernetes networking.

However, Gateway API requires a complete migration of all your ingress resources to a new set of resources: Gateway, GatewayClass, HTTPRoute, TLSRoute, etc. And this is clearly not something you can (or should) do in a short period of time.

Trying to simultaneously solve the immediate crisis of a looming deprecation deadline and the long-term goal of ingress modernization is a mistake. The prudent, risk-averse approach separates these concerns into distinct, manageable phases.

Phase 1: Ingress NGINX Decommissioning. Get off Ingress NGINX before security updates stop. Migrate to Traefik with the NGINX Provider for drop-in compatibility. Maintain existing workflows and ensure production stability through the deadline.

Phase 2: Gateway API Modernization. Plan Gateway API adoption on your timeline, not under deadline pressure. Assess readiness during normal sprint cycles, test thoroughly, implement when business conditions are optimal.

Your immediate problem is Ingress NGINX retirement, not Ingress API modernization. Solve one crisis at a time.

How to Decommission Ingress NGINX, in a Nutshell

First, let’s dig deeper into the basics with a nominal use case. Let’s say:

You operate a Kubernetes cluster
There is a running ingress-nginx controller instance
A Kubernetes IngressClass named nginx with the controller value set to k8s.io/ingress-nginx and Kubernetes Ingresses have an ingressClassName set to nginx
And a Kubernetes service of type LoadBalancer exists exposing the ingress-nginx controller instance on port 80 and 443

You have this ingress with TLS and basic authentication deployed:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: ingress-with-basicauth
  namespace: default
  annotations:
    # Configure basic authentication for the Ingress
    nginx.ingress.kubernetes.io/auth-type: "basic"
    nginx.ingress.kubernetes.io/auth-secret-type: "auth-file"
    nginx.ingress.kubernetes.io/auth-secret: "default/basic-auth"
    nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"
    # SSL Redirect
    nginx.ingress.kubernetes.io/ssl-redirect: "true"

spec:
  ingressClassName: nginx
  rules:
    - host: myappdomain
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
  tls:
    - hosts:
        - myappdomain
      secretName: external-certs

---
kind: Secret
apiVersion: v1
metadata:
  name: basic-auth
  namespace: default
type: Opaque
data:
  # user:password
  auth: dXNlcjp7U0hBfVc2cGg1TW01UHo4R2dpVUxiUGd6RzM3bWo5Zz0=

Deploy Traefik alongside your existing Ingress NGINX setup. Traefik is installed in `ClusterIP` mode to be able to test the routing.

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ helm upgrade --install traefik traefik/traefik --namespace traefik \
  --create-namespace \
  --set="image.tag=v3.6.2" \
  --set="logs.general.level=DEBUG" \
  --set="service.type=ClusterIP" \
  --set="additionalArguments={--providers.kubernetesIngressNGINX}"

Check that the routing is working with a port-forward

$ kubectl port-forward -n traefik services/traefik 9000:80 9443:443
Forwarding from 127.0.0.1:9000 -> 8000
Forwarding from [::1]:9000 -> 8000
Forwarding from 127.0.0.1:9443 -> 8443
Forwarding from [::1]:9443 -> 8443

$ curl https://myappdomain:9443
401 Unauthorized

$ curl -u user:password https://myappdomain:9443
Hostname: whoami-b85fc56b4-5pvcv
...

Depending on how your Ingress NGINX controller has been deployed, you might also need to recreate the nginx IngressClass, if it’s removed when Nginx is uninstalled.

Then, you can remove the Ingress NGINX deployment, update the Traefik deployment to change the Service type to LoadBalancer, update the DNS record, and check that Traefik serves your untouched ingress resources 🙂

$ helm upgrade --install traefik traefik/traefik --namespace traefik \
  --create-namespace \
  --set="image.tag=v3.6.2" \
  --set="logs.general.level=DEBUG" \
  --set="service.type=LoadBalancer" \
  --set="additionalArguments={--providers.kubernetesIngressNGINX}"

$ curl -u user:password https://myappdomain              
Hostname: whoami-b85fc56b4-q6427
...

It’s as simple as that. nginx.ingress.kubernetes.io annotations are handled natively by Traefik and your ingresses stay unchanged.

For large Kubernetes deployments with numerous workloads and Ingress resources, we recommend a progressive migration. Deploy new clusters with Traefik, and then gradually transition your existing workloads and associated Ingress configurations to these new clusters. This approach allows for thorough validation, provides an easy rollback mechanism if necessary, and ensures the entire migration process remains manageable.

The Traefik Labs team possesses extensive experience with various Kubernetes deployments and is well-equipped to provide commercial assistance, helping you plan and execute your migration efficiently and seamlessly. Feel free to reach out to us if you need help.

Time to Execute

The retirement of Ingress NGINX is serious, but it's not the Kubernetes apocalypse. With 120 days remaining and Traefik's proven compatibility layer, you have enough time to migrate safely if you act now.

Don't let deadline pressure force you into premature Gateway API adoption or manual configuration rewrites. Traefik's NGINX Provider offers the most realistic path forward with immediate compatibility, proven track record, and clear upgrade path to Gateway API when you're ready.

Our current implementation focuses on supporting the most commonly used Ingress NGINX custom annotations, covering 80% of real-world usage patterns. If you need additional annotation support, we can implement it. Feel free to open GitHub issues in Traefik's repository to tell us your needs. Better yet, contribute directly—adding new annotation support is straightforward and helps the entire community.

Our goal is simple: make this crisis a non-event. Every organization should be able to migrate off Ingress NGINX safely and quickly, without disrupting their operations or forcing architectural decisions under pressure.

Essential Resources

Traefik NGINX Provider Documentation
Download Traefik 3.6: Docker Hub
Documentation: Complete Documentation
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The Feature You Didn't Know You Needed: Multi-Layer Routing in Traefik

Immánuel Fodor — Mon, 10 Nov 2025 22:15:41 GMT

Here's a question: how do you route traffic when the routing decision depends on information that doesn't exist in the original request?

Maybe you need to authenticate first, then route based on the user's role. Or check a feature flag service to decide whether someone gets the new microservices or the old monolith. Or look up a customer's subscription tier to route enterprise users to stable infrastructure while free users are directed to your canary.

The problem is that routing decisions are typically based on the information present in the request when it arrives—the path, the headers, the host, or other similar information. Even with Traefik’s flexible routing expression system, if the information you need for routing resides elsewhere (an auth service, a feature flag system, a database), you're stuck with workarounds.

Most teams end up doing one of these things:

Duplicating logic—Every microservice calls the auth service, parses JWTs, or checks feature flags independently
Building a routing proxy—A separate service sits in front of your main routing logic, makes the decision, adds routing headers (and adds latency)
Complex configuration—Giant routing files that try to handle every edge case and break when requirements change
Giving up—Route everyone to the same backend and handle the logic in application code

None of these are great. They're slow, brittle, or just push the problem somewhere else.

Traefik has just released multi-layer routing, and it solves this entire class of problems. Let me show you how.

The Problem: Risk Management Meets Routing Logic

Let's focus on one scenario that nearly every engineering team faces: safely deploying a new backend version.

You've built a new version of your API service. It's faster, cleaner, better—but it's also risky. Your enterprise customers have stricter SLAs. Breaking their production workflows would be catastrophic. But your free-tier users? They're well-suited for a canary-based deployment workflow, as free tiers typically have lesser SLA commitments and a lower risk of immediate business impact.

You need to route requests to https://api.example.com/users to different backends based on customer tier. Enterprise goes to stable, free tier goes to canary.

Your current options are all flawed.

The "naive" approach: Parse subscription level in every microservice. Now every service needs to be aware of customer tiers, call your billing system, handle caching, and deal with failures. You've just coupled your entire architecture to your billing logic.

The "proxy" approach: Build an external "customer routing service" that sits in front of your main routing logic. It checks the customer tier and adds custom headers to give hints to routing. Congratulations, you've added 30-500ms latency to every request and created a critical single point of failure. The added latency may even vary depending on your custom logic, database lookup speeds, distributed infrastructure, and many other factors.

The "manual config" approach: Update your routing configuration every time a customer upgrades or downgrades. This doesn't scale, it's error-prone, and you'll eventually route an enterprise customer to your canary at 2 AM by mistake.

The fundamental problem: Routing decisions are based on the original request. If your auth service knows the customer tier, even Traefik’s intelligent routing system can't use that information for routing—it's trapped in your auth system's response.

How Multi-Layer Routing Solves the Problem

Traefik's new multi-layer routing introduces hierarchical relationships between routers. Parent routers process requests with attached middleware, enriching them with additional context. Child routers then make routing decisions based on that enriched request.

Here’s the request flow:

Diagram of Multi-Layer Routing in Traefik

The router hierarchy has three types:

Root routers sit at the top with no parents, attached to entryPoints. They define tls and observability configuration. They can either have children (making them parent routers) or directly route to a service (standalone routers). Middleware is a key piece in request enrichment—root routers typically apply shared middleware that modifies requests for downstream routing decisions.

Intermediate routers reference their parents via parentRefs and can have their own children. They also cannot define a service and inherit entryPoints, tls, and observability from their root. Both intermediate and leaf routers can define additional middleware beyond what the parent applies. Importantly, they can't be called directly—requests must flow through their parent router, ensuring you can't circumvent authentication or other parent-level logic.

Leaf routers reference their parents via parentRefs and must define a service. They also inherit configuration from their root router and can apply their own middleware.

The magic is progressive request enrichment: each layer adds context that subsequent layers use for increasingly specific routing decisions. A key benefit is that you don't have to repeat the previous layers' matchers—they all add up as you traverse the routing tree. Authentication happens once at the parent, and routing happens at the child based on the enriched result.

Show, Don't Tell: Progressive Rollout Configuration

Let's solve that customer tier routing problem. Here's the complete configuration:

## Routing configuration
http:
  routers:
    # Root router - matches all API requests and applies auth
    api-parent:
      rule: "Host(`api.example.com`) && PathPrefix(`/`)"
      middlewares:
        - auth-with-tier
      entryPoints:
        - websecure
      tls: {}
      # No service defined - this is a parent router

    # Leaf router - routes enterprise customers to stable
    api-enterprise:
      rule: "HeaderRegexp(`X-Customer-Tier`, `(enterprise\|business)`)"
      service: stable-backend
      parentRefs:
        - api-parent

    # Leaf router - routes free tier to canary
    api-free:
      rule: "Header(`X-Customer-Tier`, `free`)"
      service: new-version
      parentRefs:
        - api-parent

  middlewares:
    # ForwardAuth validates request and enriches it with customer tier
    auth-with-tier:
      forwardAuth:
        address: "http://auth-service:8080/validate"
        authResponseHeaders:
          - X-Customer-Tier
          - X-Customer-Id
          - X-User-Email

  services:
    stable-backend:
      loadBalancer:
        servers:
          - url: "http://api-v1-stable:8080"

    new-version:
      loadBalancer:
        servers:
          - url: "http://api-v2-canary:8080"

What happens when a request arrives:

Request to https://api.example.com/users hits the websecure entrypoint
api-parent router matches based on Host and PathPrefix
auth-with-tier middleware executes, forwarding the request to http://auth-service:8080/validate
Auth service validates the JWT, looks up the customer’s subscription, and returns headers:
- X-Customer-Tier: enterprise
- X-Customer-Id: cust_abc123
- X-User-Email: admin@bigcorp.com
Traefik adds these headers to the request and evaluates child routers
api-enterprise router matches because X-Customer-Tier matches enterprise
Request is forwarded to stable-backend at http://api-v1-stable:8080
When a free-tier user makes the same request:
Steps 1-4 are identical, but the auth service returns X-Customer-Tier: free
api-free router matches instead
Request is forwarded to the new-version service at http://api-v2-canary:8080

The contrast: Without multi-layer routing, you'd need to either check customer tier in every microservice (slow, duplicated), build an external routing proxy (latency, complexity), or somehow encode customer tier in URLs or domains (terrible UX). Now it's declarative: authenticate once, enrich the request, and route based on the enriched context.

What Else Is Possible

Multi-layer routing unlocks patterns that were previously painful or impossible to implement cleanly. We show two additional examples below.

Authentication-Based Routing

The problem: You have admin endpoints and user endpoints, but you don't want separate URL paths. Admins calling /api/reports should see all reports. Regular users calling the same endpoint should only see their own reports. The user's role only exists after authentication.

The solution: The parent router authenticates all /api requests using the ForwardAuth middleware. The auth service validates credentials and returns X-User-Role header (either admin or user). Child routers match on that header: one routes admin requests to the admin-backend with full data access, another routes user requests to the user-backend with restricted access.

Key config pattern:

# Parent applies auth, no service
api-parent:
  rule: "PathPrefix(`/api`)"
  middlewares: [auth-middleware]

# Children route based on enriched headers
api-admin:
  rule: "Header(`X-User-Role`, `admin`)"
  service: admin-service
  parentRefs: [api-parent]

api-user:
  rule: "Header(`X-User-Role`, `user`)"  
  service: user-service
  parentRefs: [api-parent]

Use case: Multi-tenant SaaS platforms, admin panels with role-based access, internal tools with permission-based routing.

Feature Flag-Based Migration

The problem: You're migrating from a monolith to microservices. Some customers are on the new architecture, others on the old. Feature flag decisions reside in an external service (LaunchDarkly, Unleash, custom system). You need Traefik to route based on those flags without calling the feature flag service from every microservice.

The solution: Parent router calls your feature flag service via ForwardAuth or a custom plugin. The service evaluates flags for the user and returns X-Feature-Microservices: enabled or X-Feature-Microservices: disabled. Child routers match on that header and route to either the new microservices or the legacy monolith. One feature flag check, unlimited routing decisions downstream.

Key config pattern:

# Parent checks feature flags
migration-parent:
  rule: "Host(`app.example.com`)"
  middlewares: [feature-flag-checker]

# Route to microservices if enabled
microservices-route:
  rule: "Header(`X-Feature-Microservices`, `enabled`)"
  service: new-microservices
  parentRefs: [migration-parent]

# Route to monolith otherwise
monolith-route:
  rule: "Header(`X-Feature-Microservices`, `disabled`)"
  service: legacy-monolith
  parentRefs: [migration-parent]

Use case: Progressive monolith decomposition, gradual infrastructure migrations, per-customer feature rollouts, A/B testing at the gateway level without application changes.

When to Use This

Use multi-layer routing when:

✅ You're enriching requests (authentication, feature flags, customer context, or you name it) and need to route based on that enriched data

✅ You have shared middleware that applies broadly, but different routing logic after that middleware executes

✅ Your routing rules are becoming unmaintainable—complex combinations of conditions that would be clearer as hierarchical layers

✅ You're making the same external call (auth service, feature flags) across multiple routes and want to do it once

Skip multi-layer routing when:

⏩ You have simple routing needs—basic path or host matching doesn't need this complexity

⏩ You need Docker labels or standard Kubernetes Ingress—multi-layer routing at the time of writing only works with Kubernetes IngressRoute CRD, File providers (YAML/TOML), and KV stores (Consul, etcd, Redis, ZooKeeper).

Important consideration: Multi-layer routing changes your debugging model. You're now tracing requests through a hierarchy, rather than a single router. Enable Traefik's OpenTelemetry-based observability and distributed tracing—you need visibility into which routers match at each layer and why. Think of it like debugging microservices instead of a monolith: a little more moving parts, but clearer separation of concerns.

Performance note: Each layer adds minimal overhead (microseconds for rule evaluation). The middleware execution (ForwardAuth calls, etc.) is where latency happens, but you'd be doing those calls anyway—multi-layer routing just lets you do them once instead of repeatedly, and without any hacks that could bite back in the long run.

Get Started

The fastest way to understand multi-layer routing is to try it. The "aha moment" occurs when you realize how naturally complex routing logic can be decomposed into simple layers. Authentication, then routing. Context extraction, then decisions. One job per layer, rather than cramming everything into brittle, single-line rules or scattering it across multiple services.

Take inspiration from the examples above, make adjustments to match your environment, and deploy it to a test infrastructure. Make requests with different authentication credentials and observe Traefik routing them to different backends based on enriched request properties.

For complete documentation, additional examples, and provider-specific configuration formats, see the official Traefik multi-layer routing reference.

If you're still duplicating auth middleware across routes, building external routing services just to make decisions, parsing customer data in every microservice, or maintaining complex single-line routing rules with multiple conditions, there's a better way now.

The feature has just launched. The patterns are still emerging. What will you build with hierarchical routing layers?

Do you have a multi-layer use case in mind that you think is even more interesting than the ones shared above? I’d love to hear it at product@traefik.io!

]]>

Scale Serverless Workloads with Traefik & Knative

Shedrack Akintayo — Mon, 10 Nov 2025 22:02:14 GMT

Modern infrastructure spans diverse workload types—from traditional VMs to cloud-native containers and serverless functions. Managing networking across these environments has traditionally required multiple specialized proxies, increasing complexity and operational overhead.

With Traefik 3.6, we're changing that. The introduction of Knative provider support marks a key milestone: Traefik is now the only proxy that can seamlessly operate at the edge of Cloud Native workloads (containers), traditional workloads (Virtual Machines), and serverless workloads. This unprecedented versatility makes Traefik a truly unique platform capable of handling any kind of workload your infrastructure demands.

Knative, the leading platform for serverless workloads on Kubernetes, provides scale-to-zero capabilities, automatic scaling based on traffic, and sophisticated revision management. The Knative provider in Traefik automatically handles service discovery, scaling events, and traffic routing for these serverless workloads, providing a unified experience across your entire application landscape.

Teams can now use a single proxy solution for their complete infrastructure, dramatically simplifying operations while maintaining the specialized optimizations each workload type requires.

Prerequisites

Before we begin, make sure you have:

Helm 3.x installed
k3d installed for local testing
kubectl configured to access your cluster
Basic familiarity with Kubernetes concepts

Create a k3d Cluster

For this guide, we'll use k3d to create a local Kubernetes cluster with port mappings that allow direct access to services without port-forwarding:

k3d cluster create traefik \
  --port 80:80@loadbalancer \
  --port 443:443@loadbalancer \
  --k3s-arg "--disable=traefik@server:0"

This command creates a cluster with:

Port 80 and 443 mapped to your localhost
The default k3s Traefik installation disabled (we'll install our own)

If you're using a cloud provider instead, you can skip this step and use your existing cluster.

Install Knative Serving

Let's start by installing Knative Serving. We'll be using version 1.19.0, which has been tested with Traefik's Knative provider.

Install the Knative Custom Resource Definitions (CRDs):

kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.0/serving-crds.yaml

Then install the Knative Serving core components:

kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.0/serving-core.yaml

Wait for all Knative pods to be ready:

kubectl get pods -n knative-serving

You should see the activator, autoscaler, controller, and webhook pods all in a 'Running' state.

Now configure Knative to use Traefik as its networking layer:

kubectl patch configmap/config-network \
  -n knative-serving \
  --type merge \
  -p '{"data":{"ingress.class":"traefik.ingress.networking.knative.dev"}}'

Install Traefik as Knative Networking Controller

Now let's deploy Traefik with Knative support enabled using Helm.

Create a values.yaml file that enables the Knative provider:

experimental:
  knative: true

providers:
  knative:
    enabled: true

The configuration enables both the experimental Knative feature flag and activates the Knative provider.

Next, execute the following commands to deploy Traefik in the traefik namespace using the previously described configuration:

helm repo add traefik https://traefik.github.io/charts
helm repo update
kubectl create namespace traefik
helm upgrade --install --version 33.2.0 --namespace traefik \
  traefik traefik/traefik -f values.yaml

Verify that Traefik is running:

kubectl get pods -n traefik

You should see the Traefik pod in a 'Running' state. And voilà! Your Traefik instance is now ready to handle Knative workloads.

Deploy your first Knative Service

Now that our infrastructure is ready, let's deploy a serverless application. We'll use Traefik's whoami service, which provides useful debugging information about the request.

Create a file called whoami.yml:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          env:
            - name: WHOAMI_NAME
              value: "Serverless with Traefik"

Deploy the service:

kubectl apply -f whoami.yml

Watch as Knative creates the necessary resources:

kubectl get ksvc whoami

After a moment, you should see output indicating the service is ready:

Test Your Deployment

Test your application by sending a request with the appropriate Host header:

curl -H "Host: whoami.default.example.com" http://localhost

You should see output showing request details, including the service name and headers that Knative automatically adds.

Name: Serverless with Traefik
Hostname: whoami-00001-deployment-86654cdc46-xvkcc
IP: 127.0.0.1
RemoteAddr: 127.0.0.1:36324
GET / HTTP/1.1
Host: whoami.default.example.com
User-Agent: curl/8.7.1
Accept: */*
Accept-Encoding: gzip
Forwarded: for=10.42.0.1;host=whoami.default.example.com;proto=http, for=10.42.0.6
K-Proxy-Request: activator
X-Forwarded-Host: whoami.default.example.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-784b46c9ff-l8ckj

Congratulations! You've just deployed your first serverless application with Knative and Traefik.

Observe Scale-to-Zero

One of Knative's standout features is scale-to-zero. Watch this behavior in real-time by running the following command to watch the Knative pods scale up and down:

kubectl get pods -w

After about 30 seconds of no traffic, you'll see the application pod terminate. When you make another request, Knative will automatically spin up a new pod.

Go Beyond with Traffic Splitting

One of Knative's most powerful features is traffic splitting, which allows you to route a percentage of traffic to different revisions. This is perfect for canary deployments, A/B testing, or gradual rollouts.

Modify whoami.yml to include a traffic splitting configuration:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          ports:
            - containerPort: 80
          env:
            - name: WHOAMI_NAME
              value: "Version 2"
  traffic:
    - tag: v1
      revisionName: whoami-00001
      percent: 75
    - tag: v2
      revisionName: whoami-00002
      percent: 25

Apply the updated configuration:

kubectl apply -f whoami.yml

This configuration implements two routing strategies:

Percentage-based routing: 75% of traffic goes to revision 1, while 25% goes to revision 2
Tag-based routing: Each revision gets its own dedicated URL using tags

Test the percentage-based routing by making 10 requests and observe the distribution

for i in {1..10}; do
  curl -H "Host: whoami.default.example.com" http://localhost
done

You'll notice that approximately 75% of responses show "Serverless with Traefik" (v1) as the service name and 25% show "Version 2" (v2).

You can also access specific revisions directly:

# Access version 1 directly
curl -H "Host: v1-whoami.default.example.com" http://localhost

# Access version 2 directly
curl -H "Host: whoami.default.example.com" http://localhost

More Advanced Features

One of the most powerful aspects of Knative is its automatic revision management. Every time you update your service, Knative creates a new revision, and Traefik seamlessly updates its routing configuration.

Check your revisions:

kubectl get revisions

You'll see the following output:

Each revision represents a snapshot of your service at a particular point in time. Traefik automatically routes traffic according to the traffic splitting rules you define, handling all the complexity of load balancing and service discovery.

You can also fine-tune Traefik's Knative provider with additional options:

experimental:
  knative: true

providers:
  knative:
    enabled: true
    # Watch specific namespaces only
    namespaces:
      - default
      - production
    # Filter by labels
    labelselector: "environment=production"

See the Traefik Knative provider documentation for all available configuration options.

Conclusion

With Traefik 3.6's Knative provider support, we've achieved a significant milestone in proxy technology: a single solution capable of managing traditional workloads, cloud-native containers, and serverless functions. This unprecedented versatility eliminates the need for multiple specialized proxies, dramatically simplifying your infrastructure while maintaining the performance and features each workload type demands.

You now have a unified proxy solution that automatically handles service discovery, scaling events, and traffic routing across your entire application landscape. Whether you're running VMs, Kubernetes deployments, or Knative serverless workloads, Traefik provides consistent, enterprise-grade networking with advanced routing, middleware, and observability features.

Traefik is committed to the Knative ecosystem, and we're actively working on improving the integration based on community feedback. The Knative provider is currently experimental in Traefik 3.6, with plans to enhance features like additional middleware support and deeper integration with Knative's advanced routing capabilities.

Useful Links

]]>

Traefik Proxy 3.6 "Ramequin": Where Every Layer Counts

Emile Vauge — Mon, 10 Nov 2025 22:01:39 GMT

Traefik Proxy 3.6 has arrived, and like the exquisite Ramequin cheese 🧀—a rare specialty from France's Bugey region—this release achieves something unique. With native Knative integration completing the trilogy, Traefik becomes the only proxy capable of seamlessly handling Cloud Native workloads (containers), Traditional workloads (VMs), and Serverless workloads in one unified platform. Revolutionary multi-layer routing solves complex traffic decisions by enriching requests through hierarchical layers, eliminating the painful workarounds teams have endured for years. Combined with the fresh new Gateway API 1.4 support, this makes v3.6 a significant milestone in Traefik’s journey. Let’s taste it!

Unique Features That Set New Standards

Multi-Layer Routing: The End of Workarounds

The most anticipated feature of this release solves a fundamental challenge in modern cloud-native routing: making routing decisions based on information that doesn't exist in the original request. Whether you need to authenticate first, then route based on user roles, check feature flags to decide between microservices and monoliths, or route enterprise customers to stable infrastructure while directing free users to staging infrastructure, multi-layer routing transforms complex workarounds into elegant, declarative configuration.

Traditional routing approaches force teams into painful compromises: duplicating authentication logic across every microservice, building separate routing proxies that add latency and complexity, or maintaining giant configuration files that break when requirements change.

Multi-layer routing introduces hierarchical relationships between routers, where parent routers enrich requests with additional context, and child routers make routing decisions based on that enriched information.

The magic lies in progressive request enrichment. Each layer adds context that subsequent layers use for increasingly specific routing decisions. Authentication happens once at the parent level, while routing decisions are made at the child level based on the enriched result—no duplication, no external proxies, no compromises.

Here's a real-world example: routing API requests to different backends based on customer subscription tier:

http:
  routers:
    # Root router - matches all API requests and applies auth
    api-parent:
      rule: "Host(`api.example.com`) && PathPrefix(`/`)"
      middlewares:
        - auth-with-tier
      entryPoints:
        - websecure
      tls: {}
      # No service defined - this is a parent router
    
    # Leaf router - routes enterprise customers to stable
    api-enterprise:
      rule: "HeaderRegexp(`X-Customer-Tier`, `(enterprise|business)`)"
      service: stable-backend
      parentRefs:
        - api-parent
    
    # Leaf router - routes free tier to canary
    api-free:
      rule: "Header(`X-Customer-Tier`, `free`)"
      service: new-version
      parentRefs:
        - api-parent

  middlewares:
    auth-with-tier:
      forwardAuth:
        address: "http://auth-service:8080/validate"
        authResponseHeaders:
          - X-Customer-Tier
          - X-Customer-Id
          - X-User-Email

  services:
    stable-backend:
      loadBalancer:
        servers:
          - url: "http://api-v1-stable:8080"
    new-version:
      loadBalancer:
        servers:
          - url: "http://api-v2-canary:8080"

The hierarchy supports three router types: Root Routers attached to entryPoints that apply shared middleware and define TLS configuration, Intermediate Routers that can have children while inheriting configuration from their root, and Leaf Routers that must define a service and handle the final routing decision. Child routers cannot be called directly—requests must flow through their parent, ensuring you can't circumvent authentication or other parent-level logic.

This unlocks previously complex patterns like authentication-based routing (where admin and user requests to the same endpoint route to different backends based on role), feature flag-based migrations (routing users to microservices or monoliths based on feature flag evaluation), and sophisticated progressive rollouts where routing decisions depend on customer context, subscription tiers, or experimental configurations. Learn more in the documentation and in the dedicated blog post.

Knative Integration: The Universal Proxy for Any Workload

Traefik v3.6 introduces Knative provider support, achieving a key milestone: Traefik is now the only proxy that can seamlessly operate across Cloud Native workloads (containers), traditional workloads (Virtual Machines), and serverless workloads. This unprecedented versatility makes Traefik a truly unique platform capable of handling any kind of workload your infrastructure demands.

The Knative provider automatically handles service discovery, scaling events, and traffic routing for Knative workloads, providing a unified experience across your entire application landscape. Teams can now use a single proxy solution for their complete infrastructure, dramatically simplifying operations while maintaining the specialized optimizations each workload type requires.

Let’s enable the KNative provider in Traefik first:

experimental:
  knative: true
providers:
  knative:
    namespaces:
      - "serverless-apps"
      - "production"

The following Service manifest configures the running Traefik controller to handle the incoming traffic.

---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: traefik/whoami
          ports:
            - containerPort: 80
          env:
            - name: WHOAMI_NAME
              value: "Knative Test Service"

Once everything is deployed, sending a GET request to the HTTP endpoint should return the following response:

$ curl http://whoami.default.example.com

Name: Knative Test Service
...

Special recognition goes to idurgakalyan for spearheading this innovative integration that completes Traefik's transformation into the universal proxy for modern infrastructure (#11448). Learn more in the documentation and in the dedicated blog post.

Gateway API 1.4 Support: Leading the Standards Evolution

Traefik v3.6 embraces the future with full Gateway API v1.4 support, incorporating significant advancements that solidify Kubernetes networking standards. This release brings two major features from the Experimental to the Standard channel, making them production-ready for enterprise deployments.

BackendTLSPolicy graduates to Standard, enabling secure TLS configuration from Gateway to backend services with production-grade reliability. This critical security feature allows you to enforce end-to-end encryption, ensuring that traffic remains protected even between internal services.

SupportedFeatures status reporting also reaches Standard, providing clear visibility into which Gateway API features your implementation supports. This eliminates guesswork and enables better operational planning by exposing feature compatibility directly through Kubernetes status fields.

Additional Enhancements That Make a Difference

Health Checking Evolution

TCP Health Checks (#11238 by ddtmachado)—Native TCP health checking capabilities for non-HTTP services. Learn more.
Passive Health Checks (#11351 by Nelwhix)—Intelligent health monitoring that observes actual traffic patterns to determine service health. Here’s the documentation.

New Load Balancing Strategies

Least Time Strategy (#12167)—Route requests to servers with the lowest response times for optimal performance. Learn more in our configuration guide.
Highest Random Weight Algorithm (#9946 by mathieuHa)—Advanced probabilistic load balancing for enhanced traffic distribution. Check out Traefik MCP Gateway, which already leverages it to handle agent requests more efficiently.

Provider Enhancements

AWS ECS IPv6 Support (#12179 by wizbit)—Full IPv6 compatibility for Amazon ECS deployments. ECS provider docs.
Docker Non-Running Container Discovery (#10645 by acouvreur)—Enhanced container discovery capabilities for development workflows.

Kubernetes Integration

ExternalName Service Support (#12065 by james-callahan)—Publish Kubernetes ExternalName services through Traefik for external service integration.
Highest Random Weight in CRD (#12061)—Advanced load balancing options in Kubernetes Custom Resources.

Security and Performance

ACME Certificate Resolver Options (#11977 by ldez)—Enhanced certificate management capabilities with new resolver configurations. See our ACME documentation.
HTTP/2 HPACK Table Size Configuration (#12050 by GCHQDeveloper548)—Fine-tune HTTP/2 performance with configurable HPACK table sizes.

Developer and User Experience

Plugin Syscall Support (#11939 by david-garcia-garcia)—Extended plugin capabilities with syscall support for advanced integrations.
Dashboard Improvements (#12145 by leccelecce)—More compact table layouts for improved dashboard usability.

Observability

Provider Namespace Logging (#12002 by shreealt)—Enhanced visibility during startup for Consul, Consul Catalog, and Nomad providers.
Authentication Middleware Warnings (#12085 by kianelbo)—Improved security awareness with maxBodySize configuration warnings.

The Artisans Behind Every Layer

Like the careful aging process that creates Ramequin's distinctive character, Traefik v3.6 is the result of months of dedication from our incredible community. To every developer who opened an issue, every contributor who submitted a pull request, every user who provided feedback—you are the artisans behind Traefik's continued excellence. Your passion and expertise continue to drive innovation in cloud-native networking and contribute to routing requests to millions of applications and APIs worldwide.

Ready to taste the fresh new Traefik 3.6 "Ramequin"?

Essential Resources and Next Steps

Download Traefik 3.6: GitHub Releases | Docker Hub
Documentation: Complete Documentation | Migration Guide
Community: GitHub Repository | Community Forum
Learn More: Official Website | Blog

]]>

The AI Triple Security Gap: Why Your Gateway Strategy is Already Obsolete

Immánuel Fodor — Tue, 14 Oct 2025 15:02:02 GMT

MCP Changes Everything, and Most Security Teams Don't Know It Yet

The enterprise AI landscape just experienced a seismic shift. Anthropic's Model Context Protocol (MCP) has rapidly become the de facto standard for how AI agents interact with enterprise tools, data sources, and APIs. Within months of its release, MCP achieved what typically takes years: widespread adoption across major agent frameworks and AI development platforms, with thousands of MCP servers now deployed across the ecosystem.

But here's the problem: MCP introduces attack surfaces that traditional API security was never designed to handle.

While security teams focused on securing their APIs, a new threat vector emerged. Malicious instructions hide in prompts, legitimate tools transform into data exfiltration mechanisms overnight, and AI agents can be manipulated into bypassing every security control you've carefully implemented.

The uncomfortable truth: If your security strategy relies solely on API gateways, you're protecting the wrong layer.

Your Critical Data is Now Agent-Accessible

The MCP ecosystem is expanding at an unprecedented pace across enterprise infrastructure. Within months of release, hundreds of MCP servers have been deployed, with new implementations appearing weekly. Here's what's already agent-accessible through verified MCP server implementations:

Databases & Data Stores: Oracle DB23ai (official SQLcl MCP server), PostgreSQL, MySQL, MongoDB, SQLite, Redis, InfluxDB, CockroachDB, and other popular database systems

Data Warehouses & Analytics Platforms: Snowflake (official Cortex AI integration from Snowflake-Labs), BigQuery, Databricks, Apache Druid, and modern data platforms

Cloud & Developer Platforms: Salesforce (official DX MCP Server and community implementations), AWS services (CloudTrail, Bedrock, CDK), Supabase, Heroku, Firebase, and Cloudflare Workers

Collaboration & Communication: Google Drive, Slack, GitHub, Linear, Notion, Intercom, and productivity platforms

File Systems & Storage: Local filesystems with configurable access controls, cloud storage services, and document repositories

Development & DevOps Tools: Git repositories, CI/CD pipelines, container platforms, observability systems, and infrastructure-as-code tools

This isn't some distant future. It's happening right now. The MCP ecosystem is growing exponentially, with community and official implementations appearing across every category of enterprise software. Your perimeter security remains essential, but it's no longer sufficient once autonomous agents can chain multiple tools together across different systems before any single control point can see the complete attack.

The Critical Gap

Take database MCP servers as an example: they typically run with their own static database connection credentials, exposing tools that let AI assistants execute queries. The pattern is consistent across implementations. Authorization operates at the database connection level, and whatever the MCP server's authenticated user can do determines what the agent can do.

That's true, and it perfectly crystallizes the gap. Once connected, authorization collapses to the authenticated principal. There's no agent-aware control over specific tools, methods, or parameters before execution, nor any governance over the non-database tools the agent might chain together in the same session.

This pattern repeats across every MCP server type. Database MCP servers inherit database user permissions. Salesforce MCP servers inherit the user's CRM role and permissions. Cloud platform MCP servers inherit the authenticated user's access scope. File system MCP servers inherit filesystem permissions. GitHub MCP servers use the personal access token's full permissions. Slack MCP servers can access any channel the token allows.

The Attack Scenario: Cross-System Data Exfiltration

Consider this realistic attack: "Summarize sensitive customer data and share it externally"

Without proper controls, the attack unfolds across multiple systems. First, the Slack MCP Server reads confidential customer discussions from private channels. Next, the Salesforce MCP Server queries opportunity data with revenue projections. Then the Google Drive MCP Server accesses strategic planning documents. Finally, an HTTP tool posts the consolidated summary to an external webhook. Your competitive intelligence is gone.

Why traditional security fails here: The attack chain spans multiple systems (Slack, Salesforce, Google Drive, and external HTTP) that your individual system security controls can't govern holistically. Each individual access looks legitimate. The combination is malicious.

The Triple Gate Pattern: Defense in Depth for AI

The cloud-native community learned this lesson with containers: layered defense beats perimeter security. We didn't secure Kubernetes with a single firewall. We built defense in depth with network policies, pod security standards, RBAC, and admission controllers.

AI agent security demands the same architectural rigor.

Figure: Traefik's Unified Triple Gate Pattern

Each gate enforces policies appropriate to its layer. Unlike single-gateway approaches, attacks must defeat all three gates simultaneously to succeed, dramatically reducing your attack surface.

Gate 1: AI Gateway (Securing the Conversation)

The problem starts at the conversation layer. Attackers don't need to break your encryption or steal credentials anymore. They just need to convince your AI agent to do their bidding. Traditional security tools can't help here because they were built for APIs, not conversations with language models.

This is where Traefik fundamentally changes the game. Instead of relying on a single AI security model or vendor, Traefik orchestrates multiple specialized AI models into a unified policy enforcement pipeline. Think of it as having multiple security analysts, each trained to spot different attack patterns, all working in concert.

Topic control models detect when someone's trying to extract data or integrate with external systems. Content safety models identify privacy violations across more than 22 categories of sensitive information. And jailbreak detection models catch the sophisticated attempts to bypass your AI's restrictions, the digital equivalent of social engineering.

The key advantage? Unlike point solutions or cloud-based AI services that force you to send your data elsewhere, Traefik's entire security pipeline runs in your infrastructure. Deploy GPU-accelerated models like NVIDIA NIMs for maximum performance, integrate your custom classifiers trained on your specific threats, or combine multiple vendors' models. All without changing your architecture. No external API calls putting your data at risk. No vendor lock-in constraining your choices. Just one unified policy layer that adapts to your needs.

This gate blocks prompt injection attacks, jailbreak attempts, PII extraction requests, off-policy conversations, and inappropriate interactions before they ever reach your AI systems.

Gate 2: MCP Gateway (Governing Tool Access)

The MCP protocol presents a unique challenge. By design, it delegates all access control to implementers, creating what amounts to a protocol-level security vacuum. The specification is refreshingly honest about this, explicitly stating that "the core protocol lacks standardized permission or sandbox mechanisms."

This deliberate gap demands a new approach to authorization. Traditional access control models fail spectacularly when applied to AI agents.

Why Traditional Access Control Fails for AI Agents

Consider RBAC: agents don't have job titles or departments. When an agent inherits a "sales manager" role in Salesforce, it suddenly has access to all opportunities, all forecasts, and all customer data across the entire organization. That's absurd when all it needs to do is send follow-up emails to customers who haven't responded in 30 days.

ABAC seems better at first, since you can add context like time and location. But as you add more attributes, the complexity explodes exponentially. Try managing attribute combinations for hundreds of autonomous agents accessing dozens of different systems, each with their own attribute schemas. It quickly becomes an opaque mess that no one can audit or understand.

Delegation and impersonation make things even worse. When an agent impersonates a user across Salesforce, Google Drive, and Slack simultaneously, tracking who's responsible for what becomes nearly impossible. The audit trail

turns into fiction, and agents routinely gain access far beyond what any single task requires.

The fundamental issue is that these models were designed for human users with relatively stable, role-based permissions. They assume someone clocks in at 9 AM with a fixed set of responsibilities. AI agents need different access across different systems for different workflows, often changing by the minute.

How TBAC Works: Three Dimensions of Control

This is why we developed Task-Based Access Control (TBAC), a new authorization paradigm that focuses on the work being done rather than the identity doing it. TBAC creates a progressive permission funnel through three dimensions:

📋 Tasks (High-level business objectives): Tasks represent business-level goals and intent like customer_followup, generate_sales_report, or resolve_support_ticket. This is the coarsest filter and your first gate. Before considering which systems the agent can access, you validate that the agent is authorized to perform this type of business work at all. An agent without the financial_reporting task can't even begin to access financial systems, regardless of what specific tools it might need.
🔧 Tools (System and resource access): Tools control which specific MCP servers and their exposed resources the agent can access. Even if an agent is authorized for the customer_followup task, it doesn't get blanket access to all systems. Instead, it receives precisely scoped access to specific MCP servers (salesforce_crm, email_service) and their individual tools/methods (query_opportunities, send_email). Your customer follow-up agent might need Salesforce's opportunity queries and email capabilities, but has no business accessing the export_all_data tool or your google_drive server with financial documents.
💰 Transactions (Granular runtime controls): This finest-grained layer validates the specific parameters and context of each operation. It's not just "can this agent query Salesforce opportunities" but "can it query opportunities with these specific filters at this time from this location?" Transactions act as runtime guardrails, preventing dangerous operations even when the agent has legitimate access to the tool. They block queries without filters, updates to protected fields, or operations that could exfiltrate large datasets.

# An example JWT schema
{
  "sub": "<target agent>",
  "tasks": ["<list of tasks>"],
  "tools": {
    "<mcp server>": {
      "actions": ["<action names>"],
      "filters": { <additional transaction filters> }
    }
  }
}

These three dimensions work as a progressive permission funnel. First, validate the business objective (task). If authorized, validate which systems and specific tools can be accessed (tools). Finally, validate the specific operation parameters and context (transactions). Each layer further constrains what the agent can do, ensuring minimum necessary permissions across all enterprise systems.

The Innovation: Dynamic Variable Substitution

Here's where traditional policy systems break down completely. Want to give agent-123 access to Salesforce? You write a policy. Does it also need Google Drive? Another policy. Add agent-456 with different permissions? More policies. Scale this to hundreds of agents accessing dozens of systems, and you're drowning in thousands of policy rules that become impossible to maintain or audit.

Traefik flips this entire model through JWT claim and MCP parameter substitution: ${jwt.claim_name} and ${mcp.parameter_name}.

Now one policy works for every agent:

policies:
  # Task-level authorization
  - match: "Contains(`jwt.tasks`, `${mcp.params.task}`)"
    action: allow
  
  # Tool-level authorization
  - match: "Contains(`jwt.allowed_tools`, `${mcp.params.name}`)"
    action: allow
  
  # Transaction-level constraints
  - match: |
      Equals(`mcp.params.name`, `query_opportunities`) &&
      Contains(`jwt.tools.salesforce_crm.allowed_fields`,         `${mcp.params.field}`)
    action: allow

The same policy line works for every agent. Their specific permissions come from their JWT token issued by your existing identity provider or any OAuth compliant provider like Okta, Azure AD, Auth0, Keycloak, etc. No more policy explosion. No more maintenance nightmare.

Real-World Example: Customer Follow-Up Campaign

Let's see how this works in practice. A sales agent needs to query recent opportunities in Salesforce, draft follow-up emails based on templates in Google Drive, and log activities back to Salesforce. Nothing else.

The JWT token structure defines exactly what this agent can do:

{
  "sub": "agent:sales-followup-bot",
  "tasks": ["customer_followup"],
  "tools": {
    "salesforce_crm": {
      "actions": ["query_opportunities", "create_activity"],
      "filters": {"status": "Open", "days_since_contact": ">30"}
    },
    "google_drive": {
      "actions": ["read_file"],
      "allowed_folders": ["email_templates"]
    },
    "email_service": {
      "actions": ["send"],
      "allowed_domains": ["customer.com", "prospect.com"]
    }
  }
}

The JWT token acts like a security badge that precisely defines what the sales agent can do. The tasks field authorizes only customer follow-up work (not financial reporting or other business functions). The tools section grants access to specific systems and their methods: Salesforce access is limited to querying open opportunities inactive for 30+ days and logging activities. Google Drive access is restricted to reading only the email templates folder, not financial documents. Email capabilities are constrained to sending messages only to customer domains, not internal addresses. Each nested level further restricts permissions, creating an airtight scope for this specific workflow.

The result? The agent completes its workflow with precisely scoped permissions across all three systems. If compromised, it cannot access closed deals or sensitive forecast data in Salesforce. It cannot read financial planning documents in Google Drive. It cannot send emails to internal company addresses. And it cannot access Slack, databases, or any other MCP servers.

GitOps Governance Layer

Beyond runtime enforcement, every MCP server undergoes human validation before deployment. Tool definitions are stored in Git, requiring approval before reaching production. Change detection alerts trigger when tool behavior deviates from the approved baseline. And a complete audit trail tracks who approved what, when.

This blocks tool poisoning attacks (malicious instructions in descriptions), rug pull attacks (tools that transform after approval), tool shadowing (malicious tools with identical names), privilege escalation, cross-system data aggregation attacks, and configuration drift.

Gate 3: API Gateway (Protecting the Backend)

Even with Gates 1 and 2 in place, MCP servers eventually call your backend APIs, databases, cloud services, and enterprise systems. These underlying services need their own layer of protection.

Traefik's battle-tested API management capabilities provide this final defensive layer. Centralized credentials management automatically injects service credentials, rotates tokens, and manages authentication to backend APIs without exposing secrets to MCP servers. Fine-grained API authorization ensures agents can only invoke authorized endpoints. Distributed rate limiting protects backend services from overwhelming request volumes with intelligent throttling across your infrastructure.

API plans and subscriptions define consumption quotas and access tiers for different agent types, preventing resource exhaustion. Operation filtering exposes only specific API operations to agents, hiding sensitive or administrative endpoints. TLS/mTLS prevents man-in-the-middle attacks with encrypted transport and mutual authentication. Input validation sanitizes requests before they reach vulnerable servers. And DLP-style inspection detects sensitive data patterns in responses before they leave your infrastructure.

This comprehensive protection blocks network-level attacks, API abuse and resource exhaustion, unauthorized endpoint access, command injection attempts, SSRF attacks, credential theft and token leakage, and traditional API vulnerabilities.

How the Triple Gate Stops the Attack

Let's return to our cross-system exfiltration scenario and trace exactly how each gate responds to the threat: "Summarize sensitive customer data and share it externally."

The first line of defense activates at the AI Gateway. The topic control model immediately recognizes the dangerous pattern: data extraction combined with external sharing targeting sensitive information. Before the request even reaches your AI model, it's blocked with a 403 Forbidden response.

But sophisticated attackers don't give up that easily. They'll try to obfuscate their intent, perhaps breaking the request into seemingly innocent pieces. That's when the content safety model steps in, trained to identify privacy violations even when they're cleverly disguised. And if that fails? The jailbreak detection model catches the manipulation attempt, recognizing the linguistic patterns that indicate someone's trying to bypass restrictions.

Even if an attacker somehow slips past all three AI-level defenses (perhaps through a zero-day prompt injection technique), they hit the MCP Gateway's TBAC enforcement. Now they face a completely different challenge. The agent's token only authorizes customer_followup tasks, not data aggregation. Its tool allowlist explicitly excludes access to Google Drive's strategic documents. The Salesforce query filters prevent any access to sensitive forecast data. HTTP tool calls to external domains are blocked entirely. And cross-system data correlation? Impossible with the scoped permissions.

Should the impossible happen and the attack penetrates both previous layers, the API Gateway provides the final line of defense. Rate limiting algorithms detect the unusual data access patterns. Why is this agent suddenly reading hundreds of customer records? DLP inspection identifies sensitive data patterns in the aggregated responses, blocking them before they can leave your infrastructure. And egress controls ensure that even if everything else fails, that external webhook destination remains unreachable.

Five independent security layers, each operating on different principles, each capable of stopping the attack on its own. That's defense in depth.

Threat Coverage at a Glance

Attack Vector	Without Triple Gate	With Triple Gate
Prompt Injection	❌ Undetected	✅ Blocked (AI Gateway)
Tool Poisoning	❌ Executes	✅ Blocked (AI Gateway + GitOps)
Cross-System Data Exfiltration	❌ Succeeds	✅ Blocked (All Three Gates)
Rug Pull Attacks	❌ Transforms unnoticed	✅ Detected (Change Alerts)
Tool Shadowing	❌ Hijacks requests	✅ Prevented (Namespacing)
Privilege Escalation	❌ Full access	✅ Blocked (TBAC)
External Data Sharing	❌ Posts externally	✅ Blocked (MCP Gateway)
API Abuse & Resource Exhaustion	❌ Overwhelms backends	✅ Blocked (Rate Limiting + Quotas)
Unauthorized Endpoint Access	❌ Accesses admin APIs	✅ Blocked (Operation Filtering)
Credential Theft	❌ Exposes secrets	✅ Prevented (Upstream Auth)
MITM Attacks	❌ Intercepts traffic	✅ Blocked (TLS/mTLS)

Why One Unified Platform Matters

The Problem with Fragmented Tools

Deploying standalone AI security, MCP governance, and API gateway tools creates operational nightmares that compound over time.

Attack chains span multiple systems. When an attack moves from Salesforce to Google Drive to Slack to an external webhook, you're correlating logs from three different security products across different formats and timelines. By the time you piece together what happened, the damage is done.

Policy conflicts become inevitable. The AI gateway allows a request, the MCP gateway blocks it, but the API gateway allows it again. Who wins? How do you debug this? Which team owns the resolution?

Configuration drift accelerates with every product you add. Three products mean three configuration languages, three deployment pipelines, and three teams managing different policies. What starts as minor inconsistencies becomes major security gaps.

The blind spots between products become unmonitored attack surfaces. Each vendor assumes the others are handling certain threats. Attackers exploit these assumptions.

And the cost explosion is real: three vendors, three contracts, three support organizations, three renewal cycles, three sets of professional services. Your security budget triples while your actual security posture deteriorates.

The Traefik Advantage: One Platform. One Binary. Three Gates.

Traefik delivers protocol-level MCP protection that addresses the security gap inherent in MCP's design with native TBAC enforcement across all MCP server types. True AI sovereignty means NVIDIA NIMs run entirely in your infrastructure. No external API calls. No vendor lock-in.

The platform works with your existing infrastructure, integrating seamlessly with your OAuth-compliant IdP (Okta, Azure AD, Auth0, Keycloak, etc), deployment patterns (Kubernetes, Docker, VMs), and monitoring tools. Production performance is guaranteed through sub-millisecond authorization using stateless JWT validation. No database queries, no external calls.

Unified observability lets you correlate attack chains across databases, file systems, CRMs, and communication platforms in a single platform. And this isn't theoretical. Traefik is battle-tested at scale, trusted by millions of deployments worldwide, proven in production environments.

Of course, no security solution is perfect. Traefik cannot fix vulnerable server code or prevent command injection vulnerabilities in poorly written MCP server implementations, though it provides comprehensive logging and behavioral monitoring to mitigate risks. It cannot secure vulnerable clients or fix security flaws in client-side implementations, but enforces policies regardless of client behavior. It cannot prevent sandbox escapes or stop runtime containment breakouts in underlying infrastructure, though TBAC limits the blast radius. And it cannot replace secure coding practices. Developers must still write secure MCP servers following security best practices.

What this means is that Traefik provides comprehensive protection for protocol-level and infrastructure-level threats across your entire MCP ecosystem. This covers the vast majority of real-world attack vectors. The remaining gaps require secure development practices and infrastructure hardening that no gateway can replace.

The Bottom Line

MCP adoption is accelerating faster than security teams can respond. Developers are deploying AI agents with access to Salesforce, Google Drive, Slack, databases, and dozens of other critical systems, often without security review.

The question isn't whether your organization will adopt MCP-based agents. The question is whether you'll secure them before or after a breach.

The cloud-native community learned security lessons through painful incidents. Apply those lessons to AI agents before your organization becomes the cautionary tale.

Next Steps

Traefik Labs has pioneered the Triple Gate Pattern, the industry's most unified and declarative runtime platform that delivers enterprise-grade AI security without operational fragmentation.

Ready to Secure Your Agentic Enterprise?

Download the Complete Implementation Guide - Detailed technical documentation with code examples, configuration samples, deployment patterns, and testing procedures
Schedule a Technical Briefing - See the Triple Gate Pattern in action with your MCP architecture and threat scenarios
Request a Security Assessment - We'll analyze your MCP exposure and map TBAC policies to your agent workflows

Secure your agentic enterprise with Traefik's Triple Gate Pattern—available today, proven in production, ready to scale.

]]>

Nutanix + Traefik Labs: Modern Application Intelligence for Distributed Hybrid Infrastructure

Sudeep Goswami — Tue, 30 Sep 2025 15:33:29 GMT

Enterprise Infrastructure Has Evolved, Application Delivery Must Follow

Recent analyst reports from Gartner and Forrester reveal a fundamental shift in enterprise infrastructure: container orchestration has matured, multicloud adoption is accelerating, and distributed hybrid infrastructure (DHI) is exploding from 15% to 55% of enterprises by 2028. Nutanix has led this transformation and been named a Leader in Gartner's Magic Quadrant for Distributed Hybrid Infrastructure (DHI), delivering unified infrastructure that works consistently across any environment.

Yet even as Nutanix customers achieve infrastructure excellence, they face a broader industry challenge: fragmented Layer 7 application delivery capabilities. While Nutanix Flow Virtual Networking (FVN) provides robust L2-L4 capabilities and network security, enterprises also require advanced L7 services. These include identity-aware routing and load balancing, API gateways, WAF protection, and AI governance. Traditionally, these capabilities required separate, expensive point solutions from multiple vendors.

The VMware licensing disruption has accelerated migration to Nutanix AHV, and these customers bring established application delivery requirements. Rather than force customers to assemble disparate tools, Nutanix partnered with Traefik Labs to deliver a validated, integrated solution that extends Nutanix's infrastructure excellence into comprehensive application intelligence. This partnership reflects Nutanix's strategic approach: provide best-of-breed capabilities through validated integrations rather than forcing customers into monolithic stacks.

The hybrid reality persists: virtual machines and containers will coexist for years, AI workloads demand enterprise sovereignty and governance, and edge deployments require consistent policies without constant connectivity. Organizations need unified application delivery that spans compute substrates, eliminates tool sprawl, and enables safe modernization. All of this must be delivered through the operational simplicity that Nutanix customers expect.

Modern Application Intelligence Layer: Extending Nutanix Infrastructure Excellence

The Application Intelligence Layer (AIL) transforms fragmented application delivery into a unified, portable control fabric that works consistently across any infrastructure. Unlike point solutions that solve individual problems, AIL provides comprehensive application-aware capabilities that travel with workloads regardless of where they run or how they're packaged.

Through native integration with Nutanix AHV and NKP, Traefik Labs extends infrastructure capabilities into modern intelligent application delivery. The cloud-native platform combines identity-aware routing, comprehensive security, policy-driven governance, end-to-end observability, and AI runtime policy management—while leveraging the power of familiar Nutanix constructs like Prism Central, Flow networking, and category-based policies.

This Nutanix Ready validated integration eliminates the complexity of managing separate tools for different compute substrates while providing the enterprise-grade Layer 7 services that modern applications demand. Organizations can safely modernize legacy VM-based applications alongside new containerized services, leveraging Nutanix's unified management approach with Traefik providing seamless application-layer intelligence on top.

The result is infrastructure that doesn't just run applications. It makes them intelligent, resilient, and governed across every environment, from edge deployments to sovereign clouds, all while maintaining the operational simplicity that defines the Nutanix experience.

Business Value

For organizations migrating from VMware to Nutanix, the Traefik integration completes the infrastructure story. Rather than assembling separate tools for application delivery, customers gain enterprise-grade Layer 7 services as a natural extension of Nutanix's unified platform approach. This eliminates the tool proliferation that VMware customers historically managed through separate products and vendors.

The integration strengthens Nutanix's value proposition by providing:

Complete stack consistency: Customers manage application delivery through familiar Nutanix constructs (Prism Central, Flow networking, categories)
Faster time-to-value: Pre-validated Nutanix Ready integration reduces deployment risk and accelerates production readiness
Unified operations: Same infrastructure-as-code approach (Terraform, Ansible) across compute and application layers
Future-proof architecture: Built-in AI governance capabilities position customers for emerging requirements

Engineering productivity increases dramatically as teams leverage Nutanix's unified management while gaining advanced application services. Rather than managing separate tools from different vendors, organizations extend their existing Nutanix investment to cover comprehensive application delivery requirements. AI sovereignty becomes achievable through complete governance pipelines deployed entirely within Nutanix infrastructure. This includes content safety, agent control, and more, without external dependencies that compromise data security or regulatory compliance.

Key Use Cases

The integration addresses five critical scenarios where Nutanix infrastructure excellence extends naturally into application delivery:

Use Case	Solution	Benefits
VMware Migration	Nutanix Ready validated AHV integration with enterprise Layer 7 services	Complete infrastructure and application delivery story. Validated, supported solution reduces deployment risk. Faster migration with proven integration patterns.
Hybrid Modernization	Unified policies across VM and container workloads on Nutanix	Safe incremental modernization without business disruption. Consistent security across all Nutanix compute substrates. Bridge legacy and modern apps seamlessly.
Tool Consolidation	Single integrated platform for WAF, load balancing, API gateway, AI gateway, observability	Eliminate expensive vendor sprawl and licensing costs. Reduce operational complexity and training overhead. Extend Nutanix simplicity to application layer.
AI Sovereignty	Safe and Responsible AI governance pipelines with NVIDIA NIMs	Deploy AI safely in regulated industries and air-gapped environments. Maintain complete data sovereignty on Nutanix platforms. Enterprise-controlled AI without external cloud dependencies.
Deploy Anywhere	Consistent operation across all Nutanix deployment models	Works across edge, Nutanix Cloud Clusters, private cloud, air-gapped environments. Complete operational independence from external systems. Compliance-ready evidence for regulatory audits.

VMware Migrations

Organizations moving to Nutanix AHV gain not just infrastructure replacement, but infrastructure advancement. The Nutanix Ready validated integration with Traefik delivers enterprise Layer 7 services that integrate seamlessly with Flow Virtual Networking. This provides capabilities that match or exceed what VMware customers assembled from multiple vendors, but with Nutanix operational simplicity.

Nutanix Customer Modernization

Nutanix has always excelled at hybrid infrastructure. This integration extends that excellence into application delivery, providing unified policies across AHV virtual machines and NKP containers. Organizations modernize safely using the same Nutanix management constructs they trust: Prism Central, Flow networking, and category-based policy. They gain sophisticated application-layer intelligence in the process.

Strategic Infrastructure

Nutanix customers who standardized on Nutanix for its operational simplicity and vendor-neutral approach now have a matching application delivery solution. Nutanix infrastructure excellence combines with best-of-breed application intelligence, without vendor lock-in or operational complexity.

Regulated Industries

Nutanix's strength in sovereign cloud deployments extends to application delivery. The Traefik integration operates entirely within customer-controlled Nutanix infrastructure. This supports the air-gapped and compliance-focused deployments where Nutanix excels: defense, healthcare, financial services, and government organizations that cannot accept external dependencies.

Global Operations

Nutanix infrastructure spans edge to cloud. This includes retail locations, data centers, and Nutanix Cloud Clusters on AWS and Azure. The Traefik integration extends consistently across this entire footprint, providing the same application intelligence whether workloads run in the core data center, at remote edge sites, or in public cloud extensions of Nutanix infrastructure.

Technical Solution

Technical Architecture and Implementation

The Traefik-Nutanix integration delivers a comprehensive Application Intelligence Layer across both Nutanix AHV virtual machines and Nutanix Kubernetes Platform (NKP), creating a unified control plane that extends Nutanix's infrastructure management philosophy to the application layer.

High-Level Architecture: Modern Application Intelligence for AHV and NKP

Native Integration with Nutanix Infrastructure

Flow Virtual Networking (FVN) Integration: The platform provides proven integration across Flow Virtual Networking for both Tenant and Transit VPCs. This enables sophisticated multi-tenant architectures that leverage Nutanix's native networking capabilities. Traefik integrates directly with Prism Central using v4.1 APIs for automatic VM discovery and category-based grouping. This enables zero-touch service provisioning with configurable polling intervals as low as 2 seconds. The lightweight integration extends Nutanix's policy-driven management approach to application delivery.

The auto-discovery mechanism leverages Nutanix's native category system. Infrastructure teams define application groupings using familiar Prism constructs. As VMs are created, modified, or destroyed within Nutanix environments, Traefik automatically updates its routing configuration without manual intervention. This maintains the operational simplicity that Nutanix customers expect while adding sophisticated application-layer capabilities.

High Availability Architecture: Enterprise resilience is achieved through Nutanix NLB VIP load balancing across multiple Traefik instances. In validated tenant VPC configurations, two Traefik instances provide redundancy. Transit VPC deployments typically run three instances for higher availability requirements. This distributed architecture leverages Nutanix's proven HA mechanisms to ensure that application delivery continues uninterrupted even during maintenance windows or infrastructure failures.

Unified Application Delivery Capabilities

Identity-Aware Routing and Security: The platform leverages OIDC, mTLS, and JWT processing to enforce authentication and authorization policies that span VM and container boundaries across Nutanix infrastructure. Organizations can define routing rules based on user groups or organizational identity. For example, administrators can be directed to different application endpoints than developers, or region-specific content can be served based on authenticated attributes. All of this leverages Nutanix's security constructs.

Built-in WAF protection blocks common attacks like SQL injection and cross-site scripting using Traefik's high-performance Coraza engine. Policies apply uniformly whether traffic targets a legacy VM-based application on AHV or a modern containerized microservice on NKP.

Intelligent Traffic Management: The integration provides circuit-breaking, retry logic, and canary deployment capabilities for resilient application delivery across Nutanix environments. Distributed rate limiting operates across all Traefik instances, preventing any single instance from becoming a policy enforcement bottleneck. Traffic shaping policies can be defined once and enforced consistently across the entire Nutanix hybrid infrastructure. This is critical for organizations managing thousands of application endpoints across distributed deployments.

Enterprise Networking Patterns on Nutanix

Multi-Tenant VPC Architectures: The validated reference architecture demonstrates a Transit VPC pattern with multiple Tenant VPCs, each containing isolated application workloads, and leveraging Flow Virtual Networking's advanced capabilities. Traefik instances in the Transit VPC provide a centralized entry point for external traffic. They route requests to the appropriate Tenant VPC based on path, host, or identity attributes. This architecture supports:

Path-based routing across VPC boundaries leveraging Flow networking
Unified L7 load balancing across application instances in multiple VPCs
Consistent security policies integrated with Nutanix security constructs
Centralized observability through Prism while maintaining tenant isolation

Infrastructure-as-Code Automation: Full deployment and management automation is achieved through Ansible and Terraform integrations. These are the same tools Nutanix customers use for infrastructure provisioning. Infrastructure teams can also define Traefik configurations, routing policies, and security rules as code. This enables repeatable deployments across development, staging, and production environments. GitOps workflows support version control and rollback capabilities, treating infrastructure configuration with the same rigor as application code. This is consistent with Nutanix's infrastructure-as-code philosophy.

Sovereign Cloud and Edge Deployment

Zero-Dependency Architecture: The platform supports both connected and air-gapped deployments through signed policy bundles that enable sovereign cloud operations. This is critical for the defense, healthcare, financial services, and government organizations where Nutanix excels. Traefik operates entirely within Nutanix infrastructure. All routing logic, security enforcement, and policy decisions execute locally without requiring connectivity to external control planes or policy servers. This maintains the data sovereignty and operational independence that Nutanix customers require.

Comprehensive Observability: OpenTelemetry integration provides distributed tracing, metrics, and audit-grade logging that feeds existing SIEM systems. Unlike proprietary observability solutions that create vendor lock-in, Traefik's standards-based approach aligns with Nutanix's vendor-neutral philosophy. Organizations can use their existing monitoring investments. Traces span VM and container boundaries across Nutanix infrastructure, providing end-to-end visibility into request flows regardless of where application components execute.

AI Workload Governance on Nutanix

NVIDIA Safety NIMs Integration: For AI workloads running on Nutanix infrastructure, the integration brings complete sovereignty through NVIDIA Safety NIMs integration. This enables enterprise-controlled AI deployment without external cloud dependencies. Content safety, agent control, and governance pipelines operate entirely within Nutanix private infrastructure. This makes AI adoption viable for organizations with strict data sovereignty requirements who have chosen Nutanix for exactly this reason.

The Traefik AI Gateway capability supports model routing governance. Organizations can direct inference requests to appropriate models based on request characteristics, user identity, or compliance requirements. This becomes critical as enterprises deploy multiple large language models on Nutanix infrastructure with different cost, performance, and privacy characteristics.

Performance and Scale Characteristics

The lightweight Traefik binary delivers enterprise-grade performance without compromising the efficiency that makes Nutanix infrastructure attractive. Key characteristics include:

Sub-millisecond routing decisions for typical HTTP requests
Horizontal scalability across dozens of instances without coordination overhead
Minimal memory footprint enabling deployment even on resource-constrained edge nodes running Nutanix
Near-zero configuration drift through automated reconciliation loops integrated into CI/CD pipelines

This efficiency profile enables deployment patterns across the full range of Nutanix infrastructure. Application delivery logic can run on edge devices or in highly multi-tenant Nutanix environments, all without sacrificing the operational simplicity that defines the Nutanix experience.

Infrastructure and Application Intelligence, Unified

Nutanix transformed infrastructure by eliminating the complexity of managing separate compute, storage, and virtualization layers. The Traefik integration brings the same philosophy to application delivery: unified management, consistent operations, and freedom from vendor lock-in.

Organizations choosing Nutanix gain:

Infrastructure excellence recognized by Gartner as market-leading in Distributed Hybrid Infrastructure
Application intelligence that extends naturally from that foundation through this validated integration with Traefik
Operational consistency across the entire stack, from infrastructure to application delivery
Strategic flexibility to modernize at their own pace without disruption
Future readiness for AI, edge, and emerging workload requirements

The Traefik-Nutanix integration strengthens what Nutanix customers value most: operational simplicity without sacrificing capability, vendor-neutral architecture without sacrificing integration quality, and the freedom to deploy anywhere without sacrificing functionality. It extends Nutanix's infrastructure leadership into intelligent application delivery, transforming infrastructure from a platform that runs applications into one that makes them intelligent, resilient, and governed across every environment.

Ready to evolve from "infrastructure anywhere" to "intelligent applications everywhere"?

Request your pilot and start building your application intelligence foundation.

Learn more about the Traefik-Nutanix integration at traefik.io

]]>

The Application Intelligence Layer: Beyond Infrastructure to Intelligent Applications Everywhere

Sudeep Goswami — Tue, 30 Sep 2025 14:57:18 GMT

Three Landmark Reports Paint a Clear Picture of Infrastructure's Future

In recent months, three pivotal analyst reports have defined the trajectory of enterprise infrastructure:

Gartner's Magic Quadrant for Container Management (2025) reveals that container platforms are reaching maturity, but application-layer capabilities remain fragmented
Forrester's Wave™: Multicloud Container Platforms, Q3 2025 demonstrates that multicloud complexity demands unified governance and policy management
Gartner's Distributed Hybrid Infrastructure (DHI), 2025 projects that 55% of enterprises will initiate DHI proofs of concept by 2028, driven by VMware disruption and sovereignty requirements

Together, these reports establish an undeniable consensus: the future is hybrid, multicloud, distributed, and demands application intelligence.

The catalysts driving this transformation are seismic. VMware licensing changes have created urgent migration pressures across enterprises. Sovereign cloud mandates are rewriting compliance playbooks. AI inferencing is pushing workloads closer to users and data. And the enduring reality highlighted across all three reports is that VMs and containers will coexist for years to come.

Infrastructure & Operations (I&O) leaders have already proven their capability—mastering Kubernetes adoption, driving modernization initiatives, and maintaining uptime in increasingly complex environments.

But as these reports make clear, the next critical challenge is emerging: How do we ensure that all this distributed infrastructure actually delivers resilient, secure, and intelligent applications everywhere?

What the Analysts Reveal: The Infrastructure Revolution Is Here

Reading these three reports together reveals a fundamental shift happening right now in enterprise infrastructure:

Container Orchestration Has Matured, But Application Delivery Hasn't

Gartner's Container Management Magic Quadrant shows that while Kubernetes and container platforms have achieved enterprise readiness, Layer 7 capabilities remain fragmented across vendors. Organizations are left stitching together ingress controllers, API gateways, and security tools.

Multicloud Success Requires Unified Application Governance

Forrester's multicloud analysis proves that the winners achieve policy consistency without vendor lock-in. But most organizations struggle with different authentication systems, inconsistent security policies, and fragmented observability across AWS, Azure, GCP, and sovereign clouds.

Distributed Hybrid Infrastructure (DHI) Adoption Is Exploding Due to VMware Disruption

Gartner's DHI research shows adoption expected to jump from 15% to 55% of enterprises by 2028—a once-in-a-decade transformation window. Organizations need cloud-native capabilities everywhere, but existing solutions force them to choose between infrastructure flexibility and application intelligence.

Edge and Sovereignty Are Strategic Imperatives

All three reports emphasize that latency-sensitive workloads, disconnected operations, and national sovereignty mandates are pushing infrastructure closer to users and under stricter jurisdictional control. Cloud-native principles must work without constant cloud connectivity and with complete data sovereignty.

The Hybrid Reality Demands Bridge-Building

All three reports acknowledge that virtual machines and containers will coexist for years. Yet most application delivery solutions work for one compute substrate or the other—not both seamlessly.

AI Workloads Need Immediate Governance

While not explicitly covered in these infrastructure reports, the elephant in the room is clear: AI inferencing is scaling faster than governance. Organizations deploying AI models across distributed infrastructure need consistent quotas, cost controls, and safety guardrails—not another complex tool to manage.

This convergence of analysis points to an undeniable conclusion: infrastructure has evolved, but application delivery is the bottleneck

The Solution: Modern Application Intelligence as a Unified Layer

The Application Intelligence Layer (AIL) is the missing infrastructure component that transforms fragmented application delivery into a unified, portable control fabric. Unlike point solutions that solve individual problems, modern AIL provides comprehensive application-aware capabilities that work consistently across any infrastructure.

This isn't just another tool—it's the connective tissue that transforms infrastructure investments into intelligent application delivery, regardless of where workloads run or how they're packaged.

For I&O leaders, AIL represents the natural evolution of infrastructure mastery, extending hard-won orchestration capabilities into the application layer where competitive advantage is ultimately determined.

The Five Pillars of Modern Application Intelligence

5 Key Pillars of Modern Application Intelligence

Traefik Labs has defined the essential capabilities that comprise a complete Application Intelligence Layer:

API-First & AI Readiness: moves beyond basic load balancing to offer comprehensive API management, API Gateway, and AI Gateway capabilities like model routing, semantic caching, and safety guardrails.
Security & Traffic Intelligence: delivers identity-aware routing (OIDC, mTLS), built-in WAF protection, intelligent traffic management, and end-to-end observability through OpenTelemetry.
Hybrid Orchestration: provides unified policy management across both virtual machines and containers, with auto-discovery and support for gradual modernization patterns.
Sovereign Cloud Enablement: ensures complete operational independence through zero external dependencies, offline policy bundles, and comprehensive audit-grade logging.
Edge-to-Cloud Continuity: guarantees consistent behavior across distributed deployments with local gateway enforcement during connectivity failures and central policy management.

Application Intelligence in Action: Real-World Impact

Financial Services: Safe Hybrid Modernization

A global bank routes /v1/payments to VM-based core banking systems while /v2/payments serves containerized microservices—all with unified OIDC authentication and WAF protection. This enables safe, incremental modernization without the downtime risks that could cost millions per hour.

Healthcare: Sovereign PHI Protection

Hospitals manage patient data in air-gapped environments using offline policy bundles and signed attestation. Exportable audit evidence streamlines HIPAA compliance while maintaining complete operational independence from external systems.

Government: Multi-Jurisdiction Services

Agencies deliver citizen services with per-jurisdiction policy packs and complete data sovereignty. Zero external dependencies ensure compliance while tenant isolation maintains security across different government departments.

Manufacturing: Resilient Edge Operations

Factories rely on local Traefik gateways to enforce policies during WAN outages. Intelligent back-pressure prevents equipment failures while maintaining operational continuity during network disruptions.

AI Teams: Governed Model Deployment

Enterprises route inference traffic based on cost, latency, and data sensitivity. Semantic caching reduces redundant API calls by 25%, usage quotas control GPU spending, and safety filters ensure responsible AI deployment across all endpoints.

Each scenario demonstrates the same fundamental truth: application intelligence is what transforms infrastructure investments into competitive advantage.

The Business Case: Measurable ROI Through Risk Reduction

Application Intelligence Layer implementation delivers immediate, measurable returns through risk mitigation and operational efficiency:

Downtime Prevention: Even conservative estimates place enterprise downtime costs at $300K per hour. Preventing just a few hours of outages annually pays for AIL implementation many times over.

Compliance Assurance: Regulatory fines under GDPR, HIPAA, and other frameworks routinely exceed millions of dollars. Audit-grade evidence packs and consistent policy enforcement significantly reduce compliance risk.

AI Cost Control: GPU and LLM costs are escalating rapidly. Semantic caching and intelligent quota management can reduce AI infrastructure spending by 15-25%, often saving six figures annually.

Engineering Productivity: Teams spend 20-30% of their time managing configuration drift, static IP assignments, and firewall exceptions. Unified policy management reclaims this time for innovation.

Vendor Consolidation: By unifying WAF, ingress, API gateway, and observability functions, organizations reduce vendor sprawl and eliminate redundant licensing costs.

For I&O leaders, this isn't speculative ROI—it's embedded in avoided costs and operational efficiencies that every CFO immediately understands.

The Transformation Window Is Closing Fast

The convergence of analyst insights and market forces creates an urgent, limited-time opportunity:

VMware Migration Momentum: Gartner's DHI research directly links VMware disruption to accelerated infrastructure transformation. Organizations are already re-platforming—this strategic migration window won't reopen for years. Companies that integrate application intelligence now will avoid costly retrofitting later.

AI Governance Crisis Looming: While container and DHI platforms focus on infrastructure, AI workloads are scaling into production without proper governance. GPU costs are exploding, compliance risks are mounting, and the organizations that implement AI guardrails now will avoid the financial and regulatory crises facing late adopters.

Multicloud Complexity Compounding: Forrester's analysis shows that multicloud complexity increases exponentially without unified governance. Every month of delay makes consistent policy management exponentially harder to achieve.

The future-ready organizations identified across all three reports share one characteristic: they don't just migrate or orchestrate infrastructure—they transform it with application-aware intelligence from day one.

The Traefik Advantage: Completing the Stack

Traefik Labs uniquely positions enterprises for this transformation:

Proven Integration: Native embedding in Rancher K3s, Nutanix NKP, and Nutanix AHV provides validated deployment paths
Open Source Foundation: Millions of developers trust Traefik's core technology, with clear upgrade paths to enterprise governance features
Platform Agnostic: Traefik enhances rather than replaces DHI and container platforms, extending their orchestration capabilities with application-layer intelligence

Traefik don't compete with your infrastructure investments—it completes them.

Act Now: Your Application Intelligence Foundation

The analyst consensus is clear, the market forces are aligned, and the technology is proven. The question isn't whether to add application intelligence to your infrastructure strategy—it's how quickly you can implement it to transform current investments into lasting competitive advantages.

Start with immediate validation. Choose the approach that fits your most pressing need:

5-Day Infrastructure Bridge Pilot: Deploy unified WAF protection, identity-aware routing, and observability across both VM and container workloads. Prove policy consistency and validate rollback procedures in a controlled environment.

AI Governance Quick Win: Add usage quotas, safety guardrails, and cost tracking to existing AI inference endpoints. Demonstrate immediate ROI while establishing governance patterns for enterprise scale.

Multicloud Policy Unification: Implement consistent authentication and security policies across two cloud environments. Validate the portable governance model that Forrester identifies as critical for multicloud success.

Sovereign Cloud Readiness Test: Deploy signed policy bundles in a disconnected environment and generate audit evidence packs. Prove operational independence capabilities for compliance requirements.

The transformation window identified by Gartner and Forrester is closing. Organizations that act decisively will establish application intelligence capabilities that become increasingly difficult for competitors to replicate.

Ready to evolve from "infrastructure anywhere" to "intelligent applications everywhere"?

Request your pilot and start build your application intelligence foundation now.

]]>

Aged to Perfection: Traefik's 10-Year Journey from Zero to Standard

Emile Vauge — Mon, 22 Sep 2025 13:42:32 GMT

10 years ago on September 22nd, 2015, after months of research and hacking, I posted a link on Hacker News about this small project I'd been working on called Traefik. Honestly, I was just hoping a few fellow developers might find it useful for their own container routing headaches.

Well, things escalated quickly…

Early Days: The Microservices Wild West

Let's be honest—2015 was a bit of a mess for anyone trying to manage microservices. Docker containers were getting some hype, Kubernetes was still this intimidating thing that required a PhD to set up, and everyone was manually editing NGINX configs like it was 2005.

The problem was simple: traditional load balancers didn't understand the dynamic infrastructure of modern platforms. Your services were constantly coming and going, scaling up and down, and your poor reverse proxy was sitting there completely oblivious to what was happening behind it.

My idea was equally simple: what if the reverse proxy could just... figure it out automatically? What if it could watch your containers, understand your services, and configure itself? Or to put it differently, what if we could delegate the routing configuration to the application level. In a world where there are now hundreds or thousands of services, each should come with its own ingress properties attached to it.

The response surprised me, to put it mildly. Within a week of the public announcement, Traefik was in the top trending projects on GitHub, and I found myself frantically responding to issues and pull requests from developers around the world.

Looking back, I have to admit I had no idea this would become what it is today. I set out to solve a real life problem I was experiencing, maybe thinking that a few other microservice nerds might be facing the same issue. The idea that Traefik would eventually become such a critical piece of today’s platforms? That seemed pretty unlikely at the time.

But here we are, 10 years later, with some numbers that still make me do a double-take:

3.4 billion downloads on Docker Hub
56,000+ GitHub stars
5,000+ merged pull requests
6,000+ issues (we've accidentally created the world's most persistent TODO list)
Nearly 900 contributors from around the world
500+ releases (we've been... productive)
26 cheeses for codenames (in fact the most important metric)
16 maintainers
1 company founded

The Maturing Process

Over the past decade, Traefik has evolved through fundamental milestones that each marked a critical innovation in the cloud-native revolution. Every major release wasn't just an upgrade—it was a response to where the industry was heading, often arriving just as developers realized they needed exactly those capabilities.

Traefik v1 started with the fundamentals that felt revolutionary at the time:

Automatic service discovery (no more manual config updates)
Let's Encrypt integration (because HTTPS should be easy)
Support for Docker, Kubernetes, Marathon, and other orchestrators
Live configuration reloading (remember when you had to restart everything?)

Traefik v2 was a major effort at future proofing the architecture:

Complete redesign with routers, middlewares, and services
TCP and UDP support (not just HTTP)
Proper Kubernetes CRDs
Middleware chaining for complex traffic handling

We'll be honest—the v1 to v2 migration wasn't our smoothest moment. We learned a lot about backward compatibility the hard way.

Traefik v3 focused on modern standards and smoother upgrades:

Gateway API support (the future of Kubernetes ingress)
OpenTelemetry integration (the future of observability)
Much gentler migration experience (we learned our lesson)

Looking back, it's clear that the cloud-native industry itself underwent a fundamental shift during these 10 years. What started as a pure innovation and creativity stage—where everyone was experimenting with containers, orchestrators, and new architectural patterns—has matured into a productivity era. The wild experimentation phase is largely over; now it's about making these technologies reliable, secure, and easy to operate at scale.

Traefik's evolution mirrors this industry transformation, moving from solving bleeding-edge problems to providing stable, production-ready infrastructure that teams can actually depend on. Traefik has consistently helped define what cloud-native infrastructure should look like. We didn't just adapt to the productivity era; we helped create the tools and patterns that made it possible.

What's Next

After a decade of helping shape the cloud-native revolution, we're not slowing down. The industry has moved into its productivity era, but that doesn't mean the innovation stops—it just becomes more focused on solving the real operational challenges that platform teams face every day.

The community has been pretty clear about where they want us to go next, and honestly, the next steps feel like natural evolutions of what we've been building all along.

v3.5: The NGINX Compatibility Layer

Let's talk about the elephant in the Kubernetes room: ingress-nginx is in maintenance mode. No new features, only critical fixes, and a growing list of security concerns that keep platform teams up at night.

This isn't just another deprecation announcement—it's a fundamental shift that will impact every team running Kubernetes in production. The controller that powers many clusters is entering its endgame, and platform teams everywhere are asking the same question: "What now?"

We saw this coming and decided to do something practical about it. Traefik 3.5 includes a brand new NGINX provider that understands NGINX Ingress annotations. That means you can migrate from ingress-nginx to Traefik without rewriting a single manifest.

No massive refactoring projects. No weekend migrations that keep everyone on edge.

You keep your existing annotations, your existing workflows, and your existing sleep schedule. You just get a more actively maintained ingress controller that's designed for the cloud-native world we're actually living in. Bonus: you can also plan a later Gateway API migration using the same controller thanks to Traefik’s flexibility.

We'd love contributions from the community to make it even better. If you're migrating from NGINX Ingress and hit edge cases we haven't covered, or if you have ideas for improving the compatibility layer, we're all ears.

The migration path is straightforward, the security posture is solid, and the future roadmap is clear. Sometimes the best innovation is just making the obvious choice easier to make.

v3.6: More Power, More Features

Version 3.6 is shaping up to address some complex routing scenarios that our community has been requesting.

Multi-layer routing is probably the most significant architectural addition. Right now, Traefik follows a simple path: entrypoint → router → service → server. Version 3.6 will let routers forward requests to other routers instead of just services, creating routing trees for complex use cases.

Think about it this way: you could have a first layer of routers dedicated entirely to authentication and authorization, then forward authenticated requests to a second layer that routes based on the user's permissions or role. No more cramming everything into a single routing decision, or weird loopback hacks—you can build sophisticated traffic management that actually matches how your applications work.

KNative integration (thanks to our community contributors) will enable Traefik to be a first-class citizen in serverless Kubernetes environments, making it easier to route traffic to both traditional microservices and serverless functions from a single ingress controller.

v4: Perfectly Aged

We're taking a different approach with Traefik v4. Instead of the traditional "big bang" major release with tons of new features and breaking changes, we're trying something smarter.

All the new features destined for v4 will actually land in v3.x minor releases first. NGINX support, multi-layer routing, KNative, enhanced Gateway API support—you'll get to use them while they're fresh, not after waiting years for a major release.

At the same time, we'll be deprecating legacy features well in advance, giving everyone plenty of time to migrate gradually. By the time v4 arrives, it will essentially be a cleaned-up version of the latest v3.x—all the modern features you're already using, minus the deprecated baggage you've hopefully already migrated away from.

You'll be able to adopt new features as they become available, migrate away from deprecated ones at your own pace, and when v4 finally ships, the upgrade should feel more like a routine update than a major undertaking.

This approach should make the v4 migration much smoother than previous major releases. Sometimes the best way to move fast is to move incrementally.

Open Source at its Best

Here's what really amazes me: more than half of all pull requests merged into Traefik come from contributors who aren't on the core team. The community doesn't just use Traefik—they improve it, extend it, and take it in directions we never imagined.

We've seen contributions from developers in dozens of countries, solving problems we didn't even know existed. Someone in Japan fixes a bug that someone in Brazil discovers. A developer in Germany adds support for a protocol that a team in California needs. A technical writer in France improves documentation that helps a team in Australia get started faster. A security researcher in India reports a vulnerability that protects users worldwide. Even feature requests and bug reports from users who never write a line of code help us understand what's needed. Every type of contribution—code, documentation, testing, reporting issues, suggesting improvements—makes the whole user experience better. It's open source at its best.

The community has written crazy providers (hello KNative 🫡), created awesome plugins for use cases we never considered (👋 Sablier), made the forum such a welcoming space (❤️ bluepuma77), and generally made Traefik far more useful than it would have been with just the core team.

Anniversary Contest

To celebrate this incredible milestone, we're doing something special. The next 50 contributors who get a pull request merged into the Traefik repository will receive a limited-edition 10th Anniversary Traefik t-shirt. These aren't just any t-shirts—we're talking exclusive anniversary design and enough geek cred to make your coworkers jealous.

The design itself is pretty special (see below). Since every Traefik release comes with a cheese codename, we've created a shirt featuring all the cheeses that have graced our releases over the past decade.

From Reblochon to Mimolette, from Roquefort to Saint-Nectaire – it's a delicious timeline of Traefik's journey. We have to give huge thanks to Pierre Keersbulik, who has been creating these awesome designs for us throughout the past 10 years. His creativity has given Traefik its distinctive visual identity, and this anniversary design is no exception.

Whether you're fixing a typo in the documentation, adding a feature to a plugin, or contributing to the core codebase, your contribution counts.

These will probably become collector's items, so don't say we didn't warn you 😉.

A Vintage Worth Celebrating

10 years ago, I had a simple idea about making reverse proxies smarter. Today, Traefik is part of the infrastructure that powers modern applications used by millions of people. Today, an entire team and community of developers continues to improve what has become a major open source project.

The cloud-native ecosystem will keep evolving, the AI revolution is reshaping our infrastructure, new challenges and possibilities will emerge, and developers will keep building amazing things. As long as there's traffic to route and services to connect, we'll be here—probably with some new feature you didn't know you needed.

Thank you for 10 incredible years. Thank you for the contributions, the feedback, the bug reports, and the debugging sessions. Thank you for the thoughtful feature requests, the patient explanations in forum threads, and the conference talks that help spread the word. Thank you to the maintainers who keep the community welcoming, the documentation writers who make complex concepts accessible, and yes, even the trolls who keep us on our toes and remind us that the internet is still the internet. Thank you for making this little project into something we're all proud of.

Here's to the next decade of making infrastructure just a bit more automatic.

]]>

Spring Cloud Gateway vs. Traefik Hub: When to Choose a Purpose-Built Gateway

Immánuel Fodor — Mon, 04 Aug 2025 22:46:22 GMT

After years of working with organizations navigating the microservices evolution, we want to set the record straight. The API gateway you choose today could save your teams' sanity or haunt your architecture for years to come. As teams break apart monoliths into distributed services, the need for reliable API gateway strategies has become critical.

Yet many organizations find themselves in gateway chaos as different teams implement routing, authentication, and rate limiting in entirely different ways. We've seen too many teams make the same mistake with Spring Cloud Gateway. We don’t want you to look back six months later, drowning in custom Java code scattered across a dozen microservices, wondering how you got there.

So let's talk about what Spring Cloud Gateway is, why purpose-built alternatives like Traefik Hub exist, and—most importantly—when each one makes sense for your specific situation. We want to help you understand when embedded gateway logic makes sense, and when it's time to centralize, because if you've been considering Spring Cloud Gateway for your API gateway needs, you might discover that what seemed like the obvious choice could become a significant maintenance burden as your organization scales.

What Spring Cloud Gateway Really Is (Hint: It's Not What You Think)

Let's clear up the biggest misconception first: despite its name, Spring Cloud Gateway is not a cloud-hosted service or managed gateway solution. It's a Java-based framework that you embed directly into your Spring Boot applications.

When you add it to your project via Spring Initializr, you're essentially embedding gateway functionality within your application code. This means your microservice becomes responsible for both its business logic AND gateway concerns like routing, filtering, and request transformation.

Here's what that looks like in practice:

// Code from the Spring Cloud Gateway getting-started guide:
// https://spring.io/guides/gs/gateway

@SpringBootApplication
@EnableConfigurationProperties(UriConfiguration.class)
@RestController
public class Application {

  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }

  // Gateway logic
  @Bean
  public RouteLocator myRoutes(RouteLocatorBuilder builder, UriConfiguration uriConfiguration) {
    String httpUri = uriConfiguration.getHttpbin();
    return builder.routes()
      .route(p -> p
        .path("/get")
        .filters(f -> f.addRequestHeader("Hello", "World"))
        .uri(httpUri)) // Upstream service
      .route(p -> p
        .host("*.circuitbreaker.com")
        .filters(f -> f
          .circuitBreaker(config -> config
            .setName("mycmd")
            .setFallbackUri("forward:/fallback"))) // Local path serving
        .uri(httpUri))
      .build();
  }

  // Business application logic mixed with gateway code
  @RequestMapping("/fallback")
  public Mono<String> fallback() {
    // Write your custom handler logic here
    return Mono.just("fallback");
  }
}

@ConfigurationProperties
class UriConfiguration {
  private String httpbin = "http://httpbin.org:80";
  public String getHttpbin() {
    return httpbin;
  }
  public void setHttpbin(String httpbin) {
    this.httpbin = httpbin;
  }
}

The above simple example shows the gateway code living inside your Spring Boot app, making your service a gateway and application rolled into one. While this technique offers tremendous flexibility, it also means that gateway logic becomes tightly coupled with your business logic. Every time you need to change, for example, a simple routing logic, you’ll be deploying application code. Fun times during incident response, right?

When Spring Cloud Gateway Makes Sense

Don't get us wrong, we're not here to bash Spring Cloud Gateway. Spring Cloud Gateway has legitimate use cases and impressive strengths.

If your organization has a small engineering team (think 5-10 developers) who are already neck-deep in the Spring ecosystem, and your API gateway needs are straightforward, it can be perfect. With a minimal learning curve, you can leverage familiar Spring patterns, integrate seamlessly with other Spring packages, and use existing tooling around Spring Boot. For teams that think in Java, implementing gateway logic as program code can be faster than learning new configuration formats or deployment patterns.

Because you're writing everything in Java, you can implement virtually any custom logic you need. Want to perform complex request transformations based on database lookups? No problem. Need to implement custom authentication flows that integrate with your existing Java libraries? Spring Cloud Gateway gives you complete control.

However, these strengths come with significant caveats that become more apparent as organizations scale.

The Hidden Costs of Embedded Gateway Logic

We met a fintech startup that said the Spring-only approach worked for them beautifully at first. They had three microservices, all Java-based, and needed some custom auth logic that integrated with their existing setup. Spring Cloud Gateway let them move fast without introducing new infrastructure complexity.

But the company reached out to us 18 months later when they had grown to 15 services across four product teams. Different teams were implementing policies differently, making API versioning and security requirements inconsistent. Debugging routing issues meant digging through application logs across multiple services. What started as a simple, pragmatic choice had become a coordination nightmare.

As your microservices architecture grows, embedding gateway logic in your applications creates several challenges, as Spring Cloud Gateway makes it so easy to customize everything:

Governance nightmares: Every team implements gateway logic differently. Your user service might handle rate limiting one way, while your payment service does it completely differently. Auditing security policies across dozens of services becomes nearly impossible.
Deployment complexity: Gateway changes require full application deployments. Need to update a rate-limiting rule? You're deploying an entire application, not just a tiny configuration change.
Knowledge silos: Gateway logic becomes tribal knowledge within each team. Onboarding new developers means they need to understand both business logic and gateway patterns specific to each service.
Performance overhead: The JVM startup time and memory footprint affect every service, even those that might not need complex gateway features.

And we could continue with different error handling patterns, no central visibility into your API surface, multi-service dependencies for routing changes, and so on.

We've seen it first-hand. It's not fun.

Introducing Traefik Hub: The Purpose-Built Alternative

Traefik Hub represents a fundamentally different approach. Built on the high-performance Go language and optimized for cloud-native environments, it operates as a dedicated, standalone gateway service that sits in front of your applications—completely decoupled from your business logic.

Instead of embedding code representing gateway logic in your business applications, you define routing and policies as declarative configurations. For example, in Kubernetes environments, using Kubernetes-native Custom Resource Definitions (CRDs):

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: user-api-route
spec:
  entryPoints:
    - websecure
  tls:
    certResolver: letsencrypt
  routes:
    - match: Host(`api.example.com`) && PathPrefix(`/users`)
      kind: Rule
      services:
        - name: user-service
          port: 8080
      middlewares:
        - name: rate-limit
        - name: oidc-auth-check

This configuration lives outside your application code, making it easier to audit, version control, and modify without touching your business logic.

What's great about this approach is that your gateway policies become infrastructure as code. Want to update rate limiting? You're not deploying Java applications; you're updating simple YAML files that can be reviewed, automatically tested, and rolled back independently.

Technical Head-to-Head Comparison

Let's break down the technical differences that actually matter in production:

Aspect	Spring Cloud Gateway	Traefik Hub
Technology Stack	- Java (JVM) - Spring Framework - Project Reactor - Spring Boot	- Go-based - Built on Traefik Proxy - Kubernetes-native
Environment Support	- Spring Boot microservices - JVM environments	Infrastructure-agnostic: - Cloud - Multi-cloud - Hybrid - On-premises
Configuration	- Imperative - Java program code-driven	- Fully declarative - GitOps-native - YAML/TOML/text labels
Protocols Supported	- HTTP/1 - HTTP/2 (only when using Reactor Netty, Tomcat, Jetty, or Undertow as web servers) - Other protocols are either experimental, or require further dependencies, or are not supported	- HTTP/1/2/3 - TCP - UDP - WebSockets - gRPC -All of them are production-ready and out of the box
Traffic Management	- Predicate/filter-based routing - Path rewriting	- Dynamic expression-based routing - Automatic service discovery through various providers - Load balancing - Traffic mirroring - Blue/green - Canary deployments
Governance	- Decentralized, per-team - Rate limiting - Circuit breakers - Relies on Spring Security extensions	- Centralized policies - Built-in distributed rate limiting - OPA integration - Multi-cluster dashboard - Audit trails
Security	OAuth2, JWT, mTLS, RBAC via Spring Security integration	OIDC, LDAP, JWT, HMAC, OAuth2, API keys, OPA policy enforcement, Coraza WAF
Deployment	Embedded in various sprawling Spring Boot microservices, JVM runtime required	Standalone gateway service, native Kubernetes and other container orchestration integration, and optimized for cloud-native environments
Performance	- Intermediate byte code - JVM startup + runtime overhead	- Native binary - High throughput - Low latency
Observability	Per-service monitoring, usually requiring additional tools such as Micrometer and Spring Boot Actuator	Unified real-time dataflow metrics with first-class OpenTelemetry metrics, distributed tracing, and logs; pre-built Grafana dashboards, traffic debugger
Learning Curve	Familiar with Java teams	New concepts to Java folks, but standardized, shared by 3.4+ billion Traefik downloads worldwide
Extension Model	Spring-based filters/predicates; custom Java plugins and packages	Go and WebAssembly (WASM) plugins; language-independent middleware

With Spring Cloud Gateway, as you can see, most of the time you're either writing custom code or hoping someone else has solved your problem.

Migrating Custom Logic: From Java Filters to Go Plugins

One common concern about moving from Spring Cloud Gateway to Traefik Hub is: "What about our custom logic? We're a Java shop!" The good news is that most custom gateway logic can be translated into Traefik's built-in middleware or plugins.

Here's an example of implementing API Key authentication:

Spring Cloud Gateway (Java):

// Custom-built API Key auth since it's not available out of the box
@Component
public class ApiKeyAuthFilter implements GatewayFilter {

    private static final String API_KEY_HEADER = "X-API-Key";
    private static final String VALID_API_KEY = "your-api-key"; // Ideally fetched from a configuration file with additional file handling logic polluting the gateway

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        List<String> apiKeyHeaders = exchange.getRequest().getHeaders().get(API_KEY_HEADER);
        if (apiKeyHeaders == null || apiKeyHeaders.isEmpty() || !VALID_API_KEY.equals(apiKeyHeaders.get(0))) {
            exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
            return exchange.getResponse().setComplete();
        }
        return chain.filter(exchange);
    }
}

Traefik Plugin (Go):

// You don't need a custom plugin for API Keys, it's already built-in! :)

# Use this YAML config instead:
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: apikey-auth
spec:
  plugin:
    apiKey:
      keySource:
        header: X-API-Key
      secretValues:
        - "urn:k8s:secret:apikey:secret" # your-api-key

And as you can see, most of the time, for the usual, standard auth and API traffic management use cases, it’s not even needed to write custom code, just a couple of configuration lines.

While there's a learning curve for Java teams adopting Go, even writing a custom plugin often results in simpler, more maintainable code that's easier to test and deploy independently. And here's the kicker: you can also write plugins in WebAssembly, which means you could technically write them in Java if you really wanted to (though we wouldn't recommend it).

The Strategic Shift: Centralized Gateway Governance

The real decision isn't just about technology: it's about organizational strategy. Embedding gateway logic in applications might seem a simpler effort initially, but it often creates long-term governance challenges:

With Spring Cloud Gateway, auditing authentication and authorization policies requires examining code across dozens of repositories. With centralized gateways, your security policies are defined in one place, version-controlled, and auditable by anyone with basic YAML knowledge.
When you need to optimize performance, routing, or implement new caching strategies, embedded gateways require coordinated updates across multiple teams. Centralized gateways allow instant, organization-wide improvements.
New regulatory requirements often demand changes to logging, rate limiting, or data handling. Centralized gateways make these compliance updates trivial; embedded gateways turn them into massive coordination efforts.

And the hidden costs nobody talks about: debugging time when issues span multiple services with different gateway implementations. Lost revenue during the extended time to recover from failures. Coordination overhead when you need to make changes across teams. Infrastructural cost overhead from running JVM instances under every service. Knowledge transfer costs when developers move between teams.

With centralized gateways, you're trading some initial investment for long-term operational simplicity. Most teams find the break-even point somewhere around 6-12 months.

Planning for API Growth: The Developer Portal Advantage

One often-overlooked benefit of centralized gateways like Traefik Hub is their integration with API management platforms. As your API ecosystem grows, you'll likely need:

API documentation: Centralized gateways can automatically generate OpenAPI specifications even for legacy services
Developer onboarding: Self-service API access with automated provisioning on a developer portal
Usage analytics: Understanding which APIs are used, by whom, and how
Lifecycle management: Versioning, deprecation, and migration strategies

These capabilities are much harder to implement when gateway logic is scattered across individual services.

A Decision Framework That Actually Works

Stick with Spring Cloud Gateway if:

You're a small, cohesive team (under 10 developers), and don’t plan to grow
You're deeply invested in Spring and don't plan to introduce new frameworks or languages
Your gateway needs are simple and unlikely to evolve significantly
You value development speed over operational consistency

Move to Traefik Hub if:

You're scaling beyond a single team
Governance and standardization are becoming pain points
You want to separate infrastructure concerns from application logic
Performance and resource efficiency matter
You're adopting cloud/Kubernetes/GitOps-native practices and want infrastructure as code
You need unified observability across all your APIs
You find beauty in configuring existing middlewares and plugins without writing too much custom code
You want to future-proof your infrastructure

The Migration? It's Doable!

If you're considering a migration, here's a timeline that's worked for several organizations we've helped.

Months 1-2: Assessment Phase

Start by cataloging what you actually have. Most teams are surprised by how much custom logic they've accumulated. Document the patterns, identify the truly custom stuff versus the boilerplate.

Months 3-4: Proof of Concept

Pick your simplest service and migrate it first. This gives you experience with Traefik Hub without risking your most critical flows. Translate any custom logic to built-in middleware or plugins during this phase.

Months 5-8: Gradual Rollout

New services use the centralized gateway first. Then migrate existing services in complexity order: simple routing rules first, complex custom logic last. Or the most painful first and the easiest ones last.

Months 9-12: Optimization

This is where you get the payoff. Implement organization-wide policies, set up proper monitoring, and start seeing the operational benefits.

The key is not to rush it. We've seen teams try to migrate everything in a month and create more problems than they solved.

Why This Should Matter for You

The technology choices you make today affect your team's growth and flexibility tomorrow. Teams locked into Spring Cloud Gateway are essentially committed to Java-centric environments for the foreseeable future. Teams that understand and embrace modern gateway patterns can work across different technology stacks more easily. From a hiring perspective, it's easier to find developers who can work with YAML than it is to find experienced Spring specialists. The talent pool is just broader.

The question isn't really "Spring Cloud Gateway vs. Traefik Hub." It's "embedded gateway logic vs. centralized gateway architecture." The technology choices flow from that strategic decision. And the best technology decision is the one that lets your team ship features instead of debugging infrastructure. Choose accordingly.

]]>

Beyond the Models: Operationalizing Enterprise AI

Sudeep Goswami — Wed, 30 Jul 2025 14:53:13 GMT

Today’s AI landscape moves at a breakneck speed. Hugging Face alone now hosts nearly two million AI models—hundreds of thousands added each month, exemplifying the astounding pace of innovation. While remarkable, this proliferation creates an often-overlooked challenge: the true difficulty in enterprise AI isn't picking the perfect model; it’s building a flexible, scalable architecture around it.

The Real Challenge: Operationalizing AI

Choosing a model (e.g., GPT-4, Claude, Grok, or Llama) is just the start. The deeper, more pressing issue for enterprises is operationalizing these models effectively. Organizations must navigate complex decisions:

Model Selection: Should your organization use open-source models for flexibility, proprietary models for advanced capabilities, or custom-trained models tailored to specialized applications?
Deployment Environment: Is the public cloud the right choice for its scalability, or does your enterprise require the enhanced security of a private cloud? Or maybe the best option is a carefully managed hybrid solution.
Implementation Approach: Will your teams benefit more from assembling best-of-breed components internally, choosing vendor-provided turnkey solutions for rapid deployment, or adopting a balanced hybrid approach?

Most enterprises land somewhere in between—heterogeneous environments are the norm, not the exception. Successfully managing this complexity becomes the defining factor for scalable AI operations.

No AI Without APIs. And No APIs Without API Management.

AI endpoints and APIs are deeply intertwined. APIs have become the lifeblood of agentic AI systems, acting as secure bridges for agent workflows, enabling dynamic endpoints for tools, data access, and maximizing operational efficiency. This explosion of APIs necessitates robust API management, turning APIs into the vital control plane for your entire AI infrastructure. They manage secure access, enforce governance, and facilitate dynamic behavior, which is crucial for agentic AI workflows.

APIs also enable enterprises to implement consistent policy enforcement, streamlined developer experience, and comprehensive audit capabilities across their entire AI ecosystem. The rise of agentic AI further underscores the importance of APIs as essential bridges connecting diverse models and applications securely and efficiently.

Avoiding the Biggest Anti-Pattern in AI Architecture

One common yet detrimental practice is embedding API runtime capabilities directly within the model runtime environment. This creates substantial problems that limit scalability and agility:

Vendor Lock-in: Ties your organization to specific model providers, reducing flexibility and increasing costs.
Limited Enterprise Capabilities: Embedded APIs frequently lack advanced features needed for large-scale, mission-critical operations.
Non-Standard Compliance: Frequently breaks compatibility with widely accepted standards, such as OpenAI’s API specifications, complicating integration and maintenance.

You might already face these issues if:

Your organization struggles to integrate API token generation seamlessly with existing identity management systems.
You lack standardized, vendor-neutral observability tools for effective monitoring.
Your AI endpoints aren't easily shareable through centralized developer portals.

Recognizing and addressing these indicators early can significantly enhance your ability to scale AI responsibly.

Strategic Decoupling Through AI Gateways

A superior architectural choice involves strategically decoupling your model runtime from your API runtime, connecting them via an AI gateway such as Traefik Hub. Here’s why it matters:

1. Future-Proof Flexibility

Models constantly evolve. Decoupling ensures seamless integration of new models, allowing your enterprise to adopt emerging technologies quickly without disrupting existing operations.

2. Centralized, Scalable Security

Manage authentication, authorization, and access control across diverse models centrally, maintaining consistent, enterprise-grade security and guardrail policies regardless of which model your applications consume.

3. Infrastructure and Policy-as-Code

Use Kubernetes-native, cloud-native, and infrastructure-as-code (IaC) approaches to automate deployments, significantly reducing operational overhead. Implement and enforce policy standards like semantic caching, guardrails, and content filtering in a structured and version-controlled manner.

4. Vendor-Neutral Observability

Leverage OpenTelemetry to achieve consistent, vendor-neutral monitoring of metrics, logs, and traces across your entire infrastructure, ensuring deep visibility and efficient troubleshooting capabilities.

Practical Deployment Scenarios

Let’s look at how strategic decoupling works practically:

Private Cloud with Containerized Models

Deploy GPU-accelerated, containerized models using platforms like NVIDIA NIMs or KServe:

Centralized API gateways handle developer portals, JWT authentication, rate limiting, and PII masking.
Semantic caching via Redis, Milvus, or Weaviate improves efficiency and cost management.
Advanced routing and canary deployments enable the safe and dynamic introduction of new models, reducing risk and accelerating innovation.
Observability metrics allow for custom scaling flows based on AI specific metrics like token count, latency per model, and more.

Public Cloud Managed Services

Using managed services like AWS Bedrock, Azure AI Foundry, Google Vertex AI, or OCI GenAI:

Standardize API interactions via AI gateways integrated with OpenAI-compatible proxies.
Maintain governance through centralized policy controls and cloud identity providers.
Enable model switching and updates without any client-side disruptions.
Provide unified developer experiences across different managed services.

Scaling Responsibly: Guardrails and Cost Efficiency

To scale AI responsibly and economically, you’ll need the following capabilities:

Content Guardrails: Use gateways to enforce responsible AI practices by masking, blocking, or simply flagging sensitive data, inappropriate data, or off-topic content.
Semantic Caching: Implement semantic caching to reduce redundant inferencing, significantly cutting costs and enhancing performance.
Dynamic Routing and Experimentation: Avoid hard-coding models to applications; dynamically route traffic to test and optimize model performance seamlessly, ensuring flexibility and robustness.

Start Small, Build for Scale

Enterprises often face complexity paralysis. The solution is incremental yet strategic:

Begin with an AI Gateway: Establish this crucial decoupling foundation to enable flexibility from the outset.
Incorporate API Management: Layer in robust governance, comprehensive observability, and efficient developer experience.
Progressively Add Policies: Gradually expand with semantic caching, content filters, and detailed usage policies to enhance security and efficiency.
Prepare for Agentic AI: Build readiness for autonomous workflows and multi-model dynamic interactions, positioning your enterprise as a leader in next-generation AI.

The Bottom Line

True scalability in enterprise AI isn’t about betting on today's "best" model—it’s about building an adaptable, resilient architecture. By decoupling runtimes, implementing robust API management, and adhering to open standards, enterprises can confidently face the challenges that lie ahead in AI advancements.

Remember: Models are commodities. Architecture is your strategic advantage. Build wisely, and you’ll not just scale AI, you'll scale your enterprise's agentic future.

]]>

The Great Kubernetes Ingress Transition: From ingress-nginx EoL to Modern Cloud-Native

Emile Vauge — Thu, 24 Jul 2025 15:01:13 GMT

The Uncomfortable Reality of Maintenance Mode

The Kubernetes ingress landscape faces an unprecedented crisis. With ingress-nginx officially projected to enter maintenance mode, thousands of organizations running critical production workloads suddenly find themselves at a crossroads. No new features. No major improvements. Just maintenance patches for a controller that powers much of the internet's Kubernetes traffic.

This isn't just another project lifecycle transition—it's an infrastructure emergency waiting to happen. The #IngressNightmare vulnerabilities revealed fundamental architectural flaws that go beyond patchable bugs. When your ingress controller can be compromised through malicious configuration injection, you're dealing with systemic design problems, not isolated incidents.

The proposed solution? InGate—a complete rewrite that exists today as little more than an ambitious roadmap with a handful of commits. The timeline? Years of development before reaching production readiness. The reality is organizations can't wait years to address critical security vulnerabilities that exist now.

Meanwhile, platform teams face an impossible choice: continue running outdated and vulnerable infrastructure or embark on disruptive migrations to entirely different ingress solutions, losing years of operational knowledge and configuration investments in the process.

Key Requirements for a New Ingress Controller

With ingress-nginx entering maintenance mode and InGate still years from production readiness, a gap has emerged in the ingress controller world. The Kubernetes community faces an immediate and critical challenge: the need for a new ingress controller. The stakes are incredibly high, given that ingress-nginx currently powers millions of these workloads. Therefore, any viable solution must meet four non-negotiable requirements.

1. Security by Design

The controller must be architecturally immune to the template injection vulnerabilities that plague NGINX-based solutions. This means no raw configuration templating, no dynamic library loading, and no string interpolation of user inputs into configuration files.

2. Broad Ecosystem Adoption

Any replacement must have proven production scalability and a thriving community. Organizations can't afford to bet their infrastructure on experimental or niche solutions during a critical transition period.

3. Gateway API Leadership

The future of Kubernetes networking lies in Gateway API, not “legacy” Ingress resources. The ideal controller should be a leader in Gateway API implementation, providing a clear migration path toward modern standards.

4. ingress-nginx Compatibility

Most critically, there must be a practical migration path that respects existing investments. Organizations have spent years building expertise around ingress-nginx annotations and configurations—this knowledge shouldn't become obsolete overnight.

Can Traefik be the One?

Given Traefik's wide adoption across the cloud-native ecosystem, it's worth exploring whether it could satisfy these four critical requirements and serve as a practical bridge during this transition period. Here's how Traefik aligns to the four requirements we just discussed:

Security: A Decade of Secure-by-Design Architecture

Traefik's security advantages aren't accidental—they're the result of fundamental architectural decisions made a decade ago. When I designed Traefik ten years ago, I made some critical architecture decisions:

Go over C/C++: Choosing Go eliminated entire classes of memory safety vulnerabilities and optimized for interoperability with the cloud-native ecosystem
Static linking over dynamic libraries: It is impossible to execute code not already part of the binary and the code base
Structured parsing over templating: Configuration inputs are parsed into strongly typed Go structs, not interpolated into template strings
Minimal attack surface: No external component, like admission controllers, have elevated cluster privileges

These weren't performance optimizations—they were security-first decisions that have proven prescient in light of today's threats.

Adoption: Battle-Tested at Scale

Traefik isn't an experimental solution—it's a proven platform trusted by organizations worldwide:

3.4+ billion downloads across its lifetime
55,000+ GitHub stars and growing showcasing a strong preference
800+ contributors actively developing and maintaining the codebase
Production deployments in a wide range of startups and enterprises
Cloud-native DNA built specifically for dynamic, containerized environments

Unlike solutions built for traditional infrastructure and later adapted for Kubernetes, Traefik was designed from the ground up for cloud-native environments where services appear and disappear dynamically. For a decade, Traefik's modern and flexible architecture has been proven extremely reliable at scale across a wide range of enterprise deployments.

Gateway API: Leading the Standard

Traefik leads Gateway API development and ensures seamless compatibility. As a key contributor to the Gateway API specification, Traefik Labs makes compatibility a fundamental design principle from day one.

With Gateway API now production-ready and positioned to become the new standard for exposing workloads in Kubernetes over the coming years, early adoption becomes a strategic advantage rather than a risky experiment. Organizations investing in Gateway API today are building toward the inevitable future of Kubernetes networking.

With full Gateway API v1.3 support in Traefik v3.5, organizations gain access to the most advanced Kubernetes networking capabilities available.

This represents true leadership in the future of Kubernetes networking. While other solutions treat Gateway API as an experimental add-on, Traefik has made it a core competency, positioning organizations for the networking standards that will define the next decade of cloud-native infrastructure.

Ingress NGINX Compatibility: The Missing Piece

The final requirement—practical migration support—has been the missing piece ... until now.

Making Traefik NGINX Compatible

Rather than building yet another ingress controller from scratch, we focused on three core objectives that would provide immediate value to the community:

Support the most common use cases: We analyzed real-world ingress-nginx deployments and prioritized the annotations that represent 80% of actual usage.
Preserve existing investments: We wanted organizations to use their existing ingress-nginx resources without modification. Teams shouldn't need to rewrite manifests, retrain engineers, or disrupt operational workflows in this transition.
Eliminate injection vulnerabilities: Most critically, we wanted to securely parse NGINX ingress resources and transform them into Traefik's internal object configuration.

This approach offers compelling advantages:

Leverage existing strengths: Security, performance, and Gateway API leadership
Respect user investments: Preserve years of ingress-nginx expertise and configurations
Accelerate migration: Provide immediate relief rather than years-long development cycles
Enable gradual transition: Bridge from Ingress to modern Gateway API resources.

Here's a quick look at how it works:

How to Replace ingress-nginx with Traefik, Step-By-Step

Let’s say you have an ingress resource deployed on your cluster with ingress-nginx as an ingress controller:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-nginx-annotation
  namespace: default
  annotations:
    # BASIC Authentication
    nginx.ingress.kubernetes.io/auth-type: "basic"
    nginx.ingress.kubernetes.io/auth-secret-type: "auth-file"
    nginx.ingress.kubernetes.io/auth-secret: "default/basic-auth"
    nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"
    # SSL Redirect
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: whoami.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
  tls:
    - hosts:
        - whoami.localhost
      secretName: external-certs
---
kind: Secret
apiVersion: v1
metadata:
  name: basic-auth
  namespace: default
type: Opaque
data:
  # user:password
  auth: dXNlcjp7U0hBfVc2cGg1TW01UHo4R2dpVUxiUGd6RzM3bWo5Zz0=

This snippet exposes a web server with basic authentication and http to https redirection, that can be accessed with the following `curl` command:

curl http://whoami.localhost -L -u "user:password" --location-trusted

If you want to expose with Traefik Proxy, you only need to deploy Traefik and enable the NGINX Provider:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik
  namespace: default

spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik

  template:
    metadata:
      labels:
        app: traefik

    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v3.5.0
          imagePullPolicy: IfNotPresent
          args:
            - --log.level=DEBUG
            - --api.insecure
            - --api.debug
            - --entrypoints.web.address=:80
            - --entrypoints.websecure.address=:443
            - --experimental.kubernetesIngressNGINX
            - --providers.kubernetesIngressNGINX
          ports:
            - name: web
              containerPort: 80
            - name: admin
              containerPort: 8080
            - name: websecure
              containerPort: 443

---
apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: default
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: websecure
    - protocol: TCP
      port: 8080
      name: admin

--providers.kubernetesIngressNGINX will enable Traefik to discover and parse ingress-nginx ingress resources.

Et voilà! The web server is exposed by Traefik using the same ingress resource:

Hostname: whoami-6f57d5d6b5-19ncq
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.34
IP: fe80:: c498:5aff: fef7:be99
RemoteAddr: 10.42.0.38:47476
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/8.7.1
Accept: */*
Accept-Encoding: gzip
Authorization: Basic dXNlcjpwYXNzd29yZA==
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-7578bc64c5-pdjqm
X-Real-Ip: 10.42.0.1

As you can see, the migration path from ingress-nginx to Traefik could not be simpler! Bring your existing NGINX ingress resources, Traefik will do the rest.

As a cherry on top, Traefik Hub extends the open-source foundation with advanced API & AI Gateway & Management capabilities. This allows teams to start with secure ingress migration and evolve toward full API management capabilities without architectural disruption.

Traefik API Runtime Platform Core Capabilities

Full Support for High-Usage Annotations

The NGINX Provider doesn't attempt to support every possible ingress-nginx annotation—instead, we applied a data-driven approach to annotation selection based on real-world usage patterns and migration priorities.

The resulting annotation support spans the most critical operational requirements:

Authentication and authorization mechanisms
SSL/TLS configuration and backend communication
Session management and load balancing preferences
CORS policies for modern web applications
Advanced routing capabilities

This curated approach means most organizations can migrate their core ingress configurations immediately while gradually adopting Traefik-native features for advanced use cases.

For the complete list of supported annotations, behavioral differences, and configuration examples, see the comprehensive NGINX Ingress Provider routing documentation.

The Path Forward: Community-Driven Evolution

The NGINX Provider isn't a finished product—it's a foundation designed for community-driven evolution toward becoming a true drop-in replacement for ingress-nginx. As mentioned, our current implementation focuses on 80% of use cases.

The provider's architecture makes community contributions straightforward. Adding support for a new annotation typically involves:

Mapping the annotation to an equivalent Traefik middleware or option
Adding tests and documentation for the new functionality

This isn't just about building software—it's about building a sustainable community effort around practical migration needs. Each contribution makes the provider more useful for everyone facing the ingress-nginx transition.

Our long-term vision is clear: build a true drop-in replacement for ingress-nginx that provides both compatibility and modern capabilities.

Solutions, Not Promises…

The Kubernetes ingress crisis is happening now. ingress-nginx is about to be in maintenance mode. Many critical vulnerabilities have been discovered recently. Organizations need practical solutions today, not promises about tomorrow.

The Traefik NGINX Provider represents a pragmatic approach to an urgent problem. Instead of waiting years for InGate to mature or forcing disruptive migrations to completely different solutions, we're providing a secure bridge that respects existing investments while enabling future innovation. This isn't about perfect compatibility yet—it's about practical migration paths.

The great Kubernetes ingress transition is happening whether we're ready or not. The question isn't whether you'll need to migrate from ingress-nginx—it's whether you'll choose a practical bridge that's available today or wait years for a solution that might never fully materialize.

Be pragmatic, not idealistic. Your production workloads depend on it.

The NGINX Ingress Provider is available as an experimental feature in Traefik 3.5. For complete setup instructions, visit the setup guide. For configuration examples and the full list of supported annotations, consult the routing configuration documentation. Join the conversation and contribute to the future of practical Kubernetes ingress migration.

]]>

Traefik Proxy 3.5 "Chabichou": A Delicate Masterpiece

Emile Vauge — Thu, 24 Jul 2025 14:55:20 GMT

Traefik Proxy 3.5 has arrived! Codenamed "Chabichou" (the elegant goat cheese from Poitou, France, with its distinctive conical shape and creamy, tangy flavor that develops beautifully with age), this release introduces an exciting Ingress NGINX Provider for seamless ingress-nginx migration, a completely rebuilt React-based dashboard, and full Gateway API v1.3 support, alongside post-quantum cryptography and enhanced ACME capabilities.

Let's unwrap this delicate masterpiece!

Key Features of Traefik Proxy v3.5

Ingress NGINX Provider: Your Migration Path Forward

With ingress-nginx entering maintenance mode and the ingate project still in early stages, the Kubernetes community needed a clear migration path. That's where Traefik 3.5 steps in! We're introducing an experimental Ingress NGINX Provider (#11844) designed to enable ingress-nginx users to migrate seamlessly to Traefik. With it, organizations can continue to use their existing ingress-nginx resources, without modification, directly in Traefik. This transition shouldn't require teams to rewrite manifests, retrain engineers, or disrupt operational workflows.

This isn't about building 100% compatibility with every NGINX annotation—instead, we're focused on providing a practical migration foundation that covers the most common use cases. Our goal is simple: offer you a drop-in replacement for the majority of scenarios, then enable you to progressively migrate to modern standards like Gateway API at your own pace.

This experimental feature represents our commitment to the community's needs during this transition period. It supports essential annotations and core functionality, with the potential for rapid evolution based on community feedback and contributions.

We're excited to see how the community embraces and enhances this foundation! Read more about this in this deep dive article.

Dashboard Renaissance: Modern Interface, Enhanced Dev Experience

Say hello to the future of Traefik management! The dashboard has been completely rebuilt using React (#11674), delivering a sleek, responsive, and intuitive interface that makes managing your proxy configuration a pleasure. Additionally, this provides a better development experience with the ability to use mocks in development mode.

The new React-based dashboard provides enhanced performance, better accessibility, and lays the foundation for exciting features in future releases.

Gateway API v1.3: Leading the Kubernetes Evolution

Traefik continues its leadership in Kubernetes networking with complete support for Gateway API v1.3 (#11719). This latest specification keeps improving this new standard and Traefik remains at the forefront of Kubernetes networking, keeping your infrastructure ahead of the rapidly evolving cloud-native landscape.

Additional Enhancements That Make a Difference

Future-Ready Security with Post-Quantum Cryptography

Prepare your applications for tomorrow's security challenges with the introduction of X25519MLKEM768 for Post-Quantum-Secure TLS (#11731). This cutting-edge cryptographic approach ensures your communications remain secure against future quantum computing threats.

Our sincere appreciation goes to fzoli for implementing this forward-thinking security enhancement that positions Traefik at the front line of cryptographic innovation.

ACME Excellence: Enhanced Certificate Management

Certificate management becomes even more powerful with several key improvements:

OCSP Stapling Support (#8393)—Thanks to alekitto for implementing this performance and security enhancement
HTTP Challenge Delay Configuration (#11643)—Kudos to ldez for adding fine-grained challenge control
ACME Provider HTTP Timeout (#11637)—Appreciation to tkw1536 for improving reliability and customization

Advanced Health Check Capabilities

Health monitoring gets more sophisticated and flexible with unhealthy Interval Configuration (#10610)—Gratitude to sswastik02 for more granular health monitoring control

Kubernetes Integration Improvements

Kubernetes users benefit from enhanced compatibility and consistency: consistent Prefix Matching (#11203)—Our thanks to charlie0129 for aligning Ingress behavior with Kubernetes documentation standards

Middleware and Authentication Enhancements

Improved ForwardAuth Context Handling (#11817)—Special recognition to bengentree for better context cancellation handling
Enhanced Error Middleware Visualization (#11806)—Appreciation to sevensolutions for improved StatusRewrites option display

Plugin Development Evolution

Plugin developers gain more control and safety with the ability to enable unsafe operations through plugin manifests (#11589). Thanks to Rydez for expanding the plugin ecosystem's capabilities.

A Toast to Our Exceptional Community

Like the careful aging process that creates Chabichou's distinctive character, Traefik v3.5 is the result of months of dedication from our incredible community. Every bug report, feature request, code contribution, and documentation improvement has shaped this release into something truly special.

To every developer who opened an issue, every contributor who submitted a pull request, every user who provided feedback, and every community member who helped others—you are the artisans behind Traefik's continued excellence. Your passion and expertise continue to drive innovation in cloud-native networking.

Want to be a part of this awesome community? Join the discussions on our Community Forum or on GitHub where ideas flow as freely as conversations in a French café!

Essential Resources and Next Steps

Download Traefik 3.5: GitHub Releases | Docker Hub
Documentation: Complete Documentation
GitHub Repository
Community Forum
Learn More: Official Website | Blog

]]>

Simplifying Enterprise Connectivity: Traefik Labs Expands Microsoft Partnership with Azure Arc and AKS

Sudeep Goswami — Wed, 14 May 2025 12:32:48 GMT

If you’re steering an enterprise through today’s hybrid and multi-cloud landscape, you’re likely grappling with a paradox: the flexibility of distributed systems fuels innovation, but it also breeds complexity that can stall progress. APIs and AI workloads are the engines of modern business, yet managing them across on-premises data centers, public clouds, and edge locations often feels like navigating a maze.

At Traefik Labs, we’re partnering with Microsoft to transform this challenge into an opportunity, delivering a Kubernetes-native platform that unifies API and AI connectivity via Azure Arc, Azure Kubernetes Service (AKS), and the Azure Marketplace.

Our mission? To simplify your infrastructure, secure your workloads, and accelerate your future.

The Distributed Enterprise Dilemma

The shift to distributed architectures has unlocked unprecedented agility, but it’s also exposed critical pain points that enterprises can’t ignore. Fragmented API management forces teams to wrestle with manual and UI-centric configurations, leading to security blind spots and operational inefficiencies. AI workloads, with their unique demands for specialized routing, caching, and security, strain legacy platforms, causing performance bottlenecks and architectural conflicts. Inconsistent authentication and policy enforcement create compliance risks, while disruptive migrations to modern API management drain resources and impact customer experiences.

These aren’t just technical hurdles—they’re strategic barriers to innovation and growth. As AI becomes a cornerstone of business transformation, enterprises need a connectivity platform that scales responsibly, governs effectively, and evolves without disruption. That’s where our expanded partnership with Microsoft comes in, offering a unified approach to conquer complexity and empower your teams.

Traefik and Microsoft: A Vision for Unified Connectivity

With over 3.3 billion downloads, Traefik Proxy is a cornerstone of cloud-native connectivity. Our collaboration with Microsoft, strengthened through the Azure Arc ISV Partner Program, integrates this expertise with Azure’s ecosystem to deliver seamless API and AI management across any infrastructure. By combining Traefik’s Kubernetes-native simplicity with Azure Arc and AKS, we’re redefining enterprise connectivity.

Let’s explore how.

Azure Arc: Seamless Connectivity Across Any Environment

Azure Arc enables you to manage Kubernetes clusters anywhere—on-premises, in multiple clouds, or at the edge—through a single control plane. Traefik builds on this foundation, providing a cloud-native application proxy that ensures consistent traffic management and governance across all Arc-enabled environments.

Imagine deploying zero-config API routing that eliminates manual setup, reducing operational overhead from day one. Your security policies are centralized and powered by TLS and AI-specific content guard, ensuring compliance without complexity. You’re even able to optimize AI workloads with semantic caching, slashing latency and computational costs.

This isn’t just about connectivity—it’s about freeing your teams to innovate. With Traefik for Azure Arc, you gain a platform that adapts to your infrastructure, whether you’re running Kubernetes in a data center or scaling AI models at the edge. Our declarative, GitOps-driven approach integrates seamlessly with your CI/CD pipelines, making infrastructure management as agile as your development processes.

Azure Marketplace: Streamlined AKS Deployment

Since joining the Azure Marketplace in 2023, Traefik Proxy has empowered AKS users with one-click deployment, integrated billing, and instant scalability. As Microsoft’s recommended ingress controller for AKS baseline architecture, Traefik delivers unmatched compatibility and performance. You can set up advanced traffic management—supporting modern protocols like HTTP/3 and gRPC—in minutes, not hours. GitOps automation ensures your operations are declarative and scalable, while real-time traffic routing based on service health keeps your applications resilient.

For DevOps teams, this means less time configuring infrastructure and more time delivering value. Whether you’re modernizing microservices or launching AI-driven APIs, Traefik on AKS provides a foundation that’s both powerful and intuitive, aligning your Kubernetes strategy with business goals.

A Progressive Journey for Every Enterprise

Whether you’re a retailer streamlining e-commerce APIs, a healthcare provider deploying AI diagnostics, or a SaaS company scaling customer-facing services, your enterprise has unique needs and timelines. That’s why Traefik, integrated with Azure Arc and AKS, offers a three-phase adoption journey that meets you where you are and scales strategically—without disruption or vendor lock-in. This approach ensures immediate wins and long-term success, aligning technology with your business goals.

Phase 1: Application Proxy—Lay the Foundation

For teams adopting Kubernetes or managing microservices, our Application Proxy delivers zero-config service discovery, eliminating manual traffic management. Azure Arc enables deployment across on-premises and cloud clusters, while AKS users benefit from one-click setup via the Azure Marketplace. Real-time traffic routing ensures resilience, and Kubernetes CRDs simplify configuration. This phase slashes setup time, letting you launch new services with minimal resources.

Phase 2: API & AI Gateway—Secure and Optimize

As complexity grows, our API & AI Gateway unifies security and performance. Azure AD integration secures APIs and AI workloads, while semantic caching and content guard optimize AI traffic, cutting latency and ensuring compliance. Advanced traffic shaping—circuit breaking, retries—maintains stability. Azure Arc’s control plane enforces consistent policies across environments, delivering secure, high-performance connectivity for your distributed systems.

Phase 3: API Management—Drive Strategic Value

When APIs and AI are core to your business, our API Management phase empowers you with full lifecycle governance. Self-service developer portals accelerate innovation, unified observability provides actionable insights, and controlled versioning ensures smooth transitions. Azure Arc enables governance across all environments, transforming APIs into revenue-driving assets. This phase positions you to lead with data-driven services.

This journey is your path to agility and growth. By starting with simple routing, scaling to secure AI optimization, and achieving enterprise-grade governance, Traefik and Microsoft empower you to innovate without boundaries.

The Future Is Unified and AI-Ready

AI is reshaping competition, but scaling it requires infrastructure that balances performance, cost, and compliance. Traefik’s AI Gateway, integrated with Azure Arc, delivers intelligent semantic caching, context-aware routing, and robust governance for responsible AI. These innovations position you as a leader in a data-driven world.

Our partnership with Microsoft eliminates silos, accelerates API delivery, and unlocks revenue streams. This is the future of connectivity: unified, scalable, and AI-ready.

Take the Next Step

Ready to simplify your API and AI connectivity? With Traefik and Microsoft, you have a partner to streamline, secure, and scale your workloads. Here’s how to start:

Deploy on AKS: Launch Traefik via the Azure Marketplace for a seamless deployment experience.
Deploy on Azure Arc: Launch Traefik for Azure Arc via the Azure Marketplace and experience the “Any Infrastructure, Any Kubernetes” vision.
Explore Key Use Cases through Azure Jumpstart Drops: Discover how Traefik and Azure Arc redefine Kubernetes management, through bite-sized use-cases modules.
Dive Deep with a Self-guided Hands-on Lab: Walk through the entire three-step journey through a series of self-guided hands-on lab tutorials

At Traefik Labs, we’re passionate about helping you build a future where connectivity is seamless and innovation knows no bounds. Let’s take this journey together.

]]>

5 Smart API Gateway Strategies to Unlock Developer Productivity

Immánuel Fodor — Mon, 12 May 2025 11:09:19 GMT

How forward-thinking platform teams are transforming developer experience through modern API gateway practices

Developers expect speed and autonomy—and organizations demand control and security. The API gateway sits at the heart of this balancing act. The most successful platform teams are rethinking traditional gateway models, using modern strategies to supercharge developer productivity while maintaining ironclad governance, observability, and performance. Here’s how they’re doing it.

The Hidden Cost of Undocumented Infrastructure

Before diving into our strategies, allow me to share a personal experience that many platform leaders can relate to. When taking over leadership roles at several organizations, I've encountered the substantial challenge of undocumented infrastructure at scale.

The pain of uncovering what actually exists in data centers is like navigating through the "fog of war" in a strategy game like Age of Empires—spending significant time, effort, and money just to understand what you're working with. In the game, you start with a dark, foggy world, slowly revealing the map as you move forward. But for me, this wasn't a game—it was our production data centers. Every inch of fog essentially cost us time, money, and sometimes even sleep.

Fog of War in Age of Empires: https://altarofgaming.com/age-of-empires-civilizations/

I've even seen teams terrified of restarting their production gateway—not because of potential downtime, but because nobody knew what was configured there, what rules it was enforcing, or what custom configurations existed only in memory without any documentation or persistence. The entire setup was extremely fragile.

This painful experience taught me something important: we needed to stop firefighting and start thinking like a platform team. I had to build a team that could shift the mindset from reactive operations to proactive engineering.

When it comes to exposing APIs, frustration typically hits two key personas first: the API publisher who builds, owns, and exposes APIs, and the API consumer who uses these services. The gap between them requires governance that doesn't limit but empowers productivity. Here are five proven strategies that can transform your API gateway into a developer productivity powerhouse:

Strategy #1: Implement Zero-Friction API Onboarding

Reducing API integration time from days to minutes isn't just a dream—it's achievable with the right approaches. Here's a surprise, though: when we think about onboarding, we usually think about onboarding developers. But actually, the first persona to face onboarding challenges is the API publisher.

When new team members join or when onboarding a new API into API management, it often takes days, involves tickets, and relies on tribal knowledge. This is where the whole system can break down. By focusing on streamlining the publisher experience first, we create a foundation for developer productivity.

How Traefik Labs makes this possible:

Declarative configuration that allows for version-controlled API definitions
UI vs. Cluster ownership balance - whatever you create from the UI becomes a Custom Resource Definition (CRD) that you can manage through GitOps, making onboarding APIs repeatable and protecting configurations from accidental modification
Managed subscriptions to automate consumer onboarding, potentially eliminating the need for consumers to log into developer portals

It's easy to focus solely on the developer experience—but the API consumer is just as critical. Whether it's a partner integrating your APIs or an internal team building on top of your services, they need instant clarity, autonomy, and confidence. Yet, too often, consumers are left waiting on documentation, meetings, or manual approvals. The real transformation happens when consumers can discover, evaluate, and subscribe to APIs without delays. By removing these bottlenecks, organizations can accelerate time-to-value for every new integration.

Self-service API portal that includes:
- Comprehensive API specifications
- Self-service subscription options with flexible plans
- Try-out features for testing
- AI-assisted client code generation through the Alfred AI assistant

The goal is to achieve onboarding without meetings and waiting times—reducing API integration from days to minutes. This combination dramatically reduces the friction that typically slows down API integration and adoption for both publishers and consumers.

Strategy #2: Create Intelligent Observability Feedback Loops

Combining observability with your API gateway provides developers with contextual insights that dramatically reduce debugging time. Your API gateway serves as the gatekeeper of your infrastructure—securing, monitoring, and exposing APIs—making it essential to have visibility into what's happening at runtime.

Traefik Labs' approach includes:

OpenTelemetry integration that provides metrics, traces, and logs in a unified format to the tools you're most familiar with, whether self-hosted or cloud-based SaaS offerings
Observability-as-a-Service capabilities through our partnership with Treblle—with a simple plugin installation, you get instant insights into API behavior, compliance issues, and security problems based on observed traffic
Response debugging tools that provide granular visibility into API behavior

The real power comes from creating feedback loops that correlate deployment events with actual output metrics, traces, and logs. When you configure everything as code, put it in Git, and apply changes through GitOps pipelines, you can quickly identify the cause of unexpected issues. For example, if you suddenly see 500 errors, you can correlate them with recent PRs, which appear as annotations on your monitoring dashboard. This allows you to revert problematic changes quickly, mitigate incidents, and save the business from costly outages.

These tools transform debugging from reactive to proactive, allowing developers to move faster while resolving incidents before they escalate. The result? Significant reductions in time to recover from outages and overall cost savings for the business.

Strategy #3: Deploy Runtime Governance That Empowers Rather Than Restricts

Runtime governance needs to shift left while still empowering developers. Although governance puts checks in place, these should provide safe guardrails within which teams can move quickly—not restrictive barriers that slow innovation.

Traefik Labs enables:

Static analysis of your configuration in GitOps pipelines to catch configuration errors early and fix them before reaching production
Impact analysis that shows not just what changed but who will be affected by the change—for example, determining if a rate limit or quota configuration modification will impact more users than anticipated
Pull request workflows are enhanced with these semantic outputs that help reviewers understand both the change and its impact
Open Policy Agent (OPA) policies on CRDs for flexible, powerful governance without requiring a separate OPA installation—it's built directly into the gateway
OpenAPI Specification (OAS) validation to eliminate shadow and zombie APIs, which according to Treblle's latest report, account for 35% of API traffic

Traefik API Management allows you to set a flag to block every endpoint outside what's described in the OpenAPI specification. When enabled, all undocumented methods become inaccessible immediately. For legacy systems with undocumented APIs, our partnership with Treblle allows you to build OpenAPI specifications from observed traffic, transforming previously undocumented endpoints into documented ones.

# Also works with API Versions
---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: invoice-api
spec:
  openApiSpec:
    path: /openapi.yaml
    validateRequestMethodAndPath: true  # one simple setting, much less trouble

This approach shifts governance left in the development cycle, preventing issues before they become problems while preserving developer autonomy. It's like having a Swiss army knife for governance—you're not limited to what the tool offers but have an engine you can fine-tune to create safeguards that enable developers to move faster.

Strategy #4: Tame LLMs and AI by Turning Them Into Managed APIs

As AI becomes central to development workflows, new challenges are emerging. The AI landscape is evolving at an unprecedented pace—according to Stanford University's latest study, AI is becoming more powerful each day and is being embedded into everyday life with exponential growth in areas like medical devices. Organizations are going all-in on AI, with various models available worldwide.

With AI endpoints proliferating (Hugging Face alone hosts over 1.6 million models), companies and developer teams are experimenting extensively. This creates a fundamental need for API management and governance specifically tailored to AI workloads.

Traefik Labs' AI Gateway solution includes:

Centralized secret management that can turn any AI endpoint into a managed API—you no longer need to expose your OpenAI API key to all your mobile applications deployed to the public
Semantic caching that recognizes when prompts express the same meaning in different natural language sentences, freeing up costly GPU cycles while dramatically improving latency and reducing costs
Content firewalls that protect information flowing to and from AI models, with the ability to block requests/responses or mask sensitive data through a powerful rule-based system using Microsoft's open-source Presidio engine for natural language processing

Our demo shows this in action with a "Traffic Airlines Feedback Service" that uses JWT authentication, OPA middleware for authorization, and content guards to cleanse PII data flowing through the system. We also demonstrate semantic caching with our chat example, where semantically similar queries returned cached results with dramatically improved response times.

These capabilities allow organizations to harness the power of AI while maintaining control over how it's accessed and utilized—balancing innovation with responsible AI practices.

Strategy #5: Build Resilient API Ecosystems

Configuring API networks that reduce developer on-call burden and improve system reliability is essential for sustainable operations. With AI inferencing, additional challenges emerge around versioning, scalability, cost optimization, and handling sensitive data responsibly.

Traefik Labs focuses on:

High availability configurations to ensure consistent service
Flexible API versioning that decouples exposed API versions from model versions, allowing teams to experiment with different models without breaking client applications
The shared responsibility model for API and ML model versioning

Traefik Hub API Management offers an exceptionally flexible versioning system that can match on host, HTTP method, query parameters, headers, IP, authenticated user, and many other request attributes. This puts power in your hands to manage the lifecycle of APIs and AI workloads with less chaos and fewer incidents.

For example, your API might expose a "/v2" endpoint while the model running behind it could be at version 2.2 or 2.3. This decoupling allows you to experiment with and improve models without disrupting clients. With contracts and versioning present as code, you provide a consistent view to clients while enabling experimentation within your organization.

This approach creates resilient systems that can evolve while maintaining compatibility and reducing maintenance overhead—essential for managing both traditional APIs and increasingly important AI workloads.

Putting It All Together: From Theory to Practice

These strategies aren't just theoretical, they represent practical approaches that forward-thinking organizations can implement today. Traefik Labs demonstrates a complete infrastructure-as-code environment with Terraform, deploying a K3d cluster with Argo CD, Traefik, KServe for AI inferencing, and various other tools—all fully configurable through GitOps workflows.

We demo two key services:

A Traefik Airlines Feedback Service that uses JWT middleware, OPA policies, and content guards to provide secure, compliant AI-powered sentiment analysis
A Chat Service that implemented semantic caching to dramatically improve performance and reduce costs when handling similar queries

Unlocking What’s Next: Developer Productivity Starts at the Gateway

By transforming your API gateway from a simple traffic management solution into a comprehensive platform for developer productivity, you can accelerate innovation while maintaining the control and visibility your organization needs.

Whether you're managing a handful of APIs or thousands, or working to integrate AI capabilities responsibly into your applications, these strategies can help you create a more productive, secure, and resilient API ecosystem that drives business value.

Ready to unlock developer productivity through your API gateway? Learn more about how Traefik Labs can help transform your approach to API management and developer experience in 2025 and beyond.

]]>

Traefik Proxy 3.4 “Chaource” Is Ready to Serve

Emile Vauge — Tue, 06 May 2025 11:51:53 GMT

Traefik Proxy 3.4 is here, and it’s greater than ever! Codenamed “Chaource” 🧀 (a creamy delight from Champagne, France, has been crafted since Roman times), this release is packed with zesty new features, smoother operations, and a sprinkle of magic to keep your infrastructure running like a well-oiled fondue pot. Let’s cut through the rind and dig into the goodness!

Key New Features

Distributed Rate-Limiting with Redis

“Sharing is caring” just got a whole new meaning! Traefik’s new distributed rate-limiting (#10211 by longquan0104) lets you enforce traffic rules globally across instances using Redis. No more rogue services hogging bandwidth—this feature ensures fair play, even in a multi-node setup. Perfect for stopping API abuse or managing traffic spikes without breaking a sweat.

Here is an example to deploy a rateLimit middleware in Kubernetes:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
   name: test-ratelimit
spec:
   rateLimit:
      # ...
      redis:
         secret: mysecret

---
apiVersion: v1
kind: Secret
metadata:
   name: mysecret
   namespace: default
data:
   username: dXNlcm5hbWU=
   password: cGFzc3dvcmQ=

P2C (Power of Two Choices) Load-Balancing

Meet the Power of Two Choices (P2C) load balancer—your new traffic whisperer. Instead of blindly picking a backend, P2C intelligently selects two random candidates and routes requests to the least busy one (initiated by ifross89 in #10534). Under heavy traffic, P2C avoids overloading a single server “by accident”. This results in fewer bottlenecks and a steadier, more predictable load distribution.

Enabling P2C is easy. In your service’s load-balancer config, set the method to p2c instead of the default. For example:

http:
  services:
    my-service:
      loadBalancer:
        servers:
          - url: "http://10.0.0.1"
          - url: "http://10.0.0.2"
          - url: "http://10.0.0.3"
          - url: "http://10.0.0.4"
        method: "p2c"

Custom Server URL with Labels

A long awaited feature is part of this release, Chaource lets you define server URLs via labels on Docker, Swarm, ECS, Consul & Nomad (#11374 by yelvert). This feature is needed in case you don’t want to use the container’s IP but a custom URL instead.

Here is a Docker compose example deploying an app and setting custom server urls:

version: '3.8'
services:
  main:
    image: lorem/ipsum:latest
    deploy:
      replicas: 3
      labels:
        - "traefik.enable=true"

        - "traefik.http.routers.service1.entrypoints=https"
        - "traefik.http.routers.service1.rule=Host(`foobar1.example.com`)"
        - "traefik.http.routers.service1.service=foobar1"
        - "traefik.http.routers.service1.tls=true"
        - "traefik.http.services.service1.loadbalancer.server.url=http://foobar1:80"
        
        - "traefik.http.routers.service2.entrypoints=https"
        - "traefik.http.routers.service2.rule=Host(`foobar2.example.com`)"
        - "traefik.http.routers.service2.service=foobar2"
        - "traefik.http.routers.service2.tls=true"
        - "traefik.http.services.service2.loadbalancer.server.url=http://foobar2:80"

        - "traefik.docker.network=traefik"

More Delicacies

ACME Enhancements: Introduced new options acme.profile and acme.emailAddresses to customize certificate requests (#11597 by ldez). See the ACME documentation for details.
Kubernetes Improvements: Improved CEL validation for CRDs (#11311 by mloiseleur), ingress statuses were added for ClusterIP and NodePort Service Type (#11100 by mlec1) and we can now get a root CA to be added through config maps (#11475 by Nelwhix). Refer to the Kubernetes Ingress documentation.
Gateway API: It’s now possible to set rule priority in Gateway API TLSRoute (#11443 by augustozanellato)
Middlewares: An option preserveRequestMethod has been added to the ForwardAuth middleware to preserve the original HTTP method and to send the request body to the authorization server (#11473 by an09mous). See the ForwardAuth middleware documentation. The ErrorPage middleware now supports rewriting status codes dynamically (#11520 by sevensolutions). Check out the ErrorPage middleware documentation.
Sticky Sessions: You can now specify a cookie domain for sticky sessions (#11556 by jleal52). Refer to the Sticky Sessions documentation.
TLS Enhancements: Added options to disable TLS session tickets and to add extra CA certificates via Kubernetes ConfigMaps (#11609 by avdhoot). See the TLS documentation.
UDP Routing: Traefik can now route UDP traffic with systemd socket activation (#11022 by tsiid). Check out the UDP documentation.
Web UI: The dashboard now supports an auto theme option, switching between light and dark modes based on your system settings (#11455 by zizzfizzix). Refer to the Dashboard documentation.
Under the hood: experimental maps and slices have been replaced by stdlib (#11350 by Juneezee)

A Big Slice of Gratitude

Chaource wouldn’t be the same without our amazing community! To everyone who filed bugs, brainstormed features, or geeked out with us on GitHub—thank you! 🥂

Got questions? Compliments? Cheese puns? Share them in our Community Forum!

Useful Links

Traefik 3.4 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Who's in Charge? The Shared Responsibility Model for API and AI/ML Model Versioning

Sudeep Goswami — Thu, 17 Apr 2025 13:52:14 GMT

Your ML model just broke in production. Was it the API? The model? Or the versioning gap between them?

As organizations scale their AI operations, the traditional approach of handling versioning at a single layer is breaking down. This limitation has led forward-thinking enterprises to adopt what we call the "Shared Responsibility Model" for versioning, distributing responsibilities between API management platforms and ML serving platforms.

The Growing Complexity of AI at Scale

As machine learning transitions from experimental projects to business-critical applications, organizations face increasing complexity in managing both the APIs exposing AI capabilities and the underlying ML models themselves. Effectively deploying and managing these solutions isn't just a data science puzzle; it's a complex challenge involving engineering, IT operations, and business strategy, with Deloitte noting that getting AI into production remains difficult[¹]. This challenge stems from the unique dual nature of ML systems: they are both software services defined by API contracts and statistical models requiring frequent updates, each with distinct lifecycle requirements.

Getting this right extends far beyond technical elegance; it directly impacts customer experience, operational efficiency, and the ability to innovate rapidly in an increasingly AI-driven marketplace.

With 78% of organizations reporting AI use in 2024, up significantly from 55% the previous year, the need for effective versioning strategies has never been more critical.

The Scale and API Management Challenge

One major factor amplifying this complexity is the sheer proliferation of AI models. Platforms like Hugging Face now host over 1.4 million model repositories (as of April 2025), reflecting an explosion of options available for teams to test and deploy. This rapid expansion underscores the versioning challenge.

It also leads directly to a critical related insight: The more AI you deploy at scale, the more APIs you have. And that creates an API management problem. We're witnessing a "Cambrian explosion" of AI APIs, as organizations increasingly expose AI capabilities as modular services. Managing this API sprawl, including shadow APIs created outside governance, is crucial; failing to do so can thwart AI initiatives due to data issues, security risks, and integration failures. Furthermore, AI APIs present unique management needs around aspects like token-based rate limiting, prompt handling, and multi-model routing.

The Shared Responsibility Model Explained

The Shared Responsibility Model distinguishes between client-facing API contracts and underlying model implementations, creating a clear boundary of concerns that addresses the unique challenges of AI systems at scale.

API Management Layer Responsibilities:

Contract Stability: Maintains consistent interfaces that clients can rely on regardless of backend changes
Access Control & Security: Implements authentication, authorization, and data protection at the API boundary
Traffic Management: Handles rate limiting, throttling, and request distribution
Observability: Provides API-level metrics, logging, and monitoring
Documentation & Developer Experience: Offers consistent, up-to-date API specifications and documentation
Versioning Strategy: Manages API versions and deprecation policies

ML Serving Layer Responsibilities:

Model Lifecycle Management: Handles model training, validation, deployment, and retirement
Model-Specific Versioning: Manages rapid iteration of model weights, architectures, and parameters
Inferencing Infrastructure: Optimizes compute resources for model serving
Model Monitoring: Tracks performance, accuracy, and cost metrics
Canary Deployments: Implements gradual rollout of new model versions with automated rollback
Feature Management: Maintains repositories of model input variables (features) and ensures consistent transformations across training and serving

By clearly delineating these responsibilities, organizations can achieve a balance between stability and innovation. The API layer provides a consistent contract that shields consumers from the complexity and frequent changes occurring in the ML layer, while the ML layer retains the flexibility needed for rapid experimentation and improvement. This separation of concerns allows each layer to follow its own appropriate versioning cadence and practices.

Bridging MLOps and APIOps for End-to-End Scalability

Effectively managing the ML layer requires robust MLOps practices. However, since most AI/ML capabilities are consumed via APIs, focusing only on model lifecycle management isn't enough for true scalability. This is where APIOps becomes essential – applying DevOps principles to the entire API lifecycle (design, development, testing, deployment, monitoring, retirement)[²].

The relationship between these two operational domains is crucial:

MLOps focuses on model development, training, and deployment
APIOps manages how these models are exposed to consumers

These systems must work in harmony. Even the most efficient MLOps pipeline will create bottlenecks if your API management can't keep pace. When your data science team develops a new fraud detection model version, the API exposing that capability must be ready to handle the change without disrupting consumers.

As AI adoption accelerates, the number of AI-powered APIs within an organization multiplies rapidly. Without robust APIOps practices ensuring consistency and automation in API management, you won't realize the full agility benefits promised by your MLOps investments.

The Kubernetes-Native Advantage: Bridging API and Model Versioning

Kubernetes provides the ideal foundation for implementing the Shared Responsibility Model and coordinating MLOps and APIOps in practice. Its declarative nature, robust service discovery, and native resource definitions allow different specialized tools to manage their respective layers while integrating seamlessly. Here's how it bridges API and ML model versioning:

API Versioning (The Contract): An API management platform like Traefik Hub leverages Kubernetes Gateway API or Ingress resources to define and manage the stable API contracts exposed to consumers. It can handle different API versions concurrently – for example, routing requests based on the URL path (/v1/predict vs. /v2/predict), headers (API-Version: 1 vs. API-Version: 2), or more. Each API version defined in the gateway routes to a specific, stable Kubernetes Service name. This ensures clients always interact with a consistent API contract, regardless of the underlying model changes, aligning with standard API versioning techniques documented for platforms like Traefik Hub.

ML Model Versioning (The Implementation): Behind that stable Kubernetes Service endpoint managed by the API gateway, an ML serving platform like KServe manages the actual model deployments using its own Custom Resource Definitions (CRDs), such as the InferenceService. When a new ML model version needs to be rolled out (e.g., fraud-detection-v2.1 replacing v2.0), KServe can manage this transparently to the API consumer. Using its CRD, you can specify that only a small percentage of traffic (e.g., 10% via the canaryTrafficPercent field) should initially hit the new model version (v2.1), while the stable v2.0 handles the rest. KServe manages this traffic splitting internally through its integration with Kubernetes networking capabilities.

The AI Gateway Advantage: Between the API Management and ML Serving layers, a specialized AI Gateway addresses unique challenges of AI-specific workloads. Traefik's AI Gateway provides crucial capabilities that enhance security, performance, and governance—ranging from semantic caching that optimizes response times to content protection that ensures compliance. This middle layer bridges the gap between stable API contracts and dynamic model implementations, providing AI-specific optimizations that neither the API layer nor the ML layer can deliver alone.

Connecting Theory to Practice: This demonstrates the Shared Responsibility Model in action on Kubernetes. Traefik Hub handles the external API contract and its versions (/v1, /v2), directing traffic to the appropriate stable backend service endpoint. The AI Gateway provides critical AI-specific optimizations and protections, while KServe manages the ML model versions served by that endpoint, handling canary rollouts (v2.0 vs. v2.1) without disrupting the API contract defined by the Traefik Hub API Management layer. Kubernetes acts as the orchestration layer, enabling these specialized tools to manage their respective responsibilities cohesively.

Business Impact: Why This Matters

Getting this layered, Kubernetes-native approach right directly impacts the bottom line:

Faster Time-to-Value: By separating concerns and streamlining workflows across both MLOps and APIOps, businesses can significantly accelerate innovation. AI, managed effectively, can cut product development timelines by up to 50%, while MLOps practices can slash model deployment times by 30-50%[³], allowing faster response to market needs.

Enhanced Efficiency & Productivity: CEOs are already seeing tangible results, with 56% reporting efficiency gains from GenAI. Organizations effectively leveraging AI can achieve significantly higher productivity growth.

Improved Stability & Trust: The model allows underlying AI capabilities to be updated without breaking crucial client integrations or compromising backward compatibility. Robust management also helps mitigate risks associated with AI, such as inaccuracy and security vulnerabilities.

Future-Proofing Your AI Strategy

The convergence of API management and AI delivery is fundamentally reshaping how organizations innovate and compete. Successfully scaling AI requires more than just adopting algorithms; it demands a strategic approach to managing the interplay between APIs and models. Adopting a clear strategy like the Shared Responsibility Model is essential for navigating this complexity and unlocking the full potential of your AI investments.

This is where Traefik Hub, as a Kubernetes-native API management solution, plays a pivotal role. Built for the modern stack, it provides the sophisticated routing, security, governance, and visibility needed to expertly manage the crucial API management layer – facilitating the robust APIOps practices required to govern the "Cambrian explosion" of AI APIs within your Kubernetes environment.

The inclusion of specialized AI Gateway capabilities extends this value proposition further:

Semantic Caching: Unlock Scalable, Cost-Effective AI: When AI systems scale from experimentation to production, computational costs and latency can become major hurdles. Traditional caching methods often fall short, especially for resource-heavy large language models. Traefik's AI Gateway addresses these challenges with Semantic Caching, which uses advanced embedding techniques to understand the meaning behind queries, enabling intelligent reuse of results for similar requests.

This approach delivers tangible benefits:

Dramatically reduces response times from seconds to milliseconds for semantically similar queries
Eliminates redundant, costly computations for common request patterns
Optimizes resource allocation by serving cached responses for similar queries while dedicating computing power to truly novel requests
Integrates seamlessly with popular vector databases in cloud-native environments

Content Guard: Enabling Secure, Compliant AI: As AI powers increasingly critical business functions, ensuring proper governance becomes essential. Content Guard provides superior protection by safeguarding sensitive data and ensuring both inputs and outputs meet ethical and regulatory standards. Unlike basic solutions, it leverages contextual natural language processing for highly accurate detection across data types.

With over 35 predefined PII recognizers and customizable rules, Content Guard ensures:

Comprehensive detection and de-identification of sensitive information
Consistent policy enforcement across distributed AI deployments
Compliance with evolving regulatory requirements
A foundation for responsible AI that builds user trust

Together, these capabilities empower platform teams to provide stable, secure access while ML and application teams innovate rapidly behind the scenes using MLOps.

Taking Action: Next Steps for Your Organization

How is your organization preparing for the operational realities of scaling AI? Here are three concrete steps you can take today:

Assess Your Current State: Evaluate your existing API management and ML serving capabilities against the Shared Responsibility Model. Identify gaps in your versioning strategy that could impede AI scaling.
Build Cross-Functional Alignment: Bring together your API platform teams and ML engineering teams to establish clear boundaries of responsibility and collaborative workflows.
Implement a Pilot Project: Select a non-critical AI service and implement the Shared Responsibility Model using Kubernetes-native tools. Document learnings and establish patterns that can be scaled across your organization.

Ready to learn more? Contact the Traefik sales team for a personalized consultation on how the Shared Responsibility Model can be implemented in your organization using Traefik Hub.

Ultimately, embracing the Shared Responsibility Model, implemented through coordinated MLOps and APIOps practices, fosters better collaboration across diverse teams – engineering, operations, data science, and business – creating a scalable and resilient foundation for delivering AI-driven value. Investing in the right operational architecture today, with robust API management at its core, is investing in your organization's agility and innovation velocity for tomorrow.

References

[¹] Deloitte. "AI in Enterprise Applications: Challenges and Opportunities." https://www2.deloitte.com/us/en/insights/focus/signals-for-strategists/ai-in-enterprise-applications.html

[²] Microsoft Learn. "API Design Best Practices." https://learn.microsoft.com/en-us/azure/architecture/microservices/design/api-design

[³] Cogent Infotech. "MLOps Benefits: Deployment Time." https://www.cogentinfo.com/blog/mlops-benefits-deployment-time/

]]>

Traefik vs. #IngressNightmare: Security By Design in the Age of Critical Vulnerabilities

Sudeep Goswami — Thu, 10 Apr 2025 10:46:09 GMT

The Wake-Up Call Our Industry Needed

When our security team first alerted us about the critical vulnerabilities disclosed in ingress-nginx on March 24th, 2025, we immediately understood the gravity of the situation. CVE-2025-1974, with its devastating 9.8 CVSS rating, represents one of the most significant Kubernetes security events in recent years.

The implications are sobering: any entity within the pod network could potentially seize control of your entire Kubernetes cluster—without credentials or administrative privileges. Let that sink in for a moment.

While the ingress-nginx team deserves credit for rapidly releasing patches (v1.11.5 and v1.12.1), the discovery of more than 6,500 vulnerable clusters in production environments remains deeply concerning. This isn't just another vulnerability—it's a fundamental architectural flaw that demands our industry's immediate attention.

Understanding #IngressNightmare: Beyond the Headlines

To truly appreciate the severity of these vulnerabilities, we need to examine the technical mechanics at play. The core issue stems from how ingress-nginx processes ingress objects through its admission controller.

When validating an ingress object, the controller constructs an NGINX configuration and validates it using the nginx -t command. The Wiz Research team brilliantly uncovered multiple configuration injection vulnerabilities (CVE-2025-24514, CVE-2025-1097, CVE-2025-1098) that allow attackers to inject arbitrary NGINX configuration directives by sending malicious ingress objects to the admission controller.

While these injection points alone don't immediately grant code execution, the researchers discovered that the ssl_engine directive could be exploited to load shared libraries at any point in the configuration. By combining this with a method to upload a shared library to the pod's filesystem, attackers can achieve remote code execution within the ingress-nginx controller's pod.

Most critically, because admission controllers typically run with elevated privileges and unrestricted network access, successful exploitation allows attackers to execute arbitrary code and access all cluster secrets across namespaces—potentially leading to complete cluster compromise.

Why Traefik Remains Secure: Architectural Decisions Matter

As the founder and developer of Traefik, Emile Vauge made fundamental architectural decisions a decade ago that have proven critical for security. These weren't accidental choices—–they were deliberate design principles that we've maintained throughout Traefik's evolution:

No Raw Configuration Templating: Unlike templated proxies, Traefik parses configuration inputs (Ingress, IngressRoute, Middlewares, Custom Resources, or Gateway API resources) into strongly typed Go structs. Without template mechanisms, there's simply no string injection path for attackers to exploit.
Minimal Network Surface Area: Traefik doesn't implement an admission controller or any component with unrestricted network accessibility. It simply reads ingress resources and applies them if correct—if they can't be parsed correctly, they're ignored with minimal error logging. No configuration is ever executed or interpreted.
No Dynamic Library Loading: Traefik is written in Go, producing statically linked executables by default. We deliberately compile without CGO options, making it impossible to execute code not already part of the binary. Unlike NGINX's ssl_engine directive, there's no mechanism to load external shared libraries.

These architectural principles weren't arbitrary—they were security-first decisions that have stood the test of time.

The Industry's Crossroads: Why Wait Years for a Solution?

It's telling that the ingress-nginx team recently announced plans to pivot toward developing InGate—a new Ingress/Gateway API controller written in Go. They've wisely stated that no new features will be added to ingress-nginx during this transition, with the ultimate goal of deprecating it entirely once InGate reaches GA status.

While this represents a thoughtful long-term strategy, the stark reality is that reaching stability with a comparable feature set will likely take years. With only 18 commits to InGate at the time of writing, the journey has barely begun.

The Path Forward: Security Cannot Wait

For organizations running Kubernetes in production today, the message is clear: you shouldn't wait years to address a critical security vulnerability that exists now.

The immediate priority is upgrading existing ingress-nginx deployments to patched versions (v1.11.5 or v1.12.1). However, this is merely a tactical response to a strategic problem.

The more prudent approach is migrating to an ingress controller that's secure by design. At Traefik Labs, we've spent a decade building and refining Traefik Proxy with security as a foundational principle—not an afterthought. Our architectural decisions made ten years ago—choosing a statically linked toolchain, a strongly typed language, and eliminating templating—have proven prescient in light of today's threats.

Conclusion: Design Principles Matter

As leaders in the cloud-native ecosystem, we believe #IngressNightmare offers a valuable lesson for our industry. Security cannot be bolted on—it must be built in from the ground up. The choices we make about programming languages, compilation methods, and configuration approaches have profound security implications that may not become apparent for years.

We're encouraged to see the ingress-nginx team recognizing these principles in their plans for InGate. By embracing Go, static linking, and strong typing, they're acknowledging the same architectural foundations that have kept Traefik secure for the past decade.

For organizations that can't afford to wait for InGate to mature, Traefik Proxy offers a battle-tested, secure-by-design alternative available today. Your security shouldn't be compromised while waiting for tomorrow's solutions.

Sudeep Goswami is the CEO of Traefik Labs, leading the company's global strategy and operations.

Emile Vauge is the Founder and CTO of Traefik Labs. He created Traefik, the popular open-source cloud-native application proxy, which has been downloaded over 3 billion times.

]]>

Traefik Labs’ API Management Solution Now Available on Oracle Cloud Marketplace

Traefik Labs — Thu, 03 Apr 2025 13:01:42 GMT

Read the original post on Oracle: https://blogs.oracle.com/developers/post/traefik-labs-api-management-solution-now-available-on-oracle-cloud-marketplace

We’re excited to announce that Traefik Labs’ API management solution is now available on Oracle Cloud Infrastructure (OCI) Marketplace. This cloud-native, Kubernetes-native platform enables enterprises to manage APIs seamlessly across hybrid and multi-cloud environments while maintaining security and performance requirements of enterprise customers.

With Traefik’s deploy-anywhere flexibility and OCI’s industry-leading security, performance, and cost efficiency, customers gain a comprehensive API management solution. Deployment is quick and seamless—Traefik’s solution can be launched directly from the OCI Marketplace in just a few clicks.

Powerful OCI Integrations to drive business value for OCI customers

Managing APIs across distributed environments is often challenging due to manual configurations that can lead to security inconsistencies, limited visibility, and high costs. The Traefik-OCI integration addresses these issues by combining Traefik’s modern API management capabilities with OCI’s robust ecosystem, reducing complexity, enhancing security, and accelerating innovation.

Key OCI Integrations:

Oracle Cloud Infrastructure (OCI) Kubernetes Engine (OKE)

Traefik seamlessly integrates with OKE, providing a Kubernetes-native API management experience designed for containerized applications.

Deploy Kubernetes-native API management on OKE’s managed clusters
Eliminate API silos with Traefik CRDs and OKE’s multi-cluster management
Autoscale API gateways dynamically with OKE’s flexible node pool architecture

OCI DevOps

This integration enables GitOps-driven automation, ensuring seamless API deployment and management with OCI’s CI/CD pipeline.

Automate API management with end-to-end GitOps-driven CI/CD
Deploy and scale Traefik API Gateways via OCI build pipelines
Sync Git-committed API configurations automatically to production environments
Enhance OCI’s blue-green deployments with Traefik’s zero-downtime routing

OCI Application Performance Monitoring (APM)

Gain deep observability into API traffic and performance with Traefik’s OpenTelemetry support combined with OCI APM.

Gain full-stack observability with Traefik’s OpenTelemetry support integrated into OCI APM
Monitor GenAI traffic patterns alongside traditional API metrics
Troubleshoot faster with detailed OpenTelemetry instrumentation

OCI Logging Analytics

Ensure optimal API security and performance by leveraging Traefik’s structured logs with OCI’s advanced analytics tools.

Stream Traefik’s structured logs into OCI’s analytics platform
Apply OCI’s ML-powered analytics to optimize API performance and security
Gain unified insights across all environments with tool-agnostic log integration

Key Benefits for OCI Customers

End-to-End API Lifecycle Management: Includes a developer portal, interactive API documentation, testing, mocking, monetization, security, governance controls, and analytics
GitOps-Enabled Automation: Version-controlled API definitions ensure consistency, zero-downtime updates, and automated scaling
Seamless OCI Integrations: Works with OKE, OCI DevOps, APM, and Logging Analytics for a smooth developer experience and to maximize the value gained out of OCI platform
Easy Deployment via OCI Marketplace: Deploy Traefik’s solutions easily using Terraform or Helm templates

More Choice, More Flexibility

OCI is committed to providing enterprises with a broad ecosystem of innovative solutions. With Traefik Labs, OCI customers now have the freedom to choose the best API management approach suited for their needs:

Oracle API Gateway: A fully managed, cost-effective API gateway ideal for services built on OCI. Link
Traefik Labs: A deploy-anywhere API lifecycle management solution with deep OCI integrations. It also offers an easy upgrade path from a standard Ingress Controller to a full-featured API management solution. Link

How to Get Started

Browse the OCI Marketplace – Search for "Traefik" and select your preferred option.
Choose Your Deployment Method – Deploy using Terraform or Helm with pre-validated OCI integrations.
Configure Your Environment – Connect to OCI services like OKE, DevOps, APM, and Logging Analytics.
Learn More – Visit this link to request a demo or to explore further.

This joint solution provides organizations with a comprehensive platform for building and managing cloud-native applications and APIs. By combining OCI's enterprise-grade infrastructure with Traefik's advanced API management capabilities, organizations can accelerate their digital transformation while ensuring security, compliance, and operational excellence whether in OCI, other clouds, or on-premises environments.

]]>

AI Gateways: The Missing Piece in Scalable & Responsible AI Inferencing

Sudeep Goswami — Thu, 27 Mar 2025 11:42:57 GMT

As AI solutions evolve from experimental prototypes to enterprise-critical deployments, organizations face mounting challenges in scalability, performance, and responsible delivery. While standard AI gateways offer essential routing, load balancing, and API management, truly scalable and responsible AI inference demands two advanced enhancements: semantic caching—intelligently storing and reusing responses for similar prompts—and content guard that filters data shared with AI models as well as AI-generated content against safety and compliance standards.

Our exploration builds upon foundational gateway functionality to address the unique challenges of enterprise AI deployment, providing organizations with comprehensive solutions for both performance optimization and responsible content delivery—deployable anywhere from centralized data centers to global edge locations.

Why AI Gateways Form Essential Infrastructure

Organizations deploying AI at scale recognize the value of AI gateways as a unified infrastructure layer managing inference requests. Core gateways provide:

Intelligent Routing: Directing requests to appropriate models and endpoints
Load Balancing: Distributing traffic efficiently across infrastructure
Request Management: Handling timeouts, retries, and concurrency control
Observability: Monitoring performance and operational health
API Standardization: Ensuring consistent interfaces across models
Governance Controls: Enforcing organizational policies, access controls, and compliance requirements consistently across all AI interactions

While addressing fundamental challenges of infrastructure fragmentation and API inconsistency, AI deployments that scale to mission-critical status create additional challenges requiring specialized gateway enhancements: the computational overhead of redundant inference and the need for consistent content moderation.

An AI strategy remains incomplete without a robust gateway. Organizations lacking this critical infrastructure component build on fundamentally unstable foundations. Yet even with basic gateway functionality, enterprises still face significant challenges with performance economics and responsible scaling.

Semantic Caching: Unlocking Inference Scalability

Computational costs quickly become a limiting factor when AI systems move from experimentation to production. Traditional horizontal scaling proves economically unsustainable for AI inference, particularly for large language models with significant computational requirements.

Semantic caching emerges as the critical solution to scalability challenges. Unlike traditional caching requiring exact matches, semantic caching leverages advanced embedding techniques to identify the underlying meaning of queries, enabling reuse of previously computed results for semantically similar requests. Semantic caching dramatically transforms the economics of AI deployment:

Reduced Computational Redundancy: Identifying semantic similarity avoids repeating expensive computations for equivalent requests
Dramatic Latency Improvements: Cached responses resolve in milliseconds rather than seconds
Cost-Effective Scaling: Resources focus on novel prompts while common patterns leverage cached results

Application in Financial Services

In financial services, semantic caching delivers exceptional value for customer-facing applications like chatbots and advisory tools. When implemented within AI gateways, organizations can expect:

Significant reduction in inference costs through intelligent response reuse
Response times improving from seconds to milliseconds
Increased capacity to handle peak loads without proportional infrastructure scaling
Consistent performance during high-traffic events like product launches or market volatility

The impact multiplies in distributed edge deployments, allowing organizations to efficiently scale inference capacity without additional hardware costs.

Content Guard: Foundation for Responsible AI Delivery

While performance challenges merely impede AI adoption, governance concerns can terminate projects entirely. The need for governance becomes especially critical when organizations deploy generative AI in customer-facing and high-stakes environments where inappropriate handling of data or outputs creates significant reputational or compliance risks.

Content guard addresses governance concerns by establishing a sophisticated safety layer within AI gateways that protects sensitive information shared with models and evaluates generated content to ensure compliance with ethical guidelines, industry standards, and regulatory requirements. This bidirectional approach safeguards the entire AI interaction flow, from inputs to outputs, creating a robust governance framework for responsible AI deployment.

For organizations with distributed inferencing operations, content guard delivers consistent policy enforcement across every deployment location while adapting to local requirements when necessary.

Application in Healthcare

In healthcare environments, content guard provides critical safeguards for both clinical and patient-facing AI applications. When implemented within AI gateways, healthcare organizations can:

Enforce HIPAA compliance through automated PII detection and redaction
Apply specialized medical safety filters to prevent potentially harmful recommendations
Maintain distinct policy sets for different user interfaces (clinician vs. patient)
Provide comprehensive audit trails documenting all content validations
Reduce manual compliance reviews, accelerating application deployment while improving safety

By providing consistent, documentable enforcement of organizational policies regardless of where inference occurs, content guard transforms AI from a compliance risk into a compliance-enhancing asset for enterprises in highly regulated sectors.

Deployment Flexibility: From Core to Edge

AI gateways represent a logically centralized control plane that excels across diverse deployment scenarios. The lightweight, high-performance architecture enables organizations to maintain consistent policies, interfaces, and behaviors regardless of where AI inference occurs—from centralized data centers to thousands of edge locations.

Deployment flexibility becomes increasingly valuable as AI inference requirements diversify. Organizations now deploy AI gateways to:

Optimize centralized data center operations for cost-efficiency at scale
Support hybrid architectures combining on-premises and cloud resources
Expand AI services to edge locations for reduced latency and data sovereignty
Enable consistent management across heterogeneous environments

Managing these varied deployments demands a code-first approach. As highlighted in our previous blog on a holistic API architecture, the complexity of distributed AI infrastructure makes traditional manual management fundamentally unsustainable.

The code-first operating model transforms AI infrastructure deployment and management through:

Infrastructure as Code: Gateway configurations, routing rules, and policies defined in version-controlled files
Declarative Management: Explicit definition of desired states, eliminating configuration drift
Automated Consistency: Automatic propagation of changes across distributed instances
GitOps Workflows: Changes reviewed, tested, and deployed through established pipelines
Audit and Compliance: Complete history of infrastructure changes and policy updates

For AI gateways operating at the edge, this approach must accommodate additional requirements:

Lightweight Deployment: Efficient operation in resource-constrained edge environments
Stateful Operations: Maintaining critical functions like semantic caching with minimal overhead
Consistent Policies: Enforcing global standards with local adaptations where required
Resilient Operations: Continuing to function during network disruptions

The code-first model maintains consistency across this distributed edge environment while adapting quickly to evolving requirements.

Integrated Gateway Solutions: Enhancing Performance and Compliance Everywhere

The full potential of AI gateways emerges when semantic caching and content guard operate together within a unified framework managed through code. This integration creates an optimized workflow regardless of deployment location:

Optimized AI Inference Workflow Anywhere:

Request Processing: Incoming queries reach the gateway, whether in centralized data centers or edge locations
Intelligent Cache Utilization: Gateway evaluates semantic similarity against contextually appropriate cached queries
Efficient Response: For matches, retrieve cached responses and perform rapid content guard validation
Optimized Inferencing: For novel queries, perform inference on appropriately sized models locally or route as needed
Continuous Learning: Cache validated responses for future similar queries in that environment

Organizations can implement these AI functions across their entire infrastructure ecosystem—from massive centralized clusters to hundreds of distributed locations—creating a unified system that enables consistent management while maintaining operational flexibility.

The result is an integrated system delivering AI that is simultaneously faster, more cost-effective, more reliable, and demonstrably safer—regardless of where it operates in your infrastructure.

Conclusion: Building Future-Proof AI Infrastructure

Organizations that thrive in AI's rapid evolution won't necessarily possess the most advanced models, but rather the most thoughtful infrastructure to deploy them effectively and responsibly—wherever inferencing needs to occur.

AI gateways equipped with semantic caching and content guard, managed through a code-first approach, provide everything enterprises need to scale AI responsibly across any environment. The combined solution ensures high performance, reduced costs, streamlined workflows, and robust compliance—whether deployed in centralized data centers, distributed edge locations, or hybrid architectures spanning both.

Getting Started with Advanced AI Gateway Implementation

Ready to enhance your AI infrastructure? Here are specific next steps:

Assessment: Request our complimentary AI Gateway Readiness Assessment to identify your organization's specific needs
Pilot Implementation: Start with a focused pilot in a high-value use case to demonstrate ROI
Solution Consultation: Schedule a session with our technical team to discuss integration with your existing infrastructure
Strategic Roadmap: Develop a phased implementation plan tailored to your business priorities

Explore our comprehensive AI Gateway solution or contact our solution team to discuss how semantic caching and content guard can transform your enterprise AI deployment across your entire infrastructure.

]]>

Security Alert: How Attackers Can Bypass Next.js Middleware With a Single HTTP Header

José Carlos Chávez — Tue, 25 Mar 2025 15:11:21 GMT

The Critical Vulnerability Every Next.js Developer Should Know About

Did you know that a simple HTTP header manipulation could completely bypass your authentication and authorization systems in Next.js applications? Recently, a critical vulnerability (CVE-2025-29927) was disclosed in the popular Next.js framework, allowing attackers to circumvent middleware execution—including security checks—by leveraging an internal header.

Understanding Next.js Architecture and Why It Matters

According to Next.js documentation:

Middleware [in Next.js] allows you to run code before a request is completed. Then, based on the incoming request, you can modify the response by rewriting, redirecting, modifying the request or response headers, or responding directly.

In practical terms, middleware serves as the gatekeeper for your application, handling critical functions like:

Authentication & Authorization
Rate limiting
Request logging
Response manipulation
Routing

When middleware execution is bypassed, all of these security controls fail simultaneously.

How the Attack Works

As detailed in research by security researchers Allam Rachid (zhero) and Allam Yasser (inzo_), the attack exploits the x-middleware-subrequest header—an internal implementation detail that was never meant to be exposed to users.

The Exploit Method

By supplying a specific value to this header—easily inferred from the target application's structure—attackers can completely bypass Next.js middleware execution. For example, accessing a protected admin section at dashboard/panel/admin would simply require trying requests with headers like:

For modern versions (14.x and 15.x):

x-middleware-subrequest: middleware:middleware:middleware:middleware:middleware

For older versions (11.1.4 through 12.1.x):

x-middleware-subrequest: pages/_middleware
x-middleware-subrequest: pages/dashboard/_middleware
x-middleware-subrequest: pages/dashboard/panel/_middleware

You can find more details about the vulnerability here.

Real-World Impact

This vulnerability has severe implications:

Unauthorized access
Bypassing of authentication and authorization
Exposure of sensitive data
Circumvention of rate limiting and other protective measures
Potential access to internal APIs not meant for public consumption
Denial of Service through cache poisoning

The Response from Next.js

The Next.js team addressed this issue with a combination of patches and workarounds that vary by version:

Next.js 15.x: Fixed in version 15.2.3
Next.js 14.x: Fixed in version 14.2.25
Next.js 11.1.4 through 13.5.6: No direct patches available; the official recommendation is to implement a workaround, though 13.x is a maintained version.

Importantly, the official workaround recommendation from Next.js is to prevent the x-middleware-subrequest header from reaching your application - which can be implemented through various means, including WAFs, API gateways, reverse proxies, or load balancers.

This version-specific approach creates notable challenges for organizations:

Teams face trade-offs between performing emergency upgrades or rapidly deploying header-filtering solutions, both of which can strain resources.
Organizations managing multiple Next.js apps across different versions must tailor their mitigation approach for each one. Organizations with multiple Next.js applications across different versions need different strategies
Older versions lack direct patches, requiring teams to rely on network-level filtering—and if that’s not feasible, they may need to refactor sensitive parts of their codebase to mitigate the risk.
For legacy versions, the responsibility shifts from developers to operations and network teams, as code-based fixes aren't an option.
Non-managed deployments require self-managed defenses, increasing the overhead for teams hosting Next.js independently.

For many organizations, leveraging an API Gateway and a WAF—or similar network-level protection—is essential for ensuring consistent security across their entire application portfolio. Teams that already have these in place are in a stronger position to mitigate such attacks, reinforcing the importance of adopting the GW pattern as a proactive investment in security.

The Operational Advantage: API Gateway Layer Protection vs. Application Upgrades

One of the most compelling yet often overlooked benefits of protection through an API Gateway is the dramatic operational advantage it provides compared to codebase and tooling upgrades. Consider the contrasting approaches:

Upgrades approach:

Upgrades may require product downtime
Requires end to end testing of the product
May be blocked by release freezes or change management policies
Demands coordination across multiple teams (development, QA, operations)
Requires upgrading to Next.js 14.2.25+ or 15.2.3+ depending on your current version
May introduce compatibility issues with other dependencies and a cascade of other dependency updates

API Gateway protection approach:

Allows for rapid response to the vulnerability
Can be deployed independently of the application
Can be tested in DetectOnly mode before enforcing
Is typically reversible with minimal consequences
Targets only the specific vulnerability without touching the application code
Can be managed by operations teams without developer involvement
And most importantly, will bring security governance to your platform

The difference is stark for organizations managing dozens or hundreds of applications. Implementing API Gateway protection at the network edge can shield all vulnerable applications simultaneously—while application upgrades might require weeks of coordinated work across multiple teams and systems.

Patching remains essential for comprehensive security. However, the ability to deploy an immediate API Gateway-based solution creates a critical time buffer that allows for proper planning, testing, and deployment of application upgrades without leaving systems exposed in the interim. A good security strategy is the one that relies on multiple layers of protection.

Immediate Mitigation with Traefik and Coraza: Blocking the Vulnerable Header

If you're unable to upgrade to the fixed versions mentioned above, the official Next.js guidance is to block the x-middleware-subrequest header from reaching your application. Traefik offers a streamlined and less invasive way to implement this protection—regardless of whether you are pursuing a temporary workaround or a long-term mitigation strategy.

By leveraging Traefik with its Coraza WAF extension, teams can enforce this critical safeguard at the network edge, avoiding disruptive changes to application code or infrastructure. This approach allows for rapid, centralized protection that scales easily across multiple services and environments.

http:
  middlewares:
    waf:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - SecRule &REQUEST_HEADERS:x-middleware-subrequest "@gt 0" "id:1000001,phase:1,msg:'Request contains x-middleware-subrequest header',deny,status:403"

The above configuration blocks any request containing the vulnerable header, providing immediate protection while you plan your patching strategy.

Even without Coraza WAF, Traefik's native header manipulation capabilities can implement this protection using the Headers middleware:

http:
  middlewares:
    patch-nextjs-apps:
      headers:
        customRequestHeaders:
          x-middleware-subrequest: "" # Removes the header

The key requirement is blocking this header before it reaches your Next.js application, and Traefik provides multiple flexible approaches to accomplish this critical security task.

Broader Protection: Mitigating High-Profile Attacks with Traefik Coraza WAF

The power of Traefik's WAF extends far beyond mitigating specific vulnerabilities like the Next.js middleware bypass. It provides protection against many of the most common and devastating web application attacks, including:

SQL Injection (SQLi)

SQL injection attacks remain one of the OWASP Top 10 web application vulnerabilities year after year. These attacks can lead to:

Unauthorized access to sensitive database information
Data theft or manipulation
Complete system compromise

Traefik's Coraza WAF can detect and block SQLi attempts with rules like:

http:
  middlewares:
    xss-protection:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - Include @owasp_crs/REQUEST-942-APPLICATION-ATTACK-SQLI.conf

Cross-Site Scripting (XSS)

XSS attacks allow attackers to inject malicious client-side scripts into web pages viewed by other users. These attacks can:

Steal session cookies and user credentials
Deface websites
Redirect users to malicious sites
Perform actions on behalf of authenticated users

Traefik WAF can protect against XSS with rules like:

http:
  middlewares:
    xss-protection:
      plugin:
        coraza:
          directives:
            - Include @coraza.conf-recommended
            - SecRuleEngine On
            - Include @owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf
            - Include @owasp_crs/REQUEST-949-BLOCKING-EVALUATION.conf

Defense in Depth: Beyond Patching

This vulnerability reminds us of the importance of defense in depth. Even with the Next.js patch, consider implementing these additional protections:

Use layered protection mechanisms rather than relying solely on Next.js middleware
Follow the principle of least privilege for all application components
Implement an API gateway architecture for additional security layers
Integrate WAF capabilities with the API gateway architecture
Regularly audit your dependencies for known vulnerabilities

Traefik's flexibility allows it to act not just as a simple reverse proxy but as a comprehensive security gateway that can adapt to evolving threats without requiring application changes.

Real-World Impact

Several recent reports such as the 2024 Verizon Data Breach Investigations Report, Kaspersky Incident Response Report 2024, and 2024 Unit 42 Incident Response Report, have found that web application attacks continue to be the most common attack vector in confirmed breaches. Implementing a WAF like Traefik Coraza provides a crucial defense layer against these prevalent threats.

A single deployment of Traefik with properly configured WAF rules can protect entire application ecosystems from a wide range of attacks, often without requiring changes to the underlying applications themselves.

Conclusion

HTTP header-based attacks represent a sophisticated yet increasingly common attack vector. The Next.js vulnerability demonstrates how a single header manipulation can bypass critical security controls, but it's just one of many threats targeting modern web applications.

The integration of Traefik with its Coraza WAF extension provides a powerful first line of defense against not only specialized attacks like the Next.js middleware bypass, but also persistent, high-impact threats like SQL injection and XSS attacks that continue to plague applications across industries.

This operational approach—implementing Traefik protection rather than rushing application changes—represents a fundamental shift in vulnerability response strategy. It allows organizations to implement immediate protection while planning thoughtful, well-tested application updates on normal release cycles. This balance of security and operational stability is increasingly critical as application complexity and the pace of discovered vulnerabilities both continue to accelerate.

For development teams, this serves as a reminder to:

Keep frameworks and libraries updated
Monitor security advisories from your dependencies
Have a clear incident response plan for vulnerabilities
Prioritize operational resilience through non-invasive security controls
Consider gateway-level protections that can shield multiple applications simultaneously
Implement multiple layers of security, with an API Gateway as a critical component

By understanding these risks and implementing proper mitigations with Traefik, you can significantly reduce your attack surface and better protect your applications from the ever-evolving threat landscape while maintaining operational stability.

References

Official Next.js security advisory (GHSA-f82v-jwr5-mffw)
Next.js middleware documentation
CVE-2025-29927 details
OWASP Top 10 Web Application Security Risks
Verizon 2024 Data Breach Investigations Report
Kaspersky Incident Response Report 2024
2024 Unit 42 Incident Response Report
Traefik Coraza WAF Documentation
Next.js and the corrupt middleware: the authorizing artifact - zhero_web_security
Credits: Vulnerability discovered by Allam Rachid (zhero) and Allam Yasser (inzo_).

]]>

7 Critical API Protection Strategies to Fortify Your API Security

Immánuel Fodor — Thu, 20 Mar 2025 09:48:04 GMT

API security isn't just good practice – it's a necessity. As businesses increasingly rely on APIs to power everything from mobile applications to cloud services and IoT devices, these digital interfaces have become the new perimeter – and the new target for sophisticated attackers.

The question isn't if an attack will happen, but when. A recent analysis by Treblle (2024) reveals a startling reality: over half of API requests analyzed in 2023 used no encryption whatsoever, leaving sensitive data exposed to interception. Even more concerning, 52% had no authentication mechanisms in place – essentially leaving the digital front door wide open. Furthermore, 85% of APIs implemented no rate limiting, making them vulnerable to brute force attacks and service disruptions.

These vulnerabilities aren't just theoretical concerns. High-profile breaches like the one experienced by BeyondTrust in late 2023, where API key exposure led to significant organizational damage, demonstrate the real-world consequences of inadequate security. As APIs continue to proliferate – with organizations deploying hundreds or even thousands across their ecosystem – the attack surface expands dramatically.

Compounding the problem is the rise of "zombie" APIs – those APIs that remain accessible but unmaintained. Treblle's analysis found that a staggering 35% of endpoints fell into this category, creating significant blind spots in security postures. Without proper documentation, oversight, or regular updates, these forgotten interfaces represent prime targets for attackers seeking the path of least resistance.

Against this backdrop of escalating threats and costly consequences, organizations need a comprehensive, multilayered approach to API security. Let's explore seven critical protection strategies that can help secure your APIs against emerging threats.

#1: Encrypt Your API Traffic

TLS certificates are free – so use them!

While HTTPS might seem basic, the data shows a shocking reality: 55% of API requests analyzed didn't use any encryption (Treblle, 2024). This means sensitive data is being transmitted in plaintext, vulnerable to interception.

Even if the topic feels a little arbitrary to mention, we should prioritize first and foremost to encrypt the traffic leaving or reaching the gateway.

Protection levels:

Manual certificate management: Suitable for organizations that purchase extended validity certificates from authorities

Automated certificate management: Using solutions like Let's Encrypt with the ACME protocol

Self-hosted private key infrastructure: Managing your own PKI with ACME-compatible tools like HashiCorp Vault

Remember, even internal APIs deserve dataflow encryption. There is no excuse for internal APIs to be left unsecured. Leverage wildcard certificates to protect subdomains that are only resolvable through an internal DNS server.

#2: Enforce Strong Authentication and Authorization

The data from Treblle revealed that 52% of analyzed requests had no form of authentication in 2024. This amount represents a massive vulnerability surface. Even internal infrastructure isn't immune to threats. Recent breaches like the one experienced by BeyondTrust demonstrate that API keys can be compromised with devastating effects.

Unauthenticated API calls should be the exception, a deliberate choice that you don't put any kind of authentication in front of them. Apart from these choices, you should authenticate everything.

The best approach starts with an identity provider as the foundation layer for proper API governance, with related authentication methods like JWT, OIDC, or OAuth2 building on top of it.

#3: Protect APIs From Abuse and Overuse

Avoid service disruptions, protect performance, user trust & SLAs

Despite the fundamental role of rate limiting in API security, a surprising 85% of analyzed APIs had no rate limits in place, leaving them vulnerable to denial-of-service attacks and excessive usage (Treblle, 2024). This widespread oversight exposes systems to severe disruptions, impacting both operational stability and user experience.

There are several layers of protection available to prevent such vulnerabilities. To name a few:

IP Address Filtering: This method restricts access to specific IP ranges, ensuring that only trusted sources can make requests.

Parallel Request Limitation: By controlling the number of concurrent requests from a single client, systems can prevent overloads and maintain optimal performance. Especially useful to fight web crawlers or DoS attacks.

Request Flow Control: Implementing true rate limiting with token bucket algorithms helps manage request rates effectively, thereby preventing abuse.
Distributed Rate Limiting: This approach shares rate limit counters across gateway replicas, ensuring consistent enforcement of limits even in distributed architectures. Every replica sees the same counters, enhancing security through coordinated rate limiting.

By leveraging these layered protections, organizations can significantly level up their security game, reducing the risk of service disruptions and safeguarding user trust.

#4: Block Known Vulnerabilities

Configure a Web Application Firewall (WAF)

A Web Application Firewall (WAF) is crucial for defending against known attack patterns, including SQL injection and cross-site scripting. Beyond protecting sensitive data, a WAF is also essential for maintaining regulatory compliance, such as PCI DSS requirements.

Traefik Labs utilizes an open-source core for its firewall, which has OWASP endorsement, ensuring industry-standard protection against the latest threats. Deploying a WAF is relatively straightforward, often requiring just a few lines of configuration, making it an accessible yet powerful mechanism for enhancing security.

#5: Apply Zero-Trust for Accessing APIs

Avoid data leaks with the least privilege principle

Rather than providing unrestricted access to entire APs, a zero-trust approach restricts access to specific endpoints and methods, enhancing security by enforcing least-privilege access, significantly reducing APIs' attack surface.

By defining granular access rules, you can control exactly what clients can access, even as your APIs evolve. This ensures that newly deployed endpoints are not automatically exposed, minimizing the risk of unauthorized access.

#6: Centralize Secret and Key Management

Protect API Keys for AI/LLM APIs

With the growing reliance on external services, particularly AI and LLM services that require API keys for access, centralizing API key management has become crucial. These keys are powerful – if compromised, they can allow unauthorized users to exhaust tokens, deplete account credits, or even drain linked bank accounts.

The most effective solution is to store credentials centrally and issue short-lived tokens to end users. These tokens can be easily revoked, minimizing the impact of compromised keys. By implementing this gateway pattern, organizations can prevent the direct exposure of valuable API keys, ensuring better security and control.

Centralized API key management not only safeguards sensitive data but also streamlines the administration of access credentials.

#7: Avoid Zombie and Shadow APIs

Only expose what’s documented in your OpenAPI specification

One of the most alarming findings showed that 35% of API endpoints were "zombie" endpoints – accessible but unmaintained and potentially vulnerable (Treblle, 2024).

These unmaintained APIs pose more and more threats to organizations these days. Use OpenAPI specifications to define legitimate endpoints and block everything else. Even for legacy systems without documentation, observability tools can help build specifications from observed traffic. You can block undocumented endpoints with flipping one single configuration value.

Bonus Tip: Implement Comprehensive Observability and Reduce Downtime Dramatically

While not directly a security control, observability provides the visibility necessary to identify potential threats and respond quickly.

Without this, you will not see if request rates spike up, if error rates spike up, or if an IP address bombards your infrastructure. By implementing OpenTelemetry for metrics, tracing, and logs, organizations can identify potential security incidents and correlate them with changes to systems. This visibility is essential for effective incident response.

Taking a Multilayered Approach

No single security measure is sufficient on its own. To effectively safeguard APIs, organizations should adopt a multilayered security strategy, integrating multiple protective mechanisms. Each layer – be it rate limiting, WAF, zero-trust, centralized key management, or observability – provides unique benefits that collectively enhance the security posture.

These measures are designed to work together, reinforcing each other to create a robust defense system. By implementing this comprehensive approach to API security, organizations can significantly reduce their risk profile, protecting their data and systems from emerging threats.

The multilayered strategy ensures a resilient and secure API environment, safeguarding business operations and maintaining user trust.

This article was contributed to DevOps.com by Traefik Labs for KubeCon London 2025.

Ready to see how Traefik Hub can elevate your security strategy? Request a demo today and experience the benefits of a modern, high-performance API Gateway and Management solution.

]]>

Beyond the AI Gateway: Why a Holistic API Architecture and Code-First Operating Model Are Essential

Sudeep Goswami — Wed, 19 Mar 2025 12:41:51 GMT

In our previous exploration of enterprise AI architecture, we examined how an AI Gateway serves as a critical component for managing AI operations.

Today, we're diving deeper into why an AI Gateway alone isn't sufficient—organizations need a comprehensive API architecture supported by modern "API Management as Code" practices to handle the explosive growth of AI inference endpoints.

AI Inference Everywhere: The Edge Computing Revolution

The AI landscape is experiencing a fundamental shift as inference workloads—the deployment of trained models to make real-time predictions—expand from centralized cloud environments to the network edge and everywhere in between. This expansion represents a strategic imperative for organizations seeking competitive advantage through faster, more efficient AI deployments.

Global spending on edge computing reached $228 billion in 2024, marking a 14% increase from 2023, according to IDC's Worldwide Edge Spending Guide.[1] Looking forward, IDC forecasts that edge spending will reach $378 billion by 2028, demonstrating a clear expansion of AI workloads across the entire computing spectrum.

This edge-ward expansion is driven by necessity. Millisecond response times—essential for latency-sensitive applications like computer vision, autonomous control systems, and voice interfaces—fundamentally cannot be met by cloud-based inference. As IDC's Dave McCarthy notes, edge computing is "crucial for reducing latency,"[2] enabling the split-second decisions required in modern systems. Consider a manufacturing robot analyzing visual input: waiting for data to travel to distant servers and back creates an unacceptable bottleneck for operations that demand immediate responses.

As we progress through 2025, this trend continues to accelerate as organizations discover that edge AI isn't merely about technical performance—it's about creating entirely new capabilities that weren't previously possible.

The Cambrian Explosion of AI APIs

As organizations operationalize AI models, they're increasingly exposing these capabilities via APIs. Modern software architecture trends have accelerated this shift toward microservices and AI-as-a-Service architectures, where machine learning models are deployed as independent API endpoints rather than being embedded directly into applications. This architectural approach enables organizations to build, deploy, and manage AI inference capabilities as modular, reusable services that can be consumed on-demand by multiple applications.

In practice, this means that an employee application or customer-facing system might call an internal API endpoint for an ML model (or an external API like OpenAI) to get a prediction or generate text, instead of containing the ML model code itself. This decoupling creates significant benefits: improved scalability, easier maintenance, and more efficient resource utilization. However, it also results in what industry experts have aptly termed a "Cambrian explosion" of API endpoints.

According to Forrester's 2024 API Management Wave Report, "As AI agents become mainstream, APIs will be the primary means of AI-driven commerce and agent-to-agent communication."[3]

Major vendors are rapidly expanding their offerings to make AI inference more accessible across public, private, hybrid cloud environments and increasingly at the edge.

This democratization of inference capabilities is driving an unprecedented surge in inference endpoints that must be managed, secured, and governed.

The Salt Security State of API report 2025 provides compelling evidence of this growth, with 30% of organizations reporting 51-100% growth in APIs over the past year, and 25% experiencing growth exceeding 100%. Currently, 13% of organizations manage over 1,000 APIs, with 53% of those being large organizations with more than 10,000 employees.[4]

This expansion is further fueled by the rise of agentic AI. According to Deloitte's State of GenAI Q4 2024, agentic AI—which autonomously orchestrates workflows and tasks—is gaining significant traction.[5] These autonomous agents depend heavily on APIs to interact seamlessly with various enterprise tools, databases, and third-party services, driving further API demand.

The Management Crisis: When APIs Multiply Faster Than Governance

This rapid proliferation of AI inference endpoints has pushed traditional API management approaches to their breaking point. Organizations that previously managed dozens of APIs are now handling hundreds, and those with hundreds are rapidly approaching the thousands mark. These APIs are typically developed by different teams for different purposes, all running in production with varying levels of oversight. This "API sprawl" creates a governance nightmare where management practices struggle to keep pace with innovation.

Security concerns top the list of challenges. Salt Security's 2025 State of API Security Report reveals alarming statistics: 58% of organizations monitor their APIs less than daily, and only 20% continuously monitor their APIs in real-time. Only 15% are very confident in their API inventory accuracy, while 34% admitted facing security challenges regarding lack of visibility into sensitive data exposure.[4]

The consequences are severe: 99% of organizations have encountered API issues in the past year, and 55% have slowed application rollouts due to API security concerns. Most organizations remain unprepared, with only 10% having an API posture governance strategy in place and 59% still in the planning or basic stages of their API security strategies. Only 6% report having advanced security programs, while 8% have non-existent API security strategies.[4]

Beyond security, operational complexity increases exponentially with API volume. When dozens of microservices each expose multiple AI model endpoints, orchestrating these moving parts becomes nearly impossible with manual approaches. The microservices architecture that makes AI deployment more flexible also creates a substantial governance challenge, as each AI model exposed as an API requires its own monitoring, access controls, rate limiting, and lifecycle management.

Versioning presents another critical challenge amplified by AI workloads. AI models are frequently updated as they're retrained on fresh data or improved with new architectures. These changes must often be reflected in the API interface, creating a versioning challenge that grows with each new endpoint.

Why an AI Gateway Alone Is Not Enough

While an AI Gateway provides essential functionality for managing AI interactions—including unified access, security enforcement, and operational efficiency—it represents just one piece of a larger API management puzzle. The AI Gateway excels at handling AI-specific concerns but needs to be part of a holistic architecture that addresses the entire API lifecycle.

An AI Gateway without comprehensive API management is like having a sophisticated security system for your front door while leaving windows and side entrances unprotected.

It creates a false sense of security and control while leaving critical vulnerabilities unaddressed.

For example, an AI Gateway might effectively manage authentication and rate limiting for AI model access, but without proper versioning, documentation, and lifecycle management across all APIs, organizations still face significant challenges as their AI implementations scale. Governance frameworks often lag behind technical implementations, creating gaps where policies aren't consistently applied or enforced.

Furthermore, as organizations deploy AI inference capabilities everywhere—from cloud to edge and all points in between—they need management solutions that span this distributed architecture—providing consistent controls regardless of where models are deployed or how they're accessed.

The Limitations of Traditional API Management

Established API management platforms provide essential capabilities for controlling and streamlining API usage. These platforms offer API gateways for routing and policy enforcement, authentication integration, rate limiting, monitoring, and developer portals.

However, while these tools have served well in traditional environments, they're showing limitations in the face of AI-driven API proliferation. The sheer quantity of APIs generated by AI initiatives strains manual configuration approaches. Traditional API management often relied on web-based consoles where administrators would register and configure dozens of APIs—now they face hundreds of model endpoints, making click-through management impractical.

Governance consistency presents another challenge. Even with robust API management solutions, ensuring that every new AI endpoint receives appropriate policies relies on process discipline. In fast-moving AI development cycles, teams might bypass governance for quick experiments, creating gaps where not all inference APIs are properly managed.

Performance considerations also influence management decisions. AI inference calls can be latency-sensitive, and adding an API gateway hop introduces overhead. For high-throughput or real-time inference scenarios, teams might be tempted to bypass management layers for speed, creating shadow APIs outside governance frameworks.

As we look toward the rest of 2025 and beyond, it's clear that existing API management approaches remain essential but insufficient alone for the AI-driven API ecosystem. The solution lies in creating a holistic architecture that integrates the AI Gateway with proven API management best practices, implemented through modern code-first approaches.

API Management as Code: The Path Forward

"API Management as Code" represents a paradigm shift in how organizations handle their growing API estates. Rather than configuring API gateways and management settings through web interfaces, this approach defines all API definitions, routing rules, policies, and access controls declaratively in version-controlled files. Changes to APIs occur through code modifications deployed via automated pipelines, not through manual console interactions.

This shift mirrors the evolution we've seen in infrastructure management, where Infrastructure as Code (IaC) has become the standard approach for large-scale deployments. For AI-rich environments, the benefits are particularly compelling:

First, code-driven API management enables scale through automation. With APIs defined as code, organizations can script and templatize the creation of new AI endpoints. When a data science team develops a new model, a CI/CD pipeline can automatically generate the API configuration from a template and deploy it to the gateway. As Microsoft's APIOps best practices note, treating API configurations as code helps teams deploy changes iteratively and handle complexity at scale.[6]

Second, treating API configurations as code brings version control and collaboration benefits. API definitions in Git repositories allow teams to review changes via pull requests, catch potential issues before deployment, and maintain a complete history of API evolution. This approach enables rollbacks when needed and fosters standardization through centralized policy templates that every API must follow before deployment.

Third, code-driven management facilitates multi-environment coordination. Large enterprises typically maintain multiple API gateways across development, testing, and production environments. Managing these through code means deploying consistent configurations everywhere from a single source of truth.

Fourth, API mocking capabilities combined with code-driven management can reduce costs during development and testing. When working with expensive AI services, teams can use mock endpoints to simulate responses instead of calling paid APIs for every test. This approach can significantly reduce development costs while accelerating innovation.

As organizations deploy more AI inference endpoints in 2025 and beyond, API Management as Code will move from advantageous to essential—providing the only viable path to maintain control, security, and agility at scale.

A Unified API Architecture: Bringing It All Together

A truly effective approach combines an AI Gateway with a comprehensive API architecture and code-first management practices. This holistic strategy includes:

An AI Gateway that serves as the specialized entry point for AI workloads, providing model-specific optimizations, security controls, and unified access.
Proven API Management best practices implemented through modern, code-driven approaches. These include complete API lifecycle management (design, documentation, versioning, retirement) reimagined for the scale and complexity of AI workloads.
Centralized identity and access management through an IdP, providing a foundational layer to know your users, govern their access, and enforce consistent security policies across all API endpoints.
API Governance that enforces standards, security policies, and compliance requirements across all endpoints.
API Mocking to facilitate development and testing without disrupting production environments or incurring unnecessary costs when working with expensive AI services.
API Management as Code practices that automate deployment and configuration of all these components.

This unified architecture addresses the complete spectrum of challenges organizations face when scaling AI initiatives. The AI Gateway handles the unique aspects of AI workloads, while the broader API management framework ensures consistency, governance, and developer experience across all digital interfaces.

By implementing this holistic approach, organizations can maintain control over their growing API landscape while enabling the agility needed for rapid AI innovation. Teams can deploy new models quickly and securely, knowing that appropriate controls are automatically applied through code-driven processes.

The Time to Act Is Now: How to Get Started

The AI inference explosion isn't a distant future scenario—it's happening now, and the velocity is only increasing. Organizations that delay implementing a holistic API architecture with code-first management practices aren't just missing an opportunity; they're creating existential business risk.

According to Deloitte's "State of Generative AI in the Enterprise – Q4 2024" survey, organizations are experiencing a strategic shift towards using AI for competitive differentiation, with GenAI becoming integral to core business processes.[5] As AI moves deeper into business-critical functions, companies will increasingly rely on APIs to integrate and operationalize these AI models across applications and platforms.

The survey also reveals mature adoption of GenAI within IT, cybersecurity, operations, marketing, and customer service, indicating widespread integration into existing workflows and software. Such integration inevitably increases the reliance on APIs to facilitate interactions between AI services and core enterprise systems.

Higher-than-expected ROI from advanced GenAI initiatives, particularly in cybersecurity, is motivating organizations to scale deployments. This scaling naturally increases API utilization as enterprises integrate these solutions across more users and processes.

Implementing a modern, holistic API architecture doesn't require a complete overhaul of existing systems. Organizations can begin their journey immediately with these steps:

Assess your current API landscape and identify gaps in governance, security, and scalability
Adopt API Management as Code practices for new AI initiatives first, then expand
Integrate your AI Gateway with comprehensive API management capabilities
Implement a consistent governance framework that spans all APIs, not just AI endpoints
Establish centralized identity and access management as a foundation for API security
Embrace automation and code-driven approaches throughout the API lifecycle

The question isn't whether you need a holistic API architecture—it's how quickly you can implement one before the incoming wave of AI inference APIs overwhelms your current capabilities. The organizations that act decisively now will turn API proliferation from a potential crisis into a strategic advantage, creating the foundation for sustainable AI innovation that delivers real business value while managing risk effectively.

Don't be caught unprepared for the AI inference tsunami. The time to build your unified API architecture is now—before the wave hits.

Footnotes

[1]: "Worldwide Edge Spending Guide," IDC Research, 2024.

[2]: "Global edge computing spending to reach $228 billion in 2024," Back End News.

[3]: "The Forrester Wave™: API Management Solutions, Q4 2024," Forrester Research, October 2024.

[4]: "Salt Security State of API report 2025," Salt Security, 2025.

[5]: "State of Generative AI in the Enterprise – Q4 2024," Deloitte Research, 2024.

[6]: "Automated API deployments using APIOps," Microsoft Azure Architecture Center.

]]>

From Ingress to API Intelligence: Unlocking Traefik v3.x’s Full Potential in NKP v2.14

Sudeep Goswami, Michael Matur — Thu, 06 Mar 2025 14:08:03 GMT

The integration of Traefik v3.x in Nutanix Kubernetes Platform (NKP) v2.14 marks a pivotal advancement in enterprise API capabilities. Far more than just an ingress controller, Traefik v3.x establishes a progressive pathway for organizations to evolve their API strategy—from fundamental application routing to sophisticated API Gateway functionality, AI Gateway services, and comprehensive API lifecycle management. This modular approach allows enterprises to advance at their own pace, adopting capabilities sequentially as their needs mature, without disruptive migrations or platform changes. By meeting organizations exactly where they are on their API journey, Traefik v3.x transforms NKP v2.14 from a container orchestration platform into a complete API innovation ecosystem.

Phase 1: Application Proxy - Building the Foundation

When you need it:

You're deploying microservices on NKP and need reliable traffic routing
Multiple applications require consistent load balancing
Service discovery is becoming manual and cumbersome
Team resources are limited for managing infrastructure complexity

How it works: Traefik v3.x in NKP v2.14 provides out-of-the-box service discovery that automatically detects new services as they're deployed. A retail company migrating from monolithic applications to microservices can leverage Traefik's dynamic configuration to route traffic between customer-facing web applications, inventory systems, and order processing services without manual intervention. When holiday shopping traffic increases, Traefik's load balancing capabilities ensure resources are distributed optimally across services, preventing any single component from becoming a bottleneck.

Key benefits:

Zero-configuration service discovery reduces operational overhead
Real-time traffic routing adjustments based on service health and availability
Simplified configuration through Kubernetes CRDs and annotations
Centralized entry point for all microservices traffic management

Phase 2: API Gateway - Securing and Controlling Access

When you need it:

Security requirements demand centralized authentication and authorization
You need to manage access to APIs across multiple teams or departments
API traffic requires special handling (rate limiting, circuit breaking)
AI workloads need specialized routing and management
Enterprise integration requirements are increasing

How it works: As your API ecosystem grows within NKP, Traefik v3.x transforms into a full-fledged API Gateway. A financial services company can implement OAuth2 authentication for all customer-facing APIs while using different authentication mechanisms for internal services. When they introduce a new AI-powered fraud detection system, Traefik's AI Gateway capabilities allow them to route specific transactions to this service based on risk profiles, while implementing circuit breaking to prevent cascading failures if the AI service experiences high load. Enterprise integrations with existing identity providers and security systems ensure compliance with financial regulations without custom code.

Key benefits:

Centralized security policies for all API traffic
Specialized handling for AI endpoints with appropriate rate limiting and prioritization
Advanced traffic shaping capabilities (circuit breaking, retries, timeouts)
Integration with enterprise security infrastructure
Single control plane for all API traffic

Phase 3: API Management - Full Lifecycle Runtime Governance

When you need it:

APIs have become strategic business assets requiring governance
Developers need self-service portals to discover and consume APIs
Operations teams need comprehensive observability across the API landscape
API versioning and lifecycle management become critical
Business stakeholders need insights into API usage and performance

How it works: At the most mature stage, Traefik v3.x within NKP v2.14 evolves into a complete API Management solution. A healthcare technology company can implement API governance policies ensuring patient data APIs adhere to HIPAA compliance requirements. Their developer portal enables partner companies to discover available APIs, test them in sandbox environments, and generate client libraries. The API observability dashboard gives operations teams real-time insights into performance bottlenecks, while business analysts track API usage to determine which features drive the most value. When new AI-powered diagnostic services are introduced, the existing API lifecycle management ensures proper versioning, documentation, and controlled rollout.

Key benefits:

Comprehensive API governance with policy enforcement
Developer self-service through customizable portals
End-to-end API observability for operations and business stakeholders
Controlled API versioning and deprecation workflows
Unified management across traditional and AI-enabled services

By leveraging these progressive capabilities of Traefik v3.x within NKP v2.14, organizations can evolve their API strategy at their own pace while maintaining a consistent platform and operational model. Whether you're just beginning your microservices journey or managing a sophisticated API ecosystem with AI components, Traefik provides the tools needed at each stage of maturity.

Putting Theory into Practice: Upgrading Your NKP v2.14 Traefik Deployment

Now that you understand the value progression from Application Proxy to API Gateway to full API Management within Traefik v3.x, you might be wondering:

"How do I actually implement these advanced capabilities in my NKP v2.14 environment?"

The good news is that Nutanix and Traefik Labs have designed a straightforward upgrade path that preserves your existing configurations while unlocking these powerful features. The upgrade process is deliberate yet simple, allowing you to evolve your API infrastructure without disruption to your current services.

Let's walk through the specific steps required to transition from the default Traefik OSS version in NKP v2.14 to the full enterprise capabilities:

Create the Traefik Hub Gateway Token: contact the Traefik Sales Team for access, and then login to the Traefik Hub Dashboard https://hub.traefik.io

Create a New Catalog Entry in NKP

kubectl apply -k https://github.com/traefik/nkp-traefiklabs-catalog/hub
TRAEFIK_HUB_TOKEN=XXXX {where XXXX represents the actual token from the previous step}
kubectl create secret generic license --namespace kommander --from-literal=token=$TRAEFIK_HUB_TOKEN

Enable the Traefik Hub Catalog Entry in NKP

Next, enable the Traefik Hub catalog entry to complete the upgrade process. This step integrates Traefik Hub within your NKP cluster, providing advanced API management capabilities. Note: this step could take anywhere from 1 to 5 minutes

Finally, log back in to Traefik Hub, and verify that the upgrade is now complete and your Traefik Hub API Gateway is online.

This transition bridges the conceptual journey we've discussed with concrete implementation steps, allowing you to immediately begin realizing the benefits of advanced API capabilities in your Nutanix Kubernetes environment.

The API-Powered Future Begins with Traefik and NKP

The integration of Traefik v3.x in NKP v2.14 represents more than just a technical upgrade—it's a strategic enabler for organizations looking to harness the full potential of their API ecosystem. By providing a progressive path from basic application routing to sophisticated API management and AI gateway capabilities, Traefik empowers enterprises to evolve at their own pace while maintaining operational consistency.

As we've explored throughout this post, each phase of the Traefik journey addresses specific organizational needs and challenges:

Application Proxy capabilities solve immediate routing and discovery challenges
API and AI Gateway features enhance security and control in complex environments
Full API Management delivers the governance and visibility needed for strategic API and AI programs

With the straightforward upgrade path available in NKP v2.14, organizations can now seamlessly access these capabilities without disruptive changes to their existing infrastructure.

Take the Next Step in Your API Journey

Ready to unlock the full potential of Traefik v3.x in your NKP environment? Here's how to get started:

Assess your current API maturity and identify which Traefik capabilities would deliver the most immediate value
Explore the five essential qualities of a modern API gateway to ensure your infrastructure is future-proof
Implement runtime API governance best practices to enhance security, compliance, and observability
Plan a seamless API gateway migration with a step-by-step guide that you can apply to your specific use case
Schedule a demonstration with our team to see Traefik's advanced features in action within an NKP environment

Don't let your API infrastructure limit your innovation potential. With Traefik v3.x and NKP v2.14, you have all the tools needed to build a scalable, secure, and manageable API ecosystem that can grow with your business—from your first microservice to your most sophisticated AI-powered application.

]]>

How to Size Your Traefik Hub API Gateway Instances

Nicolas Mengin — Tue, 04 Mar 2025 14:14:01 GMT

When onboarding new customers, one common question we face is about the optimal resource configuration to set for a Traefik Hub API Gateway instance. Unfortunately, there is no one-size-fits-all answer to this question. Several factors need consideration, such as the infrastructure type, traffic volume, and how Traefik Hub API Gateway will be utilized.

Let’s explore why these factors are crucial and how to customize the Traefik Hub API Gateway deployment accordingly.

Define the size

From the tests we have done internally, we know a single Traefik Hub API Gateway instance (installed on 8 CPUs and 16 GB of RAM machine) is capable of routing over 72,000 HTTP Requests per Second (RPS) (and more with bigger configurations) if there is no security or operations to do on the requests. With the introduction of the experimental fastProxy option, performance can even reach up to 102,000 RPS with the same configuration. This suggests that when provided you allocate a sufficiently powerful machine, RPS should not be your primary concern in production—at least in theory.

In practice, the maximum RPS depends heavily on the type of traffic being managed. For example, if your Traefik Hub API Gateway instance is handling TLS connections or utilizing middleware (such as header management or rate limiting), the RPS capacity will decrease.

Additionally, the more you want Traefik Hub API Gateway to handle high RPS values, the larger, and the more expensive, the machines are. While these machines offer high performance, they come with high costs and a poor return on investment (ROI) since you likely won't use their full capacity most of the time—but you'll pay for them regardless.

On top of that, a key best practice is to favor scaling out (horizontally adding more smaller instances) over scaling up (vertically increasing the resources of a single instance). This approach not only makes your infrastructure more scalable but also more resilient: you reduce the risk of a Single Point of Failure (SPOF) and ensure that your system can handle traffic spikes more effectively.

For these reasons, we recommend deploying multiple smaller instances rather than a few powerful ones. The exact number and size of instances depend on your infrastructure setup as we will see below.

Define the number

When Traefik Hub API Gateway is deployed within an orchestrator (like Kubernetes), sizing becomes more straightforward. We recommend deploying enough instances (each with 2 vCPUs, and 4GB RAM) to handle up to 3,000 RPS per instance under normal traffic conditions.

The key factor here is observability: in such an environment, monitoring your containers or pods and ensuring they scale up when resource consumption reaches 70% of CPU or RAM capacity, and scaling down when traffic returns to normal.

However, despite Traefik Hub API Gateway being a cloud-native Ingress Controller primarily designed for container environments, it performs admirably on traditional bare-metal infrastructure as well.

In these environments, the challenge is finding the right balance between avoiding a Single Point of Failure (SPOF) and minimizing over-provisioning of instances. Deploying multiple small VMs with limited CPU and memory resources can handle normal traffic loads, but this approach may fall short during sudden traffic spikes if the combined RPS capacity is insufficient.

On the other hand, using a few larger VMs and scaling them up is both time-consuming and operationally intensive compared to simply adding replicas in a Kubernetes deployment. This makes it crucial to anticipate traffic peaks, ensuring each machine is capable of handling unexpected spikes and that scaling occurs promptly.

For bare-metal environments, we recommend deploying medium-sized VMs that can reliably handle around 50% of peak traffic, providing sufficient capacity while maintaining resilience.

Real-Life Example: Traefik Hub API Gateway in Kubernetes

In today’s landscape, securing API access is non-negotiable. Therefore, it's important to configure the Traefik Hub API Gateway in a way that accounts for the additional latency introduced by the TLS handshake and JWT authentication verification.

The use of the header middleware is also common, as it allows you to pass extra information to your backend services. However, adding headers introduces additional latency, which is why we include it in this test.

Finally, gathering metrics for observability is crucial for any production platform. Exposing traffic metrics through OpenTelemetry allows you to monitor and gain insights into system performance, but it too has an impact on Traefik Hub API Gateway’s overall performance.

Let's now apply the sizing recommendations we discussed earlier to two typical use cases with different requirements:

Up to 1,000 RPS per Instance

This scenario represents a common need for exposing APIs. Here, the traffic volume is moderate, with each Traefik Hub API Gateway instance handling up to 1000 RPS.

Instance Sizing: For this scenario, we recommend deploying several small instances (e.g., 1-2 vCPUs, and 1-2 GB RAM). Traefik Hub API Gateway can scale horizontally, allowing you to start with a modest setup and add instances as traffic grows.
Middleware Impact: Even though the traffic load is internal, enabling TLS for secure communication and using JWT Authentication middleware (as well as other middleware or operations on access logs) will introduce some overhead. Injecting headers also adds to the processing time, but at this scale, the latency impact should remain manageable.
Scaling Strategy: Ensure autoscaling triggers when resource usage (CPU, memory) hits 80%, and scale down once traffic normalizes. A few well-sized instances should be enough to handle this load efficiently while maintaining cost-effectiveness.
Recommendation: Start with 2-5 replicas to ensure resiliency, and monitor performance to adjust as needed.

1,000 to 5,000 RPS per Instance

This scenario is geared toward more demanding use cases, such as exposing APIs to external partners, customers, or commercial websites. With traffic ranging from 1000 to 5000 RPS per instance, the need for scalability and performance is more critical.

Instance Sizing: For this higher traffic volume, each Traefik Hub API Gateway instance should have more resources—typically around 2-4 vCPUs and 4-8 GB of RAM per instance. However, instead of using a few large machines, it’s best to deploy multiple medium-sized instances to balance performance and cost.
Middleware Impact: As with internal API exposure, TLS termination and JWT Authentication will still introduce latency, but at this higher traffic rate, the impact is more pronounced. Ensure the resources allocated to each instance are sufficient to handle the added load from security measures and header management. Also, exporting metrics in OpenTelemetry format will increase the processing load, so factor that into resource allocation.
Scaling Strategy: For this scenario, it is essential to configure autoscaling carefully to handle traffic spikes effectively. Ensure your system scales before traffic exceeds 80% of your instance capacity to avoid bottlenecks, and consider pre-emptively adding extra replicas during anticipated peak periods (e.g., during product launches or promotional events).
Recommendation: Start with 4-10 replicas to handle external traffic spikes effectively, ensuring a minimum of 20% buffer capacity for peak periods.

More than 5,000 RPS per Instance

For scenarios where APIs are exposed to high-traffic environments, such as public-facing commercial websites or services with millions of users, a more specialized setup is needed:

Instance Sizing: Deploy large instances with at least 4-8 CPUs and 8-16 GB of RAM. You might also consider using high-performance nodes or dedicated bare-metal instances if the traffic is extremely high.
Scaling Strategy: Each instance should be optimized to handle more than 5,000 RPS. You may need to use both horizontal scaling (increasing instance count) and vertical scaling (boosting instance capacity) to meet traffic demands.
Middleware Impact: At this level, even small latencies introduced by TLS handshakes, JWT, header manipulation, and metrics collection can add up quickly. It’s critical to identify each bottleneck that slows down the traffic (using Tracing) to optimize each layer of the stack for performance, and potentially offload some tasks to specialized systems (e.g., a dedicated authentication service).
Recommendation: Start with at least 10-20 replicas and monitor usage closely to ensure a quick reaction to traffic spikes. For very high loads, consider deploying Traefik Hub API Gateway in a high-availability setup, with global load balancing across multiple regions or data centers.

Conclusion

Properly sizing Traefik Hub API Gateway instances is key to ensuring secured API publication, whether you're dealing with internal or external traffic. By understanding the specific needs of your environment—such as traffic volume, middleware usage, and infrastructure type—you can deploy a cost-effective and scalable Traefik Hub API Gateway configuration that meets your performance goals.

Regardless of the scenario, monitoring resource consumption and scaling based on usage thresholds (like 80% of CPU or RAM) is crucial to avoiding performance bottlenecks. By following these guidelines, you’ll ensure your Traefik Hub API Gateway deployment is both resilient and optimized for your specific use case.

Useful Links

Traefik API Gateway Documentation
Traefik API Gateway Webpage
Our Community Forum

]]>

The AI Gateway Imperative: Why Your Enterprise AI Strategy is Incomplete Without it

Sudeep Goswami — Mon, 10 Feb 2025 11:02:17 GMT

The enterprise AI landscape has reached a pivotal moment. According to Deloitte's latest research, 52% of organizations are prioritizing automated AI agents, while 45% are focused on multiagent systems.

Even though the uptake of these technologies is impressive, given how new they are, we're rapidly moving beyond basic automation to autonomous AI agents that can plan, decide, and act independently. These "digital workers" are transforming enterprise operations (adoption has surged to 78% of companies surveyed, according to McKinsey), but they're also introducing unprecedented complexity and risk.

In this article, we’ll explore this newfound complexity and risk, as well as how to tame them.

The Reality of Enterprise AI Usage Today

Enterprise AI adoption is accelerating, but it’s happening in silos. Development teams deploy AI-powered chatbots using their preferred providers, HR leverages AI-driven resume screening tools, and marketing automates content generation with yet another set of AI services. Each department operates independently—selecting its own vendors, enforcing separate security controls (if at all), and managing AI usage in isolation.

This fragmented approach forces organizations to navigate a growing maze of AI integrations, each with different authentication methods, governance frameworks, and cost structures. A single company might have marketing using GPT-4, customer service relying on Claude, and HR leveraging Gemini—each through separate interfaces, with inconsistent security policies and compliance measures.

As AI investments continue to rise—67% of organizations plan to increase spending (McKinsey), and 78% expect further growth in 2025 (Deloitte)—this lack of cohesion is becoming increasingly unsustainable.

The Hidden Risks of Ungated AI

The fragmented, decentralized AI adoption exposes organizations to serious risks. Without a unified mechanism to manage and monitor AI interactions, companies face mounting challenges.

Compliance Challenges
As the #1 barrier to GenAI deployment, regulatory compliance isn't just about checking boxes. When AI systems handle sensitive customer data or make automated decisions affecting individuals, a single compliance misstep can lead to substantial fines and damaged reputation.
Security Vulnerabilities
The fact that 35% of organizations cite "mistakes/errors with real-world consequences" as their top concern (Deloitte) isn't surprising. Imagine an AI-powered trading system making unauthorized transactions, or a customer service AI exposing sensitive information—these aren't hypothetical risks but real scenarios playing out in unprepared organizations.
Cost Management
The statistic that 78% of organizations plan to increase AI spending in 2025 (Deloitte) takes on new meaning when you consider uncontrolled usage. Without proper oversight, departments can unknowingly rack up massive bills through inefficient API calls or redundant model usage.

The Strategic Role of an AI Gateway

An AI gateway serves as your enterprise's command center for AI operations, fundamentally transforming how organizations manage their AI infrastructure. At its core, it functions as a unified control point that abstracts away the complexity of multiple AI providers and models. This allows development teams to focus on what truly matters—building valuable applications—rather than getting bogged down in managing complex integrations.

Security and governance are equally crucial aspects of the gateway's role. It acts as a centralized checkpoint ensuring all AI interactions adhere to enterprise policies—a critical function given that 70% of organizations need at least 12 months to resolve governance challenges (Deloitte). A centralized approach means security policies can be consistently enforced across all AI interactions, regardless of which team or department initiates them.

Additionally, as AI usage scales across the enterprise, the gateway's intelligent traffic management becomes increasingly vital. Through sophisticated caching and load-balancing capabilities, it optimizes resource utilization and controls costs while maintaining high performance. This means organizations can scale their AI operations confidently, knowing they have the infrastructure to support growing demand without compromising on efficiency or reliability.

Accelerating Enterprise-Wide Adoption

The path to widespread AI adoption in enterprises faces significant hurdles.

Current data shows that less than 40% of the workforce has access to GenAI tools (Deloitte), and even among those with access, adoption rates remain surprisingly low.

This limited penetration stems from multiple challenges.

Many employees feel overwhelmed by the complexity of AI tools.Others worry about inadvertently violating company policies. Some teams struggle to justify the investment when they can't properly track and measure AI's impact on their productivity.

An AI Gateway provides the critical infrastructure foundation that enables widespread AI adoption. It offers essential backend capabilities that organizations can build upon to create consistent, reliable, and secure AI experiences across the enterprise. This standardized infrastructure means teams can focus on creating solutions that address their specific business needs, which are supported by enterprise-grade security and governance controls.

Moreover, the Gateway's rich observability capabilities enable organizations to track usage patterns, measure ROI, and identify departments that might need additional training or support. This data-driven approach helps organizations understand where adoption is lagging and why, allowing them to make targeted interventions. For example, if certain teams show low usage rates, the organization can provide specialized training or showcase relevant use cases from successful departments.

By providing a robust foundation for AI applications, ensuring consistent security controls, and offering visibility into AI usage, the Gateway transforms AI from a specialized tool for technical teams into an enterprise-wide capability that can serve every employee's needs.

Competitive Advantages of AI Gateway Implementation

Organizations that implement AI gateways gain significant strategic advantages that extend far beyond basic infrastructure benefits. While an AI gateway may appear to be just another technical implementation, it fundamentally transforms how enterprises operate with AI technology.

Time-to-market acceleration becomes a key differentiator. Development teams can launch new AI initiatives in days rather than months by leveraging pre-built integrations and standardized security controls. Instead of each team reinventing the wheel for API connections, authentication, and compliance checks, they can focus on creating business value.

Operational costs decrease substantially through intelligent resource optimization. The gateway's ability to route traffic efficiently, cache responses, and load-balance across providers means organizations typically see 30-40% reduction in their AI operational costs (Deloitte). Teams can experiment with different AI models without worrying about unexpected cost spikes or resource waste. These cost benefits are being realized across industries—McKinsey's research shows that organizations are seeing material benefits from AI deployment, with notable cost decreases particularly in human resources and service operations.

Security and compliance posture strengthens considerably across the organization. Rather than managing security policies for each AI application individually, organizations can enforce consistent controls at the gateway level. Centralizing security not only reduces risk but also streamlines audit processes and regulatory reporting.

Technology adaptation becomes seamless as the AI landscape evolves. When new AI models emerge or existing providers enhance their capabilities, organizations can quickly integrate these advances without disrupting existing applications. Enterprises are then better able to stay current with AI innovation while maintaining operational stability.

So in short, AI gateways transform scattered AI initiatives into a cohesive enterprise capability. It's analogous to the difference between having a fleet of autonomous vehicles with no traffic management system versus having a sophisticated control tower that ensures safety, efficiency, and coordination. The AI gateway becomes the foundation that turns isolated AI experiments into a true AI-powered enterprise.

The Time to Act Is Now

With market pressures accelerating AI adoption, that stark Deloitte finding bears repeating—70% of organizations lose a whole year struggling with AI governance and risk management.

This extended implementation timeline means organizations must act now, as the window for gaining a competitive advantage through proper infrastructure is closing. Organizations that delay implementing an AI gateway risk falling behind more agile competitors who can deploy and scale AI initiatives faster and more securely. McKinsey's findings reinforce this urgency, showing that high-performing organizations are already using AI in three or more business functions, while others average just two—creating a widening competitive gap.

However, implementing an AI gateway and governance doesn't have to be a lengthy process. Modern gateway solutions leverage GitOps practices and declarative configurations, allowing organizations to get up and running in minutes rather than months. Teams can define their entire AI infrastructure as code, version control their configurations, and automate deployments. This makes the journey from decision to implementation remarkably swift.

The question isn't whether you need an AI gateway but how quickly you can realize its benefits.

In the race to harness AI's potential, having the right infrastructure isn't just an advantage—it's a prerequisite for success. And with today's tools and practices, that infrastructure is more accessible than ever.

Sources:

Deloitte’s State of Generative AI in the Enterprise Report Q4 2024

McKinsey State of AI in early 2024

Useful Links

Traefik AI Gateway Documentation
Traefik AI Gateway Webpage
Our Community Forum

]]>

Traefik Proxy v3.3 - Nectaire of the Gods

Emile Vauge — Thu, 09 Jan 2025 11:46:21 GMT

Traefik v3.3, codenamed saint-nectaire (one of the best cheeses you can find in France, made in the volcanic area Auvergne), is now generally available after two months of dedicated development and two release candidates. This release focuses on two main tracks: observability and documentation.

Observability has become essential for platform engineers to quickly detect outages and recover services in today's complex, cloud-native environments. Traefik Proxy v3.3 once again pushes the boundaries by adding state-of-the-art features with a deeper OpenTelemetry integration as well as a much more granular observability configuration. This makes Traefik Proxy one of the most flexible and powerful telemetry data providers, giving you instant and complete knowledge of all your ingress traffic.

Additionally, documentation is one of the most important (and underrated) parts of an open-source project. It’s the main entry point 😉 to your product. It should onboard newcomers to quickly learn and test while providing a clear and complete description of every option and use case to more advanced users.

Back in 2024, we started a major project to reorganize and revamp the Traefik Proxy documentation. This is quite a big journey, and we are happy to hit the first milestone today with a brand-new reference documentation.

Ready to dive into v3.3? Let’s start with OpenTelemetry.

OpenTelemetry Support for Logs and Access Logs

Since its very first release, Traefik Proxy made observability one of its core features, supporting many vendors for metrics and tracing analysis. Building upon the foundation laid in Traefik Proxy v3.0 with OpenTelemetry integration, v3.3 goes even further by adding support for logs and access logs.

This feature is tagged “experimental,” as the official underlying library is still being finalized.

To unlock it, simply add this to your configuration:

experimental:
  otlpLogs: true

Then let’s enable logs and access logs globally to the OpenTelemetry configuration:

metrics:
  otlp: {}
tracing:
  otlp: {}
log:
  otlp: {}
accesslog:
  otlp: {}

The OpenTelemetry logger exporter will send logs to the collector using HTTPS by default to https://localhost:4318/v1/logs. You can customize the exporter by referring to the logs and access logs documentation.

Flexible and Granular Observability

Until today, you could enable observability features globally on Traefik Proxy, which means, either you had metrics (or tracing) for all your routes or none. However, there are many use cases where you don’t want to have tracing for all your applications. Or you might just need to enable these features temporarily.

That’s why we have been working on a new mechanism to have more granular control over observability. In v3.3, you now have a way to define default behaviors and you can enable or disable tracing, metrics or access logs per router.

The default router observability configuration is inherited from the attached entryPoints and a router defining its own observability configuration will override those options.

Let’s take an example where you want to enable tracing by default for an entryPoint:

You still need to configure the tracing globally:

tracing:
  otlp: {}

Tracing is enabled by default on entryPoints, so you shouldn’t need to change anything in your configuration:

entryPoints:
  foo:
    address: ':80'

This is equivalent to:

entryPoints:
  foo:
    address: ':80'
    observability: 
      tracing: true

Every router attached to this entryPoint will have tracing enabled. If you want to disable a specific router, that’s pretty simple:

## Dynamic file configuration
http:
  routers:
    my-router:
      rule: "Path(`/foo`)"
      service: service-foo
      observability:
        tracing: false

Or if you use Kubernetes Ingresses, just add this annotation to the corresponding ingress:

traefik.ingress.kubernetes.io/router.observability.tracing: false

Let’s take another example where you want to disable tracing by default for an entryPoint:

entryPoints:
  foo:
    address: ':80'
    observability:
      tracing: false

And then if you want to enable tracing temporarily while troubleshooting a service:

## Dynamic file configuration
http:
  routers:
    my-router:
      rule: "Path(`/foo`)"
      service: service-foo
      observability:
        tracing: true

Or within an ingress:

traefik.ingress.kubernetes.io/router.observability.tracing: true

This simple and powerful granular control applies to tracing, metrics, and access logs.

New Reference Documentation

As mentioned before, great documentation is key for open source projects. We've been working hard to improve Traefik Proxy's docs, and we have defined new guidelines as follows:

Organize the documentation for 3 personas: beginners, advanced ops, and/or advanced developers
Describe high level use cases without listing every option possible in the first sections of the documentation (Getting Started, Setup, Expose, Observe, Migrate, etc)
Move all option descriptions to a well organized Reference section

Today we're excited to launch our brand new reference documentation as a first milestone of this global revamp.

The reference documentation used to have this structure:

├── Static Configuration
│   ├── Overview
│   ├── CLI
│   ├── Environment Variables
│   ├── File
└── Dynamic Configuration
    ├── Consul Catalog 
    └── …

We reorganized this section to provide a much more comprehensive description of all options available:

├── Install Configuration
│   ├── Boot Environment
│   ├── Configuration Discovery
│   │   ├── Overview
│   │   ├── Kubernetes
│   │   │   ├── Kubernetes Gateway API
│   │   │   ├── Kubernetes CRD
│   │   │   └── Kubernetes Ingress
│   │   ├── Docker
│   │   ├── Swarm
│   │   ├── Hashicorp
│   │   │   ├── Consul Catalog
│   │   │   ├── Consul
│   │   │   └── Nomad
│   │   ├── KV Stores
│   │   │   ├── Consul
│   │   │   ├── ETCD
│   │   │   ├── Redis
│   │   │   └── ZooKeeper
│   │   └── Others
│   │       ├── ECS
│   │       ├── File
│   │       └── HTTP
│   ├── Entrypoints
│   ├── API & Dashboard
│   ├── Observability
│   │   ├── Healthcheck
│   │   ├── Logs and Accesslogs
│   │   ├── Metrics
│   │   └── Tracing
│   └── TLS
│       ├── Certificate Resolvers
│       │   ├── Overview
│       │   ├── ACME
│       │   └── Tailscale
│       └── SPIFFE
└── Dynamic Configuration
    ├── Consul Catalog
    └── …

We hope this will help beginners and advanced users to quickly find and learn everything they want to about Traefik Proxy. The next step? Revamp high-level sections!

Other Improvements

On top of those great new features, Traefik Proxy v3.3 brings options to control ACME (Let’s Encrypt) propagation checks (#11241 by ldez). We also added configuration dump support from the API endpoint to make troubleshooting sessions easier.

Skirtan1 made the IngressRoute kind optional (#11177) and BZValoche added Kubernetes serving endpoint status support to detect and mark servers as “fenced” when stickiness is enabled. Regarding sticky cookies, you can now make the path configurable (#11166 by IIpragmaII).

Host headers in the HTTP provider can now be set (#11237 by nikonhub), and you can preserve the Location header in the ForwardAuth middleware (#11318 by Nelwhix).

Another great addition brought by michelheusschen allows you to only calculate basic auth hashes once for concurrent requests (#11143).

You can now send the request body to the authorization server with the ForwardAuth middleware (#11097 by kyo-ke).

Additionally, it is now possible to make Traefik abort the startup in case a plugin load fails (#11228 by bmagic) and finally, the API & Dashboard base path can now be configured.

Moving Forward

Traefik Proxy v3.3 is a significant release that focuses on observability and documentation. With the addition of logs and access logs to its existing support for metrics and traces, Traefik Proxy now provides even deeper insights into your ingress traffic. Additionally, the new granular observability configuration allows you to enable or disable tracing, metrics, or access logs on a per-router basis, giving you more control and flexibility over the data you collect. And the improvements to the documentation, including the new reference documentation, make it easier for users to learn about and use Traefik Proxy.

Overall, v3.3 is a powerful release that makes Traefik Proxy an even more valuable tool for platform engineers and developers.

We are immensely thankful to the contributors who are shaping this great project. Join the vibrant Traefik community, share your feedback, and help us shape the future of cloud-native networking.

Useful Links

Traefik 3.3 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

How to Keep Your Services Secure With Traefik’s Rate Limiting

Harold Ozouf — Tue, 10 Dec 2024 18:34:52 GMT

The internet can be a challenging environment for running applications.

When you expose a service to the public internet, it's crucial to assess the risks involved. Malicious actors may try to misuse your resources or even bring your service down.

You can't just make your service publicly available without protection and expect things to go smoothly. They won’t. That’s why safeguarding your service is essential. There are many threats to address. Configuring your server properly is a good start, but you also need to protect your application from harmful actions. Our Web Application Firewall (WAF) can assist with that.

However, today we're focusing on a different threat: attempts to overwhelm your resources or disrupt your service with an excessive number of requests. This is where rate limiting comes in.

What is Rate Limiting?

Rate limiting is the process of limiting the flow of requests that reach your servers. Think of it like a funnel, where a large pipe of water narrows into a smaller one, flowing at a much more manageable rate before reaching its destination:

In this analogy, each water molecule represents an HTTP request. A pipe of a certain size limits how many molecules can pass through at once. When the pipe narrows, fewer molecules can flow through, reducing the flow rate.

This illustrates how a rate limiter works to control the flow of requests. Additionally, we can fine-tune the control by limiting traffic based on characteristics like IP address, user, or other request details. You might also want to allow brief bursts of traffic without blocking them entirely.

When discussing rate limiting, the following algorithms are commonly mentioned:

Token Bucket: The one used by Traefik, which we'll focus on below.
Leaky Bucket: A first-in, first-out queue that releases traffic at a steady rate. It's less flexible than Token Bucket for handling traffic bursts.
Generic Cell Rate Algorithm (GCRA): Similar to Leaky Bucket but ensures packets follow a set timing interval instead of draining at a fixed rate.

Two other types of algorithms often come up:

Sliding Window
Fixed Window

Unlike rate limiting, which regulates the flow and timing of requests, these two algorithms count the total number of requests within a specific period. They're better suited for enforcing strict usage quotas than managing request flow.

Token Bucket

The Token Bucket algorithm controls the flow of requests by using a metaphorical bucket that holds tokens. Tokens are generated at a constant rate and added to the bucket, which has a fixed capacity.

When a request arrives, the system checks if there are enough tokens in the bucket. Each request consumes one token. If enough tokens are available, the request proceeds, and a token is removed. If there aren’t enough tokens, the request is either delayed until more tokens are available or blocked if the delay would be too long.

The bucket can't hold more tokens than its maximum capacity, so once it's full, any new tokens are discarded. This prevents tokens from accumulating indefinitely and allows the system to handle short bursts of high traffic, as long as the bucket has enough capacity.

When the bucket is low or empty, the system slows down, giving time for more tokens to be generated. The token generation rate and the bucket size are key factors that determine how the system performs.

The behavior of the Token Bucket algorithm is determined by two key factors:

Bucket size: This defines how many requests can be processed simultaneously.
Rate: This controls how frequently new request opportunities become available as tokens are generated.

Rate Limiting in Traefik

Traefik allows you to define a ratelimit middleware that can be applied to your routers. Assigning this middleware ensures that the flow of incoming requests doesn’t exceed the configured rate.

💡

Reusing the same middleware on multiple routers does not mean they share the same bucket. Each router gets its own instance of the middleware. Shared buckets and distributed rate limiting for enterprise use cases are available through Traefik Hub.

This middleware uses the Token Bucket algorithm described earlier. The bucket size is controlled by the burst parameter, while the rate is set by the average and period values. The following example configures a rate limiter with a rate of 100 requests per second and a bucket size of 200:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: my-rate-limit
spec:
  rateLimit:
    average: 100
    period: 1s
    burst: 200

This configuration means a new request can pass every 10ms. If you receive fewer than 1 request every 10ms, the bucket will fill up, eventually allowing a burst of up to 200 simultaneous requests.

Grouping Requests by Source

Grouping requests by their source and applying rate limiting is an effective way to prevent a small number of clients from overwhelming a system. Instead of limiting the system as a whole, this technique enforces rate limits per individual source, ensuring balanced resource usage.

For example, you can group requests by their IP address and apply rate limits to each IP. This prevents a single IP from sending too many requests, which could degrade the system’s performance. By limiting requests at the source level, the system gains more granular control, preventing any one source from monopolizing resources while still allowing others to access the system at normal rates.

The ratelimit middleware allows requests to be grouped by different sourceCriteria, such as:

ipStrategy: The client’s IP address (default strategy)
requestHost: The client’s hostname
requestHeader: The value of a specific request header

The ipStrategy relies on the de-facto standard X-Forwarded-For header to determine the client’s IP address. When a client connects directly to a server, its IP address is sent to the server. But if a client connection passes through any proxies, the server only sees the final proxy's IP address, which is often of little use. So, to provide a more-useful client IP address to the server, the X-Forwarded-For request header is used. When a proxy receives a request it adds itself to this header.

The X-Forwarded-For header can be a useful tool, but it also poses security risks if not handled properly. Since each proxy in a network chain adds its own IP address to this header, a malicious client could manipulate it by inserting fake IP addresses. This can mislead the server into believing that the request originated from a different location, allowing attackers to potentially spoof their IP address and bypass security measures that rely on client IP verification.

To minimize this risk, it’s essential to only trust proxies under your control or those known to handle requests securely.

In Traefik, this is managed through the --entryPoints.web.forwardedHeaders.trustedIPs setting. If a request originates from an untrusted proxy, the X-Forwarded-For header will be unset. This ensures that only IP addresses from trusted sources are considered, preventing IP spoofing and maintaining the integrity of the real client IP address.

When you are in the situation where you have proxies before Traefik, you will need to configure at least of these options:

depth: Specify from right to left, the nth IP address to use. This is needed to avoid spoofing.
excludedIPs: Specify a list of IP addresses to exclude from X-Forwarded-For header.

For example, if the X-Forwarded-For header contains the value 10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1, a depth of 3 will use 11.0.0.1 as the client's IP address, and a depth of 3 with an excluded IP of 12.0.0.1 will use 10.0.0.1.

For this example, the middleware could be configured like:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-ratelimit-with-ip-strategy
spec:
  rateLimit:
    average: 100
    period: 1s
    burst: 200
    sourceCriterion:
      ipStrategy:
        depth: 3

Conclusion

Rate limiting is essential to protect your services against overwhelming traffic, but for enterprises, the challenge often goes beyond just managing individual servers. With distributed rate limiting through Traefik Hub, you can scale your traffic management across multiple gateway replicas, ensuring consistent protection and control no matter how distributed your infrastructure becomes.

By leveraging shared buckets across routers and applying limits at both global and granular levels, Traefik Hub helps enterprises maintain fairness, prevent resource monopolization, and optimize performance. This distributed approach offers robust protection against high-traffic events, making it ideal for large-scale operations where traffic spikes and resource demands are common.

While rate limiting is an essential tool in your security arsenal, it should be part of a broader strategy that includes other measures like Web Application Firewalls (WAF), active monitoring, and proper server configuration. By combining these layers, you can create a robust defense system that keeps your services running smoothly and securely in the face of various internet challenges.

As you implement rate limiting in your Traefik setup, consider your specific use case, traffic patterns, and resource constraints to fine-tune the settings for optimal performance. Regular monitoring and adjustments will help ensure that your rate-limiting strategy continues to meet your needs as your service grows and evolves.

💡

Ready to safeguard your services with flexible, enterprise-grade distributed rate limiting? Experience it firsthand with Traefik Hub. Sign up for a 14-day free trial or speak to our experts today to discover how Traefik can elevate your infrastructure's resilience.

]]>

Taming The Wild West of LLMs with Traefik AI Gateway

Emile Vauge — Tue, 03 Dec 2024 15:13:12 GMT

Within the last ten years, deep learning has moved from a research area to wild industry adoption thanks to the recent generative AI gold rush. This has led to the arrival of Large Language Models (LLMs) into mission-critical infrastructure at an unprecedented pace. Organizations are rapidly discovering that different LLMs excel at different tasks – from OpenAI, Claude or Gemini for generic content creation and problem-solving, up to domain-specific models for highly specialized tasks. This multi-model approach has become essential for maximizing AI's business value.

However, such a gold rush has created a familiar pattern in tech history: rapid adoption leading to fragmented infrastructure and technical debt. Each AI vendor has developed their own proprietary APIs, authentication methods, and SDKs, creating a "Wild West" environment without any governance and where complexity is king. As organizations scale their AI initiatives, this uncontrolled growth threatens to undermine the very benefits these technologies promise.

The Enterprise AI Challenge Landscape

Now that GenAI is being integrated into production environments, the challenges of large-scale AI deployment is becoming more apparent. Let’s dive into the most critical ones.

The number one problem is integration complexity. Development teams struggle with managing multiple SDK implementations, each with its own quirks and requirements. This complexity slows down development and creates maintenance nightmares as teams juggle different API versions and implementation patterns. Additionally, integrations with those different stacks lead to being vendor-locked as there is no straightforward way to migrate from one model to another.

Once integrated with those SDKs, companies end up facing even more issues in production. Security being the most visible. API keys and credentials scattered across various systems create significant security risks. Without centralized control, organizations struggle to maintain secure access patterns and protect sensitive credentials from exposure.

The lack of standardized governance leads to inconsistent policy enforcement across different AI services. Without uniform and centralized authentication, authorization or rate limiting rules, compliance issues and resource waste cannot be avoided.

Finally, this fragmentation also leads to major observability black holes, with no way to optimize costs, monitor performance, or detect issues effectively.

The benefits of GenAI do not come for free and barriers are very real. That’s why at Traefik Labs, we made the decision to tackle those problems, with our unique expertise in ingress and API management.

Enter Traefik AI Gateway

Traefik Labs addresses these new challenges with Traefik AI Gateway, a powerful solution that unifies and streamlines LLM management at the API level. By transforming any AI endpoint into a secure, governable API, Traefik AI Gateway allows enterprises to connect to multiple LLMs through a unified AI API, simplifying integration and centralizing control.

Traefik AI Gateway offers effortless integration with various popular LLMs, eliminating the need for multiple SDKs and client integrations. It ensures centralized security and credential management, minimizing risk and facilitating consistent policy enforcement. It prevents vendor lock-in, enabling easy switching between LLM providers without changing client applications. It supports OpenTelemetry for comprehensive observability across LLM usage, empowering data-driven operations and performance optimizations. Furthermore, Traefik AI Gateway offers unified governance through centralized policy enforcement, ensuring compliance with industry regulations.

Traefik AI Gateway is natively integrated to Traefik Hub and lets organizations gain access to enterprise-grade API capabilities that bring LLMs connectivity to industry standards with advanced access controls, security at scale and premium integrations. It means that AI apps can now benefit from the most advanced APIs features like rate limiting by plan, enterprise access control with identity providers or API keys, deep troubleshooting thanks to traffic debugging, etc.

It’s extremely simple to deploy Traefik AI Gateway, here is a quick overview on how to set it up. You need to have Traefik Hub Gateway up and running to enable Traefik AI Gateway, head over here if that’s not already the case.

First, upgrade the CRDs and your hub deployment enabling the AI gateway feature:

kubectl apply --server-side --force-conflicts \
-k https://github.com/traefik/traefik-helm-chart/traefik/crds/

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set "additionalArguments={--hub.experimental.aigateway}" \
     traefik/traefik

Then, define & apply an AIService resource with any of the supported AI providers, for example OpenAI:

apiVersion: hub.traefik.io/v1alpha1
kind: AIService
metadata:
  name: ai-openai
  namespace: traefik
spec:
  openai:
    token: YOUR_OPENAI_TOKEN
    model: gpt-4o

Finally, attach the ai-openai AIService to an IngressRoute as a TraefikService:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ai-test
  namespace: traefik
spec:
  routes:
    - kind: Rule
      match: Host(`ai.localhost`)
      services:
        - kind: TraefikService
          name: traefik-ai-openai@ai-gateway-service

As simple as that! Now you can request OpenAI on http://ai.localhost:

curl -d '{
    "messages": [
        {
            "role": "user",
            "content": "tell me a joke"
        }
    ]
}' http://ai.localhost

{
  "id": "chatcmpl-AYGYJuSVWlp6gRFVvO9ClTN36eRkQ",
  "object": "chat.completion",
  "created": 1732730143,
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Why don't scientists trust atoms?\n\nBecause they make up everything!"
      },
      "finish_reason": "stop",
      "content_filter_results": {
        "hate": {
          "filtered": false
        },
        "self_harm": {
          "filtered": false
        },
        "sexual": {
          "filtered": false
        },
        "violence": {
          "filtered": false
        },
        "jailbreak": {
          "filtered": false,
          "detected": false
        },
        "profanity": {
          "filtered": false,
          "detected": false
        }
      }
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 12,
    "total_tokens": 23,
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0
    },
    "completion_tokens_details": {
      "audio_tokens": 0,
      "reasoning_tokens": 0
    }
  },
  "system_fingerprint": "fp_831e067d82"
}

Traefik AI Gateway implements the Open Telemetry GenAI semantic conventions which provides metrics (gen_ai.client.token.usage and gen_ai.client.operation.duration) designed to monitor generative AI client applications thanks to specific attributes like token type or model. AI apps present specific challenges compared to traditional software systems and Traefik AI Gateway provides unique LLM observability capabilities to ensure platform teams have the best situation awareness.

Traefik AI Gateway is a natural evolution of Traefik's established expertise in Ingress & API management. Built on top of one of the most widely deployed open-source ingress controllers & API Gateways, Traefik Hub brings its battle-tested experience in managing incoming traffic to the emerging challenge of egress AI traffic.

Shaping the Future of AI Gateways

As enterprises continue to scale their AI initiatives, the need for structured, governed access to LLM services becomes increasingly critical. Traefik AI Gateway provides immediate solutions to today's challenges while laying the groundwork for future developments.

Traefik Labs is deeply invested in the Gateway API effort, the next evolution of Kubernetes Ingress, and is actively working on developing specifications for LLM routing within the Gateway API framework. This ongoing commitment to open standards ensures that Traefik AI Gateway will continue to evolve with enterprise needs.

Organizations looking to bring order to their AI infrastructure should consider Traefik AI Gateway as a strategic investment in their AI strategy. With its combination of production-proven expertise in the ingress & API management space, Traefik is uniquely positioned to help enterprises tame the Wild West of LLMs and build a foundation for sustainable AI growth.

Useful Links

Traefik AI Gateway Documentation
Traefik AI Gateway Website
Our Community Forum

]]>

BigBasket Modernizes Kubernetes Traffic Management with Traefik's API Gateway

Marie Ponseel — Tue, 26 Nov 2024 12:18:08 GMT

Overview

BigBasket, India’s largest online grocer, faced growing challenges in managing traffic across their Kubernetes infrastructure. As their e-commerce platform scaled, their microservices infrastructure became more complex and increasingly difficult to manage, leading to concerns over scalability and operational efficiency. Sushant Gulati, Senior Engineering Manager for Cloud, DevOps, and Platform, alongside Sayantan Bhattacharjee, Principal DevOps Engineer, led the effort to find a robust API Gateway solution to simplify microservice traffic management, improve day-2 operations, and future-proof their infrastructure.

Challenge

Before Traefik, BigBasket’s network architecture relied on NodePort services and AWS ALB Target Groups (TG). While this worked at a smaller scale, this solution quickly reached its limits in terms of scalability and manageability. Each worker node was attached to every ALB target group, causing API throttling due to the excessive volume of register and deregister calls. As traffic grew, the system became unstable, eventually reaching AWS API limits.

In addition to stability issues, scaling became difficult as microservices and hosts multiplied. Infrastructure management was increasingly time-consuming and complex. To address these challenges, they needed a solution capable of simplifying traffic routing while accommodating their growing needs.

Solution

After evaluating several API Gateway solutions, BigBasket’s team chose Traefik for its simplicity, scalability, and performance.

During their evaluation, BigBasket found that other solutions involved complex configuration and required numerous annotations to meet their custom needs, resulting in a high administrative burden. In contrast, Traefik’s use of Custom Resource Definitions (CRDs) made configuration much simpler and easier to manage and customize. Traefik provided a more intuitive solution that significantly reduced management effort while meeting their scalability and performance requirements.

Key factors in selecting Traefik included:

Ease of Management: Traefik’s simpler configuration through CRDs allowed the team to manage their infrastructure with minimal effort, providing much-needed operational breathing room.
Distributed Architecture: Traefik’s API Gateway offered a highly available, scalable, and secure architecture, making it a reliable choice for BigBasket.
Advanced Traffic Management: Traefik’s API Gateway efficiently handled various routing scenarios, such as header-based, host-based, cookie-based, and path-based routing.
High Performance: Traefik's performance, particularly its ability to handle increased traffic loads, was a critical requirement for BigBasket.

“With Traefik, deployment was a breeze, our success criteria were satisfied, and making this major change in our architecture was worry-free knowing we had enterprise support.” Sayantan Bhattacharjee, Principal DevOps Engineer at BigBasket

Implementation

BigBasket deployed Traefik’s API Gateway as their primary solution across five Kubernetes clusters, all running in AWS. The new Traefik architecture became the foundation of their network, handling every transaction across their e-commerce platform. All their critical services are now routed through Traefik’s API Gateway, making it an indispensable component of their infrastructure.

BigBasket architecture with Traefik

The team significantly simplified their routing configuration by eliminating the need for multiple AWS target groups and application load balancers (ALBs). This not only reduced complexity but also allowed them to automate much of their setup and significantly improve the management of their architecture.

Traefik’s API Gateway also enabled BigBasket to manage five different versions of the same service within a single cluster, supporting their testing needs without requiring additional tools or investments.

“Traefik is simple to configure, doesn’t have component sprawl, and has significantly reduced our service onboarding and route management efforts.” Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

Visibility into traffic management has improved substantially. With Traefik, the team gained deeper insights into traffic flow, making it easier to monitor and troubleshoot issues. The ability to track everything in real-time has been transformative and allowed them to efficiently manage and optimize their system. Everyone involved at BigBasket found Traefik simple to manage, resulting in smoother day-to-day operations.

Grafana dashboard showing Traefik’s request count

Additionally, the support from Traefik Labs played a crucial role in their success. The team praised Traefik's proactive approach, noting how helpful their experts were during setup and in answering ongoing questions. Regular meetings with the Traefik Labs team ensured that BigBasket received continuous guidance and assistance in resolving any challenges.

“The support from Traefik has been very helpful, the team is very knowledgeable, forthcoming, and open to discussions.” Sayantan Bhattacharjee, Principal DevOps Engineer at BigBasket

Results

Since adopting Traefik’s API Gateway, BigBasket has experienced the following improvements:

Scalability without limitations: Traefik has enabled BigBasket to scale seamlessly alongside their growing business The team has already increased their Traefik adoption by 50% within the first 12-months to support the expansion of their infrastructure needs. Had they remained on their previous solution, they would not have been able to scale to this extent.
High availability and reliability: Over the past year and a half, BigBasket experienced no stability issues in their clusters, even during periods of significant traffic spikes, thanks to Traefik’s robust and stable environment.
Simplified Traffic Management: Traefik eliminated the need for managing thousands of AWS Target Groups and ALBs, significantly reducing complexity and administrative overhead. This allowed BigBasket to handle a 2x increase in internal traffic without performance degradation.
Reduced Complexity: By consolidating traffic management through Traefik’s API Gateway, BigBasket reduced the number of components needed to manage their system, freeing up engineering resources for more strategic tasks.
Enhanced Visibility: Traefik’s built-in metrics and dashboard provided BigBasket with real-time insights into their traffic flow. This increased visibility, made troubleshooting easier, and allowed the team to respond quickly to system changes. The ability to monitor traffic at a granular level—down to requests-per-second (RPS)—has been critical in optimizing system performance and ensuring smooth operations.

“Traefik has exceeded our expectations; it has done a lot more for BigBasket than what is normally expected out of an API Gateway.” Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

Key Takeaways

Traefik’s API Gateway has transformed BigBasket’s approach to managing Kubernetes traffic, delivering significant improvements in efficiency, scalability, and system stability. Traefik’s distributed architecture, automation, and performance have allowed BigBasket’s team to future-proof their infrastructure. Key outcomes include:

50% increase in Traefik usage, to support the expansion of their infrastructure.
2x increase in east-west traffic, handled seamlessly without impacting performance.
Elimination of 7,000 NodePort configurations, drastically reducing the operational complexity.
Mitigated AWS API throttling, resulting in fewer errors and operational headaches.

Sushant and his team at BigBasket continue to expand their use of Traefik, confident that it provides the flexibility, visibility, and stability needed to support their business at scale.

“Traefik has been a game-changer for us. Its ease of configuration, scalability, and seamless support have exceeded our expectations. Traefik is a strategic tool that has transformed how we manage traffic across our infrastructure. I would highly recommend Traefik to any organization looking to modernize their Kubernetes environment.” — Sushant Gulati, Senior Engineering Manager – Cloud, DevOps and Platform at BigBasket

]]>

How Traefik Labs is Pioneering the Kubernetes Gateway API Revolution

Emile Vauge — Tue, 29 Oct 2024 15:00:01 GMT

Since Kubernetes’ inception, the Ingress specification has been a reliable solution for managing inbound HTTP/S traffic, helping to expose services externally. As a leader in the Cloud-Native industry, Traefik was one of the first Ingress Controllers on the market and played a key role in the adoption of this API.

However, as organizations increasingly scale up their Kubernetes deployments and require more flexibility, the limitations of Ingress become apparent. Enter Kubernetes Gateway API, a new specification designed to address those limitations, introducing a more advanced and extensible model. It is no surprise that Traefik is playing a key role in this specification effort being one of the most complete Gateway API implementations on the market.

This article explores why the Gateway API is set to replace the traditional Ingress specification and how Traefik Labs drives this joint effort.

Exposing Workloads in Kubernetes: A Brief History

When Kubernetes was first introduced, it had different networking concepts: Services, Load Balancers and Ingress. Kubernetes Services allowed applications within a cluster to communicate, load balancers were used to provision cloud provider LBs, but when it came to exposing workloads externally, we used (and are still using) Ingress, a resource designed to handle HTTP/S routing to services. Ingress was a good foundation, but it soon became clear that it had limitations. It relied on provider-specific implementations with countless specific annotations, lacked flexibility, and offered only basic HTTP traffic management. Mastering the Ingress API covers only about 20% of the knowledge needed to address your needs, the next 80% is the “annotation hell”:

kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: https://example1.com/
    nginx.ingress.kubernetes.io/session-cookie-name: route
    nginx.ingress.kubernetes.io/too-many-annotations: "true"

This left many teams needing more advanced solutions to handle complex networking scenarios. That’s why we introduced the IngressRoute resource with Traefik v2.0 in 2019. The idea was to solve the annotation hell with a much more powerful structured CRD, but this was at the cost of vendor neutrality as it was only available for Traefik.

It was clear that Kubernetes needed a standardized way to manage inbound networking more elegantly.

Gateway API Comes to the Rescue

Kubernetes Gateway API is the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs. It has been incubating since 2019 and used to be called Service API at first. It introduces new concepts that simplify and standardize how Kubernetes clusters manage traffic, both internally and externally. Gateway API is designed to be vendor-neutral, layered and flexible at its core, and future-proof, addressing many of the shortcomings of Ingress resources.

Role-Based Resources

Gateway API introduces a set of resources that decouple networking responsibilities: GatewayClass resources which define the gateway controller and Gateway resources represent entry points into the cluster, can be managed by the operators of the cluster.

As for Developers, they will specifically use HTTPRoute or TLSRoute resources which define how traffic coming via the gateway maps to the services.

This separation of responsibilities makes it easier to define clear boundaries between different roles, allowing developers, network admins, and operations teams to work seamlessly together.

Layered Flexibility

Unlike the Ingress resource, which focused on HTTP/S traffic, Gateway API supports routing for multiple protocols like HTTP/S but also TCP, UDP and gRPC. This makes it an ideal choice to handle a diverse set of traffic types.

Another powerful concept is how policies can be applied at various layers—global, gateway-specific, or route-specific—allowing for a fine-grained control over how traffic is managed. Thanks to the Policy Attachment pattern, it is even possible to augment the behavior of an object that can't be described within the spec of that object.

Expressiveness & Vendor Neutrality

One of the key goals of Gateway API is to provide a consistent API across different environments. The core features set comes with header-based matching, traffic weighting, and many other capabilities that were only possible in Ingress through custom annotations. This allows developers to define routing rules and policies without being tied to a specific vendor’s implementation (through annotations). Whether you use Traefik, NGINX, or another provider, Gateway API aims to offer a powerful yet consistent experience.

Extensibility

Gateway API has been designed with extensibility in mind. Custom resources can be “augmented” at basically any layer of the API with Metaresources or ExtensionRef. This enables precise customization at the right levels within the API structure.

As you can see, Gateway API is much more powerful and flexible than Ingress. Most common exposing patterns can be tackled out of the box, and more complex needs can be addressed with extension mechanisms. This is a breath of fresh air for anyone who has been dealing with dozens or hundreds of Ingresses in production, familiar with the vendor-centric annotation hell.

Traefik Labs, Leader in the Gateway API Effort

Traefik Labs has always been a leader in the Kubernetes industry, being one of the very first Ingress Controllers back in 2016. Today is no different, Traefik is strongly involved in the Kubernetes Gateway API effort, playing a significant role in its development and adoption. Traefik was amongst the first controllers to adopt and support the Gateway API first alphas, recognizing its potential to simplify Kubernetes inbound traffic. Since then, Traefik Labs has been actively involved in the Kubernetes SIG-Network community, contributing to the development and refinement of the Gateway API specification. This involvement has allowed Traefik to influence the direction of the API, pushing for features that enhance user experience, security, and flexibility.

Traefik has been integrating Gateway API 1.0 since v3.0 and is continuously investing to provide the latest version available. Today, Traefik v3.2 brings support to fresh new Gateway API 1.2 adding GRPCRoute, TLSRoute, as well as more features on the HTTPRoute like h2c (HTTP/2 without TLS) and WebSocket support for backends, port matching in rules, response header modification, etc.Traefik Labs is committed in the long term to continually investing in the support and enhancement of Gateway API, ensuring that the community will always benefit from the latest progress in that space.

Conclusion

Gateway API is not just an improved Ingress resource. It’s basically a full replacement of Ingress, addressing all major flaws of this (too) basic specification and bringing advanced features to cover every need in cloud-native connectivity. By standardizing how inbound and east-west traffic is managed, Gateway API is making Kubernetes networking easier and more accessible for everyone. Its extensibility and flexibility will enable companies to tackle more complex workload exposure use cases, without sacrificing vendor neutrality.

As a pioneer of the Cloud-Native ecosystem, Traefik Labs has been at the forefront of designing the future of Ingress. This journey started back in 2019 and five years were needed to complete this giant step to the first GA of Gateway API. Today, as one of the most complete Gateway API implementations in the industry, we are extremely proud of the work accomplished. Seeing the impressive ongoing activity on the project, there is no doubt that Gateway API is only at the beginning of this joint effort.

Useful Links

Gateway API documentation
Traefik 3.2 release notes on Github
Getting started with Kubernetes Gateway API and Traefik

]]>

Traefik Proxy v3.2 - A Munster Release

Emile Vauge — Tue, 29 Oct 2024 14:58:03 GMT

After 3 months of intensive development and two release candidates, we are thrilled to announce the general availability of Traefik v3.2. With Halloween just around the corner, we wanted a munster codename for this new version 👹. Let’s dig deeper into the key changes.

As Traefik Labs continues to lead the Kubernetes Gateway API effort, it is no surprise that the new Gateway API v1.2 is supported out of the box in this latest Traefik release. We will explore the latest additions brought by the most recent development.

And that’s not all! The team has been working very hard on building an alternative HTTP reverse proxy engine for quite some time now, and the day has finally come to make it available to everyone. This Fast Proxy engine offers a whopping ~ 50% performance boost compared to the standard engine. Yes, that means approximately 50% more requests per second 💥

The clue's in the name—we're calling it a 'Munster' release for a (spooktacular) reason! Let’s dive in!

Performance Breakthrough

The Traefik team has been working on writing a complete new HTTP/1 engine. It took a long time to converge and we are extremely excited to announce that the new Fast Proxy engine is available for everyone in experimental.

experimental:
  fastProxy: {}

The Fast Proxy engine is a high-performance reverse proxy designed to enhance the performance of routing based on a zero allocation pipeline. This new engine significantly improves performance, boasting a remarkable 50% increase in speed compared to the standard engine.

Here are the results of an analysis conducted on our benchmark platform using the standard vs. the Fast Proxy Engine.

The difference between both engines is mind-blowing, you can now expect around 50% increase in throughput and 50% decrease in latency handled by Fast Proxy Engine through this new high performance proxy engine.

Kubernetes Gateway API v1.2

Traefik Labs has always been a key player in Kubernetes inbound traffic management. Gateway API, seen as the next generation for Ingress, is probably one of the most active Kubernetes projects. Traefik was amongst the first controllers to adopt and support the Gateway API first alpha release, and since v3.0, Traefik has continuously integrated updates. Today, Traefik v3.2 introduces support for the newly released Gateway API 1.2, and brings new additions like GRPCRoute and TLSRoute, as well as additional HTTPRoute features such as h2c (HTTP/2 without TLS), WebSocket support for backends, response header modifications, and more.

Let’s start with GRPCRoute, which is a type for specifying routing behavior of gRPC requests. gRPC is a widely adopted RPC framework popular across the industry, within Kubernetes itself. Therefore, a specific type to route this protocol provides a more granular configuration.

Similarly to HTTPRoute, the specification let you define hostnames (a list of hostnames to match against the Host header of the gRPC request), matches (conditions used for matching gRPC requests), filters (to process or alter the request) and backendRefs (defines the backend services to which matching requests should be sent). Here is an example of a GRPCRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
  name: grpc-route
spec:
  parentRefs:
  - name: traefik-gateway
  hostnames:
  - "example.com"
  rules:
  - filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: my-header
            value: foo
  - matches:
    - method:
        service: com.example.User
        method: Login
    backendRefs:
    - name: login-v1
      port: 50051
  - matches:
    - headers:
      - type: Exact
        name: magic
        value: foo
      method:
        service: com.example.Foo
        method: DoFoo
    backendRefs:
    - name: foo-v1
      port: 50051
      weight: 90
    - name: foo-v2
      port: 50051
      weight: 10

TLSRoute support has also been added to Traefik in v3.2’s experimental channel, to enable multiplexing TLS connections via SNI.

apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: tls-route
spec:
  hostnames:
  - example.com
  parentRefs:
  - name: traefik-gateway
  rules:
    - backendRefs:
        - name: backend-tls
          kind: Service
          port: 443

Introduced with Gateway API v1.2, Backend Protocol (through the appProtocol field) can now be set to specify the underlying protocol when a Route's backend references a Kubernetes Service. Currently, you can choose between http, https, kubernetes.io/h2c for HTTP/2 without TLS and kubernetes.io/ws for websocket over HTTP.

apiVersion: v1
kind: Service
metadata:
  name: test
spec:
  selector:
    app: test
  ports:
  - protocol: TCP
    appProtocol: kubernetes.io/h2c
    port: 8080
    targetPort: 8080

The ResponseHeaderModifier filter is now supported since Traedfik v3.2. It allows to setsetting a custom header for all responses being sent.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-response-header
spec:
  hostnames:
    - response.header.example
  rules:
    - backendRefs:
      - name: example-service-beta
        weight: 50
        port: 80
        # set a custom header for all responses being sent from the
        beta build of the backend server.
        filters:
           - type: ResponseHeaderModifier
             responseHeaderModifier:
               add:
                 name: build
                 value: beta
      - name: example-service-stable
        weight: 50
        port: 80

To learn more about how Traefik supports Gateway API, we encourage you to visit the Traefik Documentation.

Other Improvements

In addition to these exciting new features, Traefik v3.2 brings several updates to ACME certificate management (Let’s Encrypt), including the ability to use different email addresses in certificate resolvers (#11019 by Emrio), the support for custom CA certificates in certificate resolvers (#10816 by ldez), and a 30-days certificatesDuration threshold (#10970 by luker983).

BasicAuth support has been added to the Docker and Swarm endpoints (#10776 by 985492783).

Middlewares have been improved with IPv6 subnet support in ipStrategy for RateLimit and InFlightReq (#9747 by michal-kralik), an option to log the user in the ForwardAuth middleware (#10833 by GaleHuang), a new compression encodings option (#10943 by wollomatic), and the ability to mirror the full body in the mirroring (#11032 by MatteoPaier).

OpenTelemetry metrics now allow you to configure service.name for scenario with several Traefik instances (#10917 by cmartell-at-ocp), while access logs now capture trace IDs and EntryPoint span IDs (#10921 by weijiany).

A configurable maximum request header size option has also been added, allowing users to specify the maximum size for HTTP request headers beyond the default 1MB limit (#10995 by lucasrod16).

For detailed information on all changes, please refer to the full release notes on GitHub.

We extend our gratitude to all contributors for their efforts and dedication, which has made Traefik 3.2 an exceptional release. Thank you for driving Traefik forward!

Moving Forward

Traefik Proxy v3.2 is a major leap forward in performance, functionality, and security. This release introduces a groundbreaking Fast Proxy engine, which delivers unparalleled speed and efficiency. As a key contributor to the Kubernetes Gateway API project, Traefik Labs ensures cutting-edge support for the latest Gateway API v1.2 specifications. Along with many other enhancements, Traefik 3.2 empowers developers and organizations to effortlessly build and manage modern, cloud-native applications with unprecedented efficiency.

We encourage you to explore the new features and capabilities of Traefik 3.2 and experience the impact they can make in your infrastructure. Join the vibrant Traefik community, share your feedback, and help us shape the future of cloud-native networking.

Useful Links

Traefik 3.2 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Deepening The Synergy: HashiCorp and Traefik Labs’ Journey Toward a Seamless Cloud-Native Stack

Emile Vauge — Mon, 14 Oct 2024 15:28:44 GMT

Cloud-native infrastructure is evolving rapidly, with agility, security, and scalability increasingly becoming key table stakes. As such, organizations are constantly seeking integrated solutions to streamline their operations, and HashiCorp and Traefik Labs are one of the best examples of a deep integration between two stacks. What began with early integrations, with Traefik Proxy and Hashicorp Consul, has matured into a deep symbiosis of technologies with the addition of Traefik Enterprise, Traefik Hub, Hashicorp Vault and HashiCorp Nomad. Traefik Labs being recognized as the “2022 HashiCorp Integration Partner of the Year” showcases this strong unity that helps enterprises architect modern cloud-native infrastructures with ease.

The Early Days

The foundation of this partnership was laid with the integration of HashiCorp Consul into Traefik Proxy, Traefik Labs’ open-source reverse proxy and ingress controller. As early as Traefik’s inception back in 2015, the team at Traefik Labs was embracing service discovery and dynamic configuration patterns. Obviously, Consul from HashiCorp, was a natural fit.

At its core, Consul is designed to handle service discovery, health checks, and distributed system coordination across environments, making it the perfect complement to Traefik’s dynamic routing capabilities. Consul is integrated as a KV store and as a service catalog within Traefik, offering the maximum flexibility to fit companies’ needs.

As enterprises adopted cloud-native infrastructures, this combination of Traefik’s efficient ingress management and Consul’s robust service discovery made the duo indispensable in production environments.

Growing the Partnership

The expansion of both companies’ business offerings created the opportunity to deepen this collaboration, particularly with the introduction of Traefik Enterprise and HashiCorp’s Vault and Nomad into the mix.

Traefik Enterprise & HashiCorp Vault: Security has become an increasingly critical concern for organizations as they shift to microservices-based architectures. Vault, HashiCorp’s flagship solution for secrets management, integrates seamlessly with Traefik Enterprise, enabling enterprises to securely manage sensitive data. Companies can leverage Vault PKI as a TLS certificate resolver in Traefik Enterprise or can unlock the potential of a fully distributed ACME (Let’s Encrypt) setup. This is particularly essential in highly dynamic production environments, where breaches of sensitive data can be catastrophic.

Traefik Enterprise & HashiCorp Nomad: HashiCorp’s Nomad, a highly efficient workload orchestrator, became another pivotal piece in the collaboration. Designed to work across cloud and on-premises environments, Nomad excels at scheduling and managing containerized services at scale. Paired with Traefik Enterprise, this combination ensures that workloads are automatically discovered and routed without downtime or manual updates.

Nomad’s orchestration, when aligned with Traefik’s smart routing, delivers a streamlined solution for managing microservices, making deployment and scaling more seamless than ever before. This integration is particularly valuable for organizations looking for flexible, multi-cloud deployment options while maintaining high performance and operational efficiency.

Together, these integrations create a secure, highly automated, and flexible platform that enables enterprises to deploy, scale, and manage applications across any environment. With Traefik Enterprise and HashiCorp’s Vault and Nomad, organizations can achieve greater levels of automation while maintaining security, simplifying operations, and minimizing human intervention.

Maturing the Partnership

Traefik Hub & HashiCorp Vault: The maturity of the HashiCorp-Traefik Labs partnership reached a new level with the introduction of Traefik Hub, the industry's first Kubernetes-native API Management solution for publishing, securing, and managing APIs. Traefik Hub represents the most advanced solution in Traefik Labs’ portfolio, providing end-to-end API management that supports complex environments at scale. Built on the foundations of Traefik Enterprise’s proven routing technology, Traefik Hub API Gateway integrates deeply with Vault to offer an automated TLS certificate PKI for all APIs handled by Traefik Hub. The Hub & Vault combo provides the most advanced and secured API Gateway solution on the market.

Traefik Proxy/Hub & HashiCorp Nomad: The Nomad integration into Traefik Proxy and Traefik Hub continues to be improved and the latest Traefik v3.2 release brings a new watch option to the Nomad provider which refreshes the configuration on a per-event basis instead of the less effective pull mechanism.

Finally, both companies have been working hand in hand to publish a workload modernization integration guide with Traefik Enterprise to embed an API Gateway into Hashicorp Nomad clusters running Consul and Vault. The idea was to create a definitive guide for operation, platform and security teams to deploy a modern platform to seamlessly and securely publish APIs.

The impact? Outstanding success stories from clients across the world, highlighted recently by BT Group:

“We adopted Traefik OSS to help build a new container platform on HashiCorp Nomad in AWS, following the closure of one of our on-prem data centers. We needed something that integrated smoothly with Consul and had minimal overhead while handling spikes in demand during major events. Traefik's seamless integration with Consul made it easy to implement, and it’s been so reliable that we rarely have to think about it. As our platform grew, we needed to support more use cases around security and wanted operational support, which led us to adopt Traefik Hub. Being able to secure our APIs quickly by adding a few Consul tags has been a huge time-saver—our teams can focus on development without worrying about complex configuration.” ⎯ Tom Davies, Principal Software Engineer, BT Group

Conclusion

The deep integration of HashiCorp and Traefik Labs stacks has been transformative for enterprises navigating the complexities of cloud-native environments. With the synergy between Traefik Enterprise, Vault, and Nomad already providing robust solutions for secure microservices orchestration, the introduction of Traefik Hub represents a leap forward in API management.

This partnership continues to evolve, driving innovation in cloud-native technologies and creating tools that allow enterprises to thrive in an era where security, automation, and flexibility are key. As this collaboration deepens, the future of cloud-native infrastructure looks more agile, resilient, and secure than ever before. Visit our Traefik & HashiCorp page to learn more.

]]>

Implementing Runtime API Governance in Traefik Hub

Immánuel Fodor — Wed, 18 Sep 2024 03:41:00 GMT

In a recent blog post, we discussed the top five policies for runtime API governance. Now, let’s look at how Traefik Hub implements these policies. Traefik Hub is a Kubernetes-native API management solution built with scalability, flexibility, and simplicity in mind. Traefik Hub helps route traffic to services in dynamic, microservices-oriented environments.

In the following sections, we will provide examples of configuration implementation in Traefik Hub to support the standards we have discussed. But note that our examples are not exhaustive - Traefik Hub supports many more capabilities and configuration options!

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments

As a Kubernetes-native solution, Traefik Hub supports declarative resource configuration. Traefik Hub configuration can be done through Kubernetes resources like Ingress and Service, and Traefik Custom Resource Definitions (CRDs) like IngressRoutes, Middleware, API, APIPortal, APIAccess, APIVersion, and APIRateLimit. Configuration can be version-controlled in Git and automatically applied to the Kubernetes cluster using GitOps tools like ArgoCD and Flux.

For example, defining external traffic routing to a service involves creating an API CRD and exposing the API with an IngressRoute. The following resource definition declares an API called hello-api, and provides the path to its OpenAPI description.

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
name: hello-api
namespace: apps
spec:
openApiSpec:
path: /openapi.yaml

Next, specify an IngressRoute CRD with routes to connect incoming requests to services that can handle them. The following resource definition declares a router that matches a request with the host api.example.com to the hello-api-service.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: hello-api-ingress
namespace: apps
annotations:
hub.traefik.io/api: hello-api # Name of the API object
spec:
routes:
- match: Host(`api.example.com`)
kind: Rule
services:
- name: hello-api-service
port: 8080

Developers can save configurations to a Git repository, and GitOps operators ensure that these configurations are consistently applied across environments. The diagram below illustrates this process.

The declarative nature of Traefik configuration, combined with GitOps workflows, supports advanced and progressive deployment capabilities. Teams can follow blue-green deployments and canary releases using GitOps operators like ArgoCD and FluxCD. If any issues are detected, changes can be rolled back to a stable version by reverting the changes in the Git repository.

TIP: For an in-depth treatment on how to do load-balancing in Traefik and support advanced deployment techniques, see Traefik’s Advanced Load Balancing course.

Policy 2: Make comprehensive and correct API documentation available to users

Traefik Hub API management includes an API Developer Portal to make API documentation accessible to developers and other API users. In the portal, users can explore available API endpoints, understand their usage, and test them in real time.

The APIPortal CRD creates and configures the portal, generating a web interface for browsing the API documentation. The visibility of API documentation depends on user groups. An API will be visible to an API consumer if they belong to a specified group with access to the API, as configured by the APIAccess CRD.

You can create an API developer portal by applying an APIPortal resource as follows:

---
apiVersion: hub.traefik.io/v1alpha1
kind: APIPortal
metadata:
name: my-portal
namespace: default
spec:
title: "My Portal" # The title for the Portal
description: "API documentations" # A short description of the Portal
trustedUrls:
- "https://portal.example.com"
ui:
logoUrl: https://traefik.io/favicon.png # URL to a picture used as logo

To expose the portal, create an IngressRoute resource, as demonstrated below:

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: my-portal-ingress
namespace: traefik-hub
annotations:
hub.traefik.io/api-portal: my-portal@default # Reference the APIPortal
spec:
routes:
- match: Host(`portal.example.com`)
kind: Rule
services:
- name: apiportal
port: 9903

Policy 3: Secure all API traffic from malicious and unauthorized access

With Traefik Hub, you can ensure secure API access using protocols such as Oauth2, OpenID Connect (OIDC), API keys, and JSON Web Tokens (JWTs). Additionally, Traefik integrates with identity providers such as Keycloak and Okta. Traefik also integrates with any other IdP that supports the OIDC protocol to handle user identities and authorize access to the APIs and API portals.

To protect APIs from DDoS attacks or excessive requests from a single client, you can implement rate limits by defining an APIRateLimit resource. Traefik allows you to do this for all APIs or specific user groups. The following declarative config defines a rate limit of 100 requests per minute.

apiVersion: hub.traefik.io/v1alpha1
kind: APIRateLimit
metadata:
name: my-rate-limit
namespace: apps
spec:
# Rate limit configuration, this config allows 100 requests/minute.
limit: 100 # 100 requests
period: 1m # One minute
groups:
- support
apiSelector:
matchLabels:
module: crm

Using Traefik’s Distributed Rate Limit (which is based on the Token Bucket algorithm), you can limit requests over time across your entire cluster rather than just an individual proxy.

In addition, Traefik Proxy v3 includes the Coraza Web Application Firewall (WAF) integrated as a plugin. Coraza is an open-source OWASP project and a high-performance WAF that supports Modsecurity's seclang language and OWASP core rule sets. Enabling this involves two simple steps:

- First, updating Traefik’s static configuration that loads the plugin as follows:

experimental:
plugins:
coraza:
moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
version: v0.2.2

- Then, updating the WAF CRD as follows:

---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: coraza-waf-block-admin-path
namespace: apps
spec:
plugin:
coraza:
directives:
- SecRuleEngine On
- SecDebugLog /dev/stdout
- SecDebugLogLevel 9
- SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"

Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes

Traefik Hub helps you understand the impact of API configuration changes by performing static checks on manifest files. It includes a Static Analyzer tool that allows you to check your Traefik Hub manifest files for consistency and quality. The Static Analyzer can lint manifest files and generate differential reports. It can be used as a CLI tool or as part of a CI/CD pipeline. The linter can detect issues such as:

Childless resources
Duplicate resources and resource references
Unknown operation sets
Orphan resources
Invalid resource references
Invalid regular expressions
Duplicate releases of a given API
Invalid resource definitions
Invalid selector definitions

The following is an example of the linting error message displayed on the PR that introduces a change.

The Static Analyzer tool can also run a diff to generate change reports. For example, a report like the following makes it clear that the rate limit was changed. It shows the user group and APIs affected by the change. If that was not the developer’s intention, the developer can take remedial action.

To aid API evolution, Traefik Hub also supports several API versioning schemes. Teams can define and use an API version based on host, URI path, HTTP method, query parameters, media/content type headers, custom headers, client IP, auth user (basic/forward auth), JWT claims, and more. Traefik even allows teams to mix different versioning schemes based on logical expressions.

Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery

Traefik Hub provides metrics and tracing information using the OpenTelemetry (OTel) format, to help teams monitor and gain insights into the traffic flowing through their services and APIs. Traefik Hub has the most comprehensive OTel support in the industry, with over 20 metrics and 15 labels. Additionally, it supports vendor-specific metric systems such as Prometheus, Datadog, InfluxDB, and StatsD.

For instance, to set up the OTel metrics, first get the configuration values for the Traefik Hub Helm release:

helm get values traefik-hub -n traefik | tail -n +2 > values.yaml

Then, modify and apply the configuration values in values.yaml.

metrics:
# Disable Prometheus (enabled by default)
prometheus: null
# Enable providing OTel metrics
otlp:
enabled: true
http:
enabled: true
endpoint: http://myotlpcollector:4318/v1/metrics
# Enable providing OTel traces
tracing:
enabled: true
http:
enabled: true
endpoint: http://myotlpcollector:4318/v1/traces

By integrating with monitoring systems, you can configure alerts and notifications for critical metrics and performance indicators. Traefik can also be connected to pre-built Grafana dashboards to visualize a wide range of API metrics. This allows you to correlate deployment events with incident management, enhancing your ability to monitor and respond to issues effectively and improve MTTR. For more details on Traefik Hub’s integration with Grafana, see this webinar.

Additionally, Traefik provides a built-in dashboard that offers a real-time view of the current status of routes, services, middleware, and other components.

GitOps Implementation Example

The first policy we highlighted at the beginning of this article was about providing a developer-friendly, traceable, and easy-to-rollback approach to API deployment. We also mentioned in Standard 1.2 that GitOps was an implementation of this approach. In the Traefik Hub GitOps Tutorial project in GitHub, we provide a hands-on tutorial to demonstrate the power of GitOps for API configuration deployment and change management. It also demonstrates Traefik Hub’s integration with API monitoring and observability visualization tools. The tutorial uses the following tools:

Aspect	Tool
API Management	Traefik Hub
Software platform	Kubernetes cluster, running locally in kind or k3d
GitOps Deployment Tool	FluxCD
Git repository	GitHub
Metrics visualization	Grafana
API monitoring and alerting	Prometheus

To work on this project, request a Traefik Hub trial here. Then follow the instructions on the README page, clone the repository, and set up kind and FluxCD. In the tutorial, FluxCD watches your fork of the GitHub repository for changes. When a change to the master branch is detected, it deploys the latest version of the configuration file to the cluster.

Summary

In the part 1, we introduced five essential policies for effective runtime API governance:

A developer-friendly API deployment process
Comprehensive and correct API documentation available to users
Secure both ingress and egress API traffic,
Clear API change impact analysis
Proactive API monitoring and deep observability.

By establishing standards based on these policies, organizations can enforce runtime API governance more effectively. The primary objective of improving runtime API governance is to improve software delivery performance (as measured by the DORA metrics - DR, LTC, CFR, and MTTR). It also aims to enhance the API developer experience for both API producers and API consumers.

To successfully implement these runtime governance policies, it's crucial to leverage a modern API management platform, such as Traefik Hub, which emphasizes dynamic change management and runtime API governance at its core. As a Kubernetes native API management solution, Traefik Hub is optimized for end-to-end API delivery automation which improves operational efficiency. To learn more, get a personalized demo of Traefik Hub today.

]]>

Top 5 Policies for Runtime API Governance to Ensure Your APIs are Consistent, Compliant, and Secure

Ikenna Nwaiwu — Wed, 18 Sep 2024 03:15:03 GMT

API governance provides a framework for defining policies and standards to ensure APIs are consistent, compliant, and secure. However, a common misconception about API governance is that it is solely about API design concerns, such as following a specified pagination pattern or ensuring API descriptions adhere to specific linting rules. While these design-time considerations are important, they’re only part of the picture. API governance is not just about defining design standards; it’s the backbone of ensuring that APIs are consistent, compliant, and secure across their entire lifecycle.

Overlooking runtime governance—like monitoring API performance, managing access controls, and enforcing security protocols—can lead to catastrophic failures in production. For instance, imagine a beautifully designed API that lacked proper runtime governance. Without robust security measures, attackers could exploit it, leading to data breaches and significant financial loss. Or consider an API with no runtime monitoring; a sudden surge in traffic could cause unanticipated outages, bringing down critical services and eroding user trust. Effective API governance addresses design and runtime concerns, ensuring that APIs meet standards during development and perform reliably and securely in the real world.

API management teams are responsible for defining and enforcing runtime API governance. Runtime API governance is the structured framework that manages and enforces policies for API deployment, security, observability, and lifecycle during their operational phase. It ensures APIs are deployed in a traceable and secure manner, with all configurations version-controlled and changes managed through a standardized process. This governance also mandates that live APIs be centrally documented, monitored for performance, and secured via gateways and firewalls. Overall, API governance provides a controlled environment to maintain API integrity, security, and efficiency in real-time operations.

Given how crucial API governance is to an organization’s API strategy's success, let's explore the topic further. In this post, we discuss five policies we recommend for API teams. They are listed below.

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments
Policy 2: Make comprehensive, correct API documentation available to users
Policy 3: Secure ingress and egress API traffic from malicious and unauthorized access
Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes
Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery.

Each policy has one or more actionable standards that teams should adopt to meet these criteria.

Before we dive in, here is a note on the terminology we use in this post. By policy, we mean a high-level statement that requires compliance with one or more associated standards. A policy focuses on results and not implementation. On the other hand, by standards, we mean detailed mandatory, recommended, and optional rules used to gauge compliance with a given policy. Standards are specific and measurable, using capitalized keywords from IETF RFC-2119 (‘MUST’, ‘MUST NOT’, ‘SHOULD’, ‘SHOULD NOT’, ‘MAY’ etc.) to indicate the level of requirement compliance.

Policy 1: Use a developer-friendly, traceable, and easy-to-rollback approach for API deployments

The DORA metrics, Deployment Frequency (DF), Lead Time for Change (LTC), Change Failure Rate (CFR), and Mean Time to Recovery (MTTR), are widely used to assess a software team's performance. However, there's now an increased emphasis on developer experience and productivity. The State of Developer Experience Report 2024 revealed that 69% of developers are losing 20% or more of their time due to inefficiencies at work. The survey also explored the key areas that engineering leaders believe will enhance developer productivity and satisfaction. One of the top five areas identified was improving collaboration tooling.

When making changes to your API, it's crucial to have a deployment process that allows for fast deployment, easy experimentation, and the ability to roll back failed deployments and restore service quickly. Such a deployment process requires an automated deployment system to minimize manual errors. A deployment process like this is a key enabler for reducing LTC and improving software delivery performance. Additionally, the deployment process should facilitate seamless collaboration between development and operations teams and provide robust access controls to determine who can make changes. Furthermore, the process should also maintain an auditable history for traceability and accountability.

Here are three standards that can help realize this policy.

Governance Standard 1.1: All API descriptions and configuration MUST be version-controlled in Git

Managing API descriptions (like OpenAPI) and API configuration files (like Kubernetes Ingress manifests) in Git enables a workflow that provides a clear audit trail, giving a history of what was changed, when, and who changed it. This traceability makes it easier to troubleshoot changes and roll back a change that introduces a failure with a single Git revert command. It also enables developers to edit the files using editors, CLIs, and any preferred tool.

Governance Metric: The ratio of the organization's APIs whose configuration is under version control.

Governance Standard 1.2: All API configuration changes MUST follow a GitOps approach

GitOps has emerged as the modern way to manage application and infrastructure deployments. As part of the GitOps approach, the application state is managed in Git repositories, which act as the source of truth for the system's desired state. Application state is stored using declarative descriptions in formats like YAML and JSON. When the configuration files are updated, continuous delivery tools synchronize the application's live state with the desired state.

A GitOps approach allows for the fast delivery of API changes. Automated governance checks in the deployment pipeline enable organizations to achieve fast LTC and improved API consistency. Pull requests can be a lightweight governance technique to promote collaboration on changes. Teams following the GitOps approach disallow configuration changes from the application UI, eliminating error-prone TicketOps and ClickOps-based API deployments.

Governance Metric:

Number of API configurations managed with GitOps.
API config change review speed: The average time it takes from opening a config change PR to completing a review on it.

Governance Standard 1.3: The CI/CD pipeline MUST act as the primary change agent, driving all changes, feedback, and collaboration through automated processes, with no manual steps outside the pipeline.

A CI/CD pipeline should include all the necessary steps to validate and deploy working software to production. These steps involve compiling and packaging source code, conducting unit, integration, and acceptance tests, and deploying to progressively higher environments. In this way, the pipeline acts as both a central falsification mechanism and a primary change agent, ensuring that all changes, feedback, and collaboration are processed through automated workflows, with no manual intervention. If a new software change (that is, a commit) presents the hypothesis that “this change complies with standards and does not break working software,” then the CI/CD pipeline serves as a falsifiability test to evaluate that hypothesis.

By automating collaboration patterns, the pipeline ensures that feedback loops are consistent and transparent across teams. In this way, the CI/CD pipeline becomes not only an objective evaluator of changes but also a facilitator of cross-team feedback and collaboration. No manual steps should occur outside of the pipeline, ensuring that all changes and collaboration are fully automated and trackable. By automating previously manual processes, the pipeline evolves into an automated knowledge base that outlines the steps required to validate working software.

Such a pipeline has a profound effect on the team’s culture. Changing the pipeline changes the culture. Whenever a new standard is introduced, the team must first ask, “Is it possible to build this into the pipeline to solidify our expectations of collaboration from others?” The development team can refer to the pipeline logs when a build fails. The platform and infrastructure teams can use data produced by the pipeline to verify whether the software has been successfully deployed. The security team can review the security scan and testing results from the pipeline to validate a release candidate, and so on.

Governance Metric: % of changes validated through the pipeline vs. not

Policy 2: Make comprehensive and correct API documentation available to users

Large enterprises often struggle with API sprawl, which refers to the uncontrolled proliferation of APIs. API sprawl usually occurs when there is a lack of proper documentation and centralized API governance. Simply put, the organization may not know where all its APIs are located or what they are used for. API sprawl hinders integration and the exploitation of new business opportunities and poses a security risk. This risk can stem from Shadow APIs, which are used without the knowledge of the API governance teams and exist outside the established governance process. Additionally, the security risk can result from Zombie APIs that are no longer actively maintained, supported, or monitored but are still functional and accessible to users.

To combat API sprawl, a comprehensive API discovery mechanism should be implemented. API discovery is about making it easy for anyone in the organization to find any API they need. When people can locate the available APIs within an organization and understand what they do, it helps promote API consistency and reuse and reduces duplication of effort.

API documentation documents (that is, API descriptions like OpenAPI) also have to be comprehensive and accurate. Nothing produces an annoying user experience, and reduces trust in API providers, like available but inaccurate API documentation. Therefore it is important that runtime platforms are able to validate that API documentation matches the runtime traffic.

Governance Standard 2.1: Edge technologies SHOULD detect undocumented APIs

One way to manage API sprawl is to have policies and controls in place to detect undocumented and unmanaged APIs. Detecting APIs can be done by monitoring traffic at the edge. This detection can be one of the first steps for organizations suffering from API sprawl. From there, the organization can collate the documentation and store it in the internal API catalog, which leads to the next standard.

Apart from detecting undocumented APIs, runtime API platforms can also validate the correctness of API documentation by comparing it to run time traffic. This runtime validation mitigates API drift and is a security measure that identifies unexpected and potentially malicious API behaiviour.

Governance Metric:

Number of unmanaged APIs detected by edge controls.
Number of API description document defects

Governance Standard 2.2: All live APIs MUST be documented in a central API catalog

As part of your API deployment pipeline, API documentation should be stored centrally in an internal API catalog or registry. Additionally, a selection of these APIs can be made available to partners or consumers through an external developer portal. The catalog should have a browseable UI for users, but it should also provide API registry features. That is, it should provide the API descriptions for programmatic access by machines.

Governance Metric: The number of APIs in the API catalog.

Governance Standard 2.3: The API management platform SHOULD support self-service onboarding

Along with making APIs discoverable, users should be able to request and obtain access to an API quickly without jumping through administrative hoops. Self-service onboarding of APIs by obtaining an API key through a developer portal is one of the runtime requirements for a great developer experience for API users.

Governance Metric: Number of APIs documented in a self-service developer portal.

Policy 3: Secure ingress and egress API traffic from malicious and unauthorized access

The rapid growth of APIs and API traffic has exponentially expanded the attack surface for malicious actors. In its State of API Security Report 2024, Salt Security states that 95% of respondents in the research experienced security problems in production APIs, with 23% experiencing a breach. 25% of respondents also stated that one of their biggest concerns with their company’s API program was that it didn’t adequately address runtime or production API security.

It is crucial to protect APIs and invest in API runtime protection to guard against threats resulting from authentication weaknesses, sensitive data exposure, and account takeover or misuse. Public APIs can especially face a host of attack vectors—from SQL and code injection to cross-site scripting (XSS) and other OWASP Top 10 API vulnerabilities. They can also face bot attacks ranging from data scraping and intellectual property theft to account takeover and distributed denial of service (DDoS).

While organizations need to secure the APIs they build, they also consume APIs from third parties. These third-party APIs include APIs of external LLMs, CRM systems, payment processing platforms, messaging APIs, and more. However, the unsafe consumption of APIs is an OWASP API Security Top 10 threat. It arises when an application, such as an API, consumes external, third-party APIs and blindly trusts the data they return without adequately validating or sanitizing the data it receives. This exposes the application to SQL injection, code injection attacks, or the exposure of sensitive information.

Here are a few standards you can implement to achieve this policy in practice.

Governance Standard 3.1: All governed APIs SHOULD be exposed through an API gateway configured with appropriate security controls

API gateways provide control points for runtime API governance, enabling governance teams to enforce common runtime standards such as authentication and authorization, granular API access, rate limiting and throttling, input validation, and monitoring. API gateways can also integrate with an identity provider (IdP) to provide user authentication and authorization.

Governance Metric:

The number of governed APIs not exposed through the gateway
The number of APIs with insufficient authentication

Governance Standard 3.2: All public APIs MUST be protected by a WAF

A web application firewall (WAF) actively scans and filters incoming HTTP traffic. It blocks malicious API requests in real-time, typically using a set of threat signatures—pre-defined rules or patterns that identify malicious activity. In addition to maintaining a set of threat signatures, some WAFs use machine learning and behavioral analysis for pre-emptive protection, minimizing the need to update signatures when a new threat is identified continuously. WAFs may integrate with a global network of sensors to provide real-time threat intelligence on emerging threat patterns. WAFs can also provide detailed audit logs and reports required to meet various industry compliance requirements, such as PCI DSS, HIPAA, and GDPR.

Governance Metric: The number of public APIs unprotected by a WAF.

Governance Standard 3.3: Third-party API traffic MUST be validated

While organizations may put a lot of thought into securing the APIs they provide externally (such as encrypting the traffic and routing it through a gateway), they may not pay as much attention to the security of the APIs they consume. That is, API requests to third parties. Developers tend to trust the data from third-party APIs they consume, especially if they are provided by well-known companies. As such, they may adopt weaker security standards for API consumption.

A way to mitigate the security risk from third-party API traffic is to ensure third-party API traffic is encrypted, validated, and sanitized. Organizations can proxy the traffic through an API gateway that acts as an egress API gateway (also called a consumption gateway or reverse gateway). In the egress gateway, teams can use strong API contracts from the third party to validate that the traffic complies with the types, formats, and constraints defined in the contract. The egress gateway can also be a control point for implementing robust error handling and preventing the exposure of sensitive data. This egress gateway pattern is illustrated in the figure below.

Governance Metric: The number of unvalidated third-party APIs.

Policy 4: Provide precise API change impact analysis to minimize service failure and breaking changes

Change impact analysis involves identifying the potential consequences of a change, the components affected, and the possible risks associated with the change. It helps developers identify areas that can be affected by a change so that changes do not introduce system stability, performance, and security issues.

A way to analyze the impact of an API configuration deployment is to perform a static code analysis of the deployment artifacts. This analysis can help build a representation of a change and visualize its impact and any dependencies that may also be affected. For example, a static change analysis check may show that a rate limit change may affect not just one API but more. It may also show that changing a rate limit for a user group affects multiple groups. Such insights help reduce CFR.

Another way to analyze the impact of an API change is by detecting breaking changes, which can disrupt API users. A breaking change is a non-backward compatible change that forces API consumers to change their code to ensure their API integrations keep working.

Let’s look at two standards an organization can use to implement this runtime governance policy.

Governance Standard 4.1: All API configuration changes must pass static checks

One advantage of GitOps is that API configuration is stored in version control. This allows the team to run a static analysis on API configuration in the CI/CD pipeline. The analysis ensures that configuration files are valid and prevents any API misconfiguration from arising. It also helps the dev team understand the impact and scope of their changes on different APIs and API consumer groups.

Governance Metric: The ratio of the organization’s APIs whose configuration is under version control and change impact analysis checks.

Governance Standard 4.2: All APIs must follow the recommended versioning scheme

API versioning helps providers evolve an API and add new versions without breaking existing integrations. Many different versioning schemes exist for REST APIs (such as using the URI path, query parameters, custom headers, content type, and more). Your runtime infrastructure must support the versioning mechanisms that best fit your requirements. As an API provider, defining and adopting a consistent versioning mechanism for your APIs gives users a clear migration path to a new version and helps you manage the lifecycle of the API. Be sure to run breaking change checks before deployments to ensure no breaking changes are introduced.

Governance Metric: The number of APIs that do not follow the versioning scheme.

Policy 5: Provide proactive API monitoring and comprehensive, granular observability to reduce time to recovery

API monitoring involves tracking key metrics of API performance, such as response times and error rates. API Observability goes beyond monitoring a known set of API performance metrics. Observability allows API publishers to troubleshoot novel problems and understand what is happening inside an application. Observability is based on the ability of applications and infrastructure to emit telemetry signals: metrics, logs, and traces (MLT), which API publishers can analyze. API publishers can run distributed traces using telemetry, observing requests as they flow through a distributed system. This tracing gives them visibility into a system’s health and enables debugging the behavior of the production system.

API observability should be based on open standards for instrumenting, generating, collecting, and exporting telemetry data. Open observability standards like OpenTelemetry guarantee compatibility with various backend API analytics platforms and prevent vendor lock-in. They provide a way to decouple your observability strategy from your gateway strategy. Open standards also provide teams with a common language for describing cross-application interactions and troubleshooting.

Governance Standard 5.1: Teams MUST provide dashboards and configure alerts to monitor API request rate, error rate, and request latency

Having runtime visibility into an API’s performance is essential not only for understanding its usage but also for incident mitigation. API publishers should capture API metrics on uptime, requests per minute, latency, and errors per minute. They should also set up monitoring dashboards and proactive alerts to make API performance information available for the teams that support them.

Governance Metric: The number of APIs not set up for monitoring.

Governance Standard 5.2: APIs SHOULD use OpenTelemetry to publish telemetry data

OpenTelemetry is a widely supported open-source observability framework that has become the de facto standard for cloud-native applications. OpenTelemetry is built on rules and conventions known as the OpenTelemetry Protocol (OTLP). This protocol dictates how components related to OpenTelemetry exchange telemetry data between a source system that generates the data and its destination. The OTLP data model consists of traces, which describe the flow of execution; metrics, which offer statistical information on application performance; and logs, which provide detailed information on the application's state.

Governance Metric: The ratio of APIs instrumented with OpenTelemetry.

Governance Standard 5.3: API deployments SHOULD emit telemetry data

Application performance telemetry can be correlated with API deployment events to quickly visualize and investigate any API performance problems resulting from new deployments. Deployment event correlation with API runtime metrics is a powerful incident mitigation technique. This event correlation lets teams see if a deployment introduces problems and roll back the change.

Governance Metric: The ratio of API deployment pipelines that publish telemetry data.

Let's now explore how these runtime API governance policies are implemented in practice with Traefik Hub in Part 2 of this blog post.

]]>

Stop SQLi and XSS Attacks Easily with Traefik's WAF Integration

Immánuel Fodor — Tue, 03 Sep 2024 18:25:03 GMT

In today’s digital landscape, web applications and APIs are integral to virtually every aspect of modern life. From the familiar world of online shopping and social media to the behind-the-scenes functionality of mobile apps, government websites, and even entertainment platforms, APIs serve as the backbone of online interactions.

However, this ubiquity also makes them prime targets for cyber threats, and securing web applications and the flow of data that keeps our world running is a critical concern for enterprises. Malicious actors constantly search for vulnerabilities to exploit, aiming to steal sensitive data, disrupt operations, or even inject malware into unsuspecting systems. The exponentially increasing threat landscape demands a robust defense strategy, and that's where Web Application Firewalls (WAFs) come into play.

The Role of WAFs in API Security

WAFs are the first line of defense for your web-based applications and their APIs. They stand guard, meticulously examining every incoming request, ensuring only authorized and legitimate data passes through. WAFs prevent attackers from submitting malicious or unwanted data to applications as well as from exfiltrating sensitive data. Here's how they safeguard your business from a variety of threats:

Known attack detection: WAFs recognize the signatures of common attack strategies employed by cybercriminals. These include attempts to exploit vulnerabilities like SQL injection (SQLi), Cross-Site Scripting (XSS), file inclusions, code injection, session hijacking, and even metadata or error leaks. By identifying these known threats, WAFs can effectively block them before they can cause you any damage.
Beyond the obvious: Malicious activity isn't always so straightforward. WAFs can also detect abnormal or malformed requests deviating from regular user behavior. These red flags could indicate a more sophisticated attack, allowing the WAF to take preventative measures.

This level of protection provided by WAFs is crucial. But it doesn't stop there. WAFs offer additional functionalities to enhance security further:

Virtual patching: Sometimes, vulnerabilities in your system might be identified before a permanent fix is available. WAFs can provide temporary protection by blocking exploits targeting those vulnerabilities, effectively plugging the hole until a proper patch can be deployed.
Blocking malicious bots: The internet is rampant with automated bots. While some bots serve legitimate purposes, others can be malicious, launching denial-of-service attacks or scraping valuable data. WAFs can help filter out these malicious bots, keeping your APIs safe from unwanted traffic.
Audit logging: Staying informed about potential security breaches is critical. WAFs provide valuable audit logs detailing matched rules and anomaly scores, allowing your security team to analyze potential threats and take appropriate action.

Beyond Technology: WAF as a Legal Necessity

In addition to protecting against cyber threats, WAFs are critical in helping enterprises meet various security standards and regulatory requirements. Many industries are governed by strict regulations that mandate specific security controls to protect sensitive data. Some of the most common standards that require or strongly recommend the use of WAFs include:

Payment Card Industry Data Security Standard v4: Under PCI DSS 4.0, organizations must deploy an automated technical solution for public-facing web applications that continually detects and prevents web-based attacks. This is articulated in Requirement 6.4.2, which explicitly states that WAFs are no longer just a recommended practice but a must for compliance.
Federal Risk and Authorization Management Program: FedRAMP requires stringent security controls for cloud service providers working with US federal agencies, including protections that WAFs can enhance.
Health Insurance Portability and Accountability Act: While HIPAA does not explicitly require WAFs in the healthcare sector, it mandates the protection of electronic protected health information (ePHI), which can be supported by using WAFs to prevent unauthorized access and attacks.
General Data Protection Regulation: Although GDPR requires appropriate technical and organizational measures to protect personal data, it does not explicitly mention WAFs. However, organizations should use them as part of their overall security posture to reduce the risk of data breaches that could lead to significant fines under GDPR.

To learn more about WAFs’ importance, check our detailed article: Why does WAF matter in API security?

However, keep reading if you’re curious about how Traefik can help you effortlessly mitigate these threats while complying with security standards.

A Free and Open-Source WAF in Traefik Proxy

The release of Traefik Proxy v3 marked a significant leap forward in enhancing application security through the introduction of two groundbreaking features: WASM-based plugins and the Coraza Web Application Firewall (WAF).

WASM (WebAssembly), a binary instruction format designed for efficient execution in web browsers, has found its way into the world of proxy servers. Traefik v3 embraces this technology, enabling developers to compile code in virtually any language into a portable format that runs seamlessly within the proxy environment. This innovation opened the door for a new breed of plugins, offering greater flexibility.

Capitalizing on this breakthrough, Traefik v3 introduced its first WASM-based plugin, the Coraza Web Application Firewall. Coraza is an open-source, Go-based WAF that has quickly become a favored choice for security-conscious enterprises. It stands out for its performance and endorsement by OWASP, a globally recognized authority on web application security.

Coraza is built on best practices and standards that are widely recognized in the cybersecurity community:

It understands and can enforce rules written in ModSecurity’s Security Language (seclang). This compatibility is a significant advantage for organizations that are already familiar with ModSecurity, as they can easily transition to using Coraza without needing to rewrite existing security rules.
Coraza comes equipped to enforce the OWASP Core Rule Set (CRS), a set of generic attack detection rules that protect against a wide range of threats. The CRS helps automate threat mitigation by blocking common attack vectors such as SQL injection, cross-site scripting (XSS), and other OWASP Top 10 vulnerabilities, saving valuable time and resources.

The WASM-based Coraza WAF in Traefik is best suited for:

One of the most compelling reasons to adopt a plugin-based WAF is to simply meet compliance requirements. A plugin-based solution like Traefik Proxy simplifies the process of implementing and managing security controls.
Furthermore, a plugin-based WAF allows for granular protection. You can selectively apply WAF rules to specific pages or applications, such as protecting sensitive administrative areas like /admin while leaving less critical pages unfiltered. This approach optimizes performance and reduces the risk of false positives.
A plugin-based WAF is particularly beneficial for organizations with internal applications that experience low traffic volumes but may still contain sensitive information. With Traefik Proxy, you can easily deploy and manage WAF protection for these applications without compromising performance or budget.
Finally, filling in the cybersecurity skills gap is a significant challenge for many organizations. Attracting and retaining skilled security personnel is expensive and time-consuming. A plugin-based WAF like Coraza in Traefik Proxy can help address this issue by providing user-friendly configuration and baked-in rule sets based on best practices.

A 2023 study from the Information Systems Security Association (ISSA) indicates that 71% of companies report being severely affected by the shortage of skilled cybersecurity professionals. A plugin-based WAF can help bridge this gap by simplifying the management of API security.

Let’s illustrate how simple configuring the Coraza WAF WASM plugin is! Add this to your Traefik’s static configuration to load the plugin:

experimental:
  plugins:
    coraza:
      moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
      version: v0.2.2

Then, leverage the WAF plugin in a middleware configuration. Using the Kubernetes CRD provider as an example:

---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: coraza-waf-block-admin-path
  namespace: apps
spec:
  plugin:
    coraza:
      directives:
       - SecRuleEngine On
       - SecDebugLog /dev/stdout
       - SecDebugLogLevel 9
       - SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"

Then accessing /admin will return a 403 error as specified by the configuration rule, but everything else will return 200 as there is no matching rule. You’ll see its power even more when combining it with IP address checking, header-based filtering, or chaining any other middleware or plugin available for Traefik Proxy.

The plugin-based WAF offers a versatile solution for a variety of use cases. Its ease of use but still powerful capabilities make it an essential tool for organizations looking to secure their web applications in a cost-effective and manageable way. But if needs evolve and you need to juice out as much filtering from your WAF as possible, you’ll need a bigger wall.

A Performance-Optimized WAF for High-Volume APIs in Traefik Hub

Security needs become more complex as companies evolve and scale their digital infrastructure. While traditional solutions like application proxies serve many essential functions, they often fall short regarding advanced security requirements like advanced authentication and authorization. This is particularly true for organizations that rely on WAFs to protect their web applications and APIs. For them, transitioning from a simple application proxy to a more robust API Gateway, like Traefik Hub, can offer significant advantages, including enhanced security and vastly improved performance.

Traefik Hub represents the next step in the evolution of serving APIs, offering a comprehensive platform that goes beyond the capabilities of an application proxy. As a modern API Gateway and API Management solution, Traefik Hub is designed to handle the complex needs of today’s enterprises, providing advanced features such as its native WAF integration, which directly addresses the performance limitations many organizations face when using WAF plugins in a proxy setup.

The Need for a High-Performance WAF

One of the key challenges with using a WASM-based WAF plugin in Traefik Proxy is the performance overhead associated with the plugin. While WASM offers a flexible and portable way to deploy WAFs, it introduces certain inefficiencies that can impact the overall performance of the application:

Boot-time overhead: Since WASM offers a secure runtime environment for plugins, the WASM binary associated with the WAF needs to be loaded at boot time for all middleware configurations. This loading process can delay application startup, especially in environments with complex configurations or high traffic demands.
Run-time overhead: Once loaded, the WASM-based WAF analyzes each incoming request and outgoing response in real time. This continuous analysis introduces latency, which can slow down the request processing and reduce the application's overall efficiency. The performance overhead can become a significant bottleneck with high traffic volumes, limiting scalability and increasing operational costs.

To address these performance challenges, Traefik Hub offers a native WAF integration that eliminates the inefficiencies associated with WASM-based plugins. The native integration in Traefik Hub results in a WAF that is 23 times faster than its WASM-based counterpart in Traefik Proxy, providing a significant performance boost for your applications. You’ve read it correctly, not 23%, but 23 times!

The key to this performance improvement lies in the architecture of the native WAF integration in Traefik Hub. Unlike WASM-based plugins, the Coraza WAF codebase is natively integrated into Traefik Hub at compile time. Since no WASM runtime is involved in running the WAF, it eliminates the boot-time and run-time overhead associated with the plugin-based solution, resulting in faster startup times and more efficient request processing.

Benefits of a Native WAF Integration

The native integration of the Coraza WAF in Traefik Hub offers several significant benefits:

Further enhanced security: The native WAF integration provides all the robust security features expected from Coraza, including protection against OWASP Top 10 vulnerabilities—but now with additional benefits. There’s no need to download the plugin during boot time, reducing the dependency on external components. This is perfect for air-gapped environments where external network access is restricted and enhances overall security by reducing the attack/risk surface.
Simplified management and configuration: With native integration, managing security configurations becomes more straightforward, as there’s no need for static plugin configuration. Instead, security rules and configurations are handled only within the dynamic configuration.
Improved performance and scalability: The native WAF integration allows for faster request processing, leading to better scalability. The same number of API Gateway replicas can handle more requests, improving resource utilization. This means you can run leaner infrastructure, serving the same or larger traffic with fewer resources, leading to lower operational costs.

Now, let’s see some proof of the claimed 23x performance improvement!

Performance Benchmark: WASM-Based vs. Native WAF

We conducted a performance benchmark to compare the efficiency of the native WAF integration in Traefik Hub against the WASM-based WAF plugin used in Traefik Proxy.

The benchmark results highlight the performance advantages of the native WAF in Traefik Hub:

WASM-Based WAF: In our test environment, the WASM-based WAF managed to handle 29.04 requests per second.
Native WAF: Under the same conditions, the native WAF processed a remarkable 683.34 requests per second.

This represents a 23.53x performance improvement when using the Coraza WAF natively in Traefik Hub!

Comparison between WAF integrations based on requests per second.

These metrics were obtained in a controlled environment using a local k3s cluster with just one gateway replica. While this setup provides a clear comparison of the two WAF approaches under identical conditions, it’s important to recognize that this is just one snapshot of performance as it is based on a simplified scenario and doesn't fully capture the capabilities of either the WAF or the API Gateway in a complex environment. The actual observed performance could vary depending on several factors:

Scaling: In production, the API Gateway and WAF would probably be deployed with multiple replicas to handle higher traffic volumes in production. The native WAF's performance benefits would likely become even more pronounced in such scaled environments.
Environment complexity: The nature of the applications, the complexity of the enforced security rules, and the overall architecture will all influence the WAF's performance.
Resource allocation: Differences in CPU, memory, and other resource allocations between environments can also impact the measured throughput.

While the native WAF integration offers superior performance, it’s also essential to consider the trade-offs involved in running a WAF, even a high-performance one. Adding a WAF to your traffic processing pipeline introduces some overhead due to the need to inspect and potentially filter every request and response.

Our tests showed that disabling the WAF altogether could result in at least 4 times higher request throughput in Traefik Hub. This is an important consideration for organizations that must balance security with performance. For some applications or routes with lower security requirements, disabling the WAF might be acceptable to achieve higher performance.

However, for critical applications or sensitive data, the performance overhead introduced by a WAF is typically a necessary trade-off to ensure robust security. The key is to use the flexibility offered by Traefik Hub to apply the WAF selectively, protecting the most vulnerable parts of your infrastructure while maximizing performance where possible.

Migration from WASM-Based WAF to Native WAF

If you’re currently using the WASM-based WAF in Traefik Proxy, migrating to the Native WAF in Traefik Hub is straightforward. Just like other transitions within the Traefik ecosystem, this migration is designed to be seamless, allowing you to enhance your security setup without disrupting your existing configurations or workflows.

Traefik Hub has been built with user experience in mind, ensuring that the migration is as painless as possible:

Remove static configuration: One key difference in the migration process is removing the static configuration required by the WASM-based WAF (the first block in the code example shown previously). The Native WAF in Traefik Hub does not require any static configuration, making the setup simpler and less prone to errors.
Retain existing middleware configuration: The rest of your WAF middleware configuration will remain the same, so you won’t need to make any changes to your existing setup!

The migration process is quick and efficient, allowing you to immediately take advantage of the highly improved performance after upgrading from Traefik Proxy to Traefik Hub.

For more information on the WAF configuration options, check out the official Traefik Hub documentation.

Wrapping It Up

If you’re looking for tools to enhance your web application security within the Traefik ecosystem, the availability of Coraza WAF as a WASM plugin represents a significant step forward. It’s free and open-source, and you can start experimenting with it today within Traefik Proxy with optional commercial support.

When you feel you want more, Traefik Hub is equipped to handle the most demanding security requirements, providing both protection and performance at scale with its native WAF integration. You can easily migrate from plugin-based to native WAF through a smooth transition with further simplified configuration, keeping all your security defenses in place while optimizing your resource utilization with an outstanding 23x performance improvement. Visit our WAF solution page to learn more.

Ready to see how Traefik Hub can elevate your security strategy? Request a demo today and experience the benefits of a modern, high-performance API Gateway and Management solution.

]]>

From Ingress to API Management: How Traefik OSS Grows with You

Immánuel Fodor — Fri, 23 Aug 2024 22:00:10 GMT

In the ever-evolving landscape of cloud-native technologies, managing and securing microservices and APIs effectively is crucial for businesses of all sizes. But as your API ecosystem grows and matures, so do your needs.

This blog will guide you through the seamless migration journey across Traefik Labs' comprehensive product portfolio, highlighting the effortless upgrade process that bridges it together. Starting from the foundational Traefik Proxy, you can now elevate your API operations to new heights with the Traefik Hub API Gateway and API Management tools.

And while we’re going to get deep in the weeds, I hope you’ll see the beauty in its simplicity—upgrading Traefik is essentially a replacement of the running binary, making it a non-disruptive process.

Start With Us and Grow With Us

Traefik Labs helps DevOps and Platform engineers throughout their cloud-native operations journeys, from a simple reverse proxy implementation to more advanced API lifecycle management capabilities.

Key API Gateway & API Management capabilities that can be added to Traefik OSS.

Traefik Hub API Gateway and API Management are built on the open-source Traefik Proxy. They include all the capabilities and characteristics of Proxy and extend it with specialized features targeting APIs. Each level-up takes just a few seconds and is an additive extension, nothing is taken away. Existing configurations and the UX remain the same. Everything you appreciate about Traefik Proxy today is carried through into each upgrade stage.

Since Proxy can run in many different environments, our API Gateway and Management can also publish and secure APIs from every Kubernetes flavor, big cloud providers, Hashicorp Nomad, Docker Swarm, and many more. And all these solutions are based on declarative configuration and GitOps. Leverage labels, middlewares, plugins, and CRDs declaratively, no matter which product you run.

Let’s explore the three journey steps in detail with short hands-on labs, starting with the origins and significance of Traefik Proxy.

Traefik Proxy: Born in The Trenches

The Traefik story began in 2015, addressing a real-world production challenge encountered by Emile Vauge, the founder of Traefik Labs. Deploying thousands of microservices demanded a solution that transcended the limitations of existing reverse proxies at that time. Traditional solutions lacked the automation, service discovery, and scalability required for their environment. This led to the birth of Traefik Proxy, a solution designed from the ground up for the cloud-native world.

Traefik Proxy's focus on ease of use and powerful features quickly propelled it to global recognition. With over 3.3 billion downloads and a place among the top 15 most downloaded images on Docker Hub, Traefik has become a cornerstone of cloud-native infrastructure. This widespread adoption is a testament to its core strengths:

Simplicity: Declarative configuration and GitOps integration streamline management and ensure infrastructure-as-code best practices.
Automation: Automatic service discovery and certificate management eliminates manual configuration burdens by automatically detecting and configuring routes to newly deployed services, reducing administrative overhead.
Scalability: Traefik scales effortlessly alongside your infrastructure, handling even the most demanding workloads.

Its versatility allows deployment on various cloud environments, including all Kubernetes distributions and other container orchestrators like Hashicorp Nomad or Docker Swarm, as well as bare metal and virtual machines, making it a comprehensive solution for all possible scenarios. Traefik serves as the gatekeeper of infrastructure, ensuring security, observability, and efficiency while scaling up to meet evolving needs.

Unsurprisingly, Traefik Proxy is a trusted solution for organizations of all sizes. Financial institutions, major retailers, and countless service providers leverage Traefik to manage mission-critical production workloads. It’s an ideal choice for:

Users prioritizing best-of-breed integrations: Traefik Proxy integrates seamlessly with a vast ecosystem of tools and technologies, allowing you to leverage your existing infrastructure.
Environments requiring advanced network routing: Traefik Proxy functions as a versatile application proxy, handling tasks like ingress control, reverse proxying, and load balancing. It also supports content caching and circuit breaking to enhance performance and resilience.
Organizations seeking a free and open-source solution: Traefik Proxy is a fully open-source project allowing complete transparency and customization. However, you can also get commercial support with 24/7 access to the Traefik Labs team.

Now that it’s clear what Traefik Proxy is and what it can do for you, let’s get our hands dirty and explore the journey of an API in the Traefik ecosystem.

Journey Lab 1: Deploy a Weather Service with Traefik Proxy

In this blog post, apart from covering the obvious upgrade steps from Proxy to API Gateway then to API Management, we’ll follow the life of a simple Weather API for illustration purposes.

The first step with the open-source Traefik Proxy is where everybody gets familiar with the Traefik ecosystem. Let’s deploy a Proxy to simulate the Kubernetes-based infrastructure of our imaginary weather forecasting service provider company.

helm repo add --force-update traefik https://traefik.github.io/charts
helm install traefik -n traefik --create-namespace --wait \
  --set ingressClass.enabled=false \
  --set ingressRoute.dashboard.enabled=true \
  --set ingressRoute.dashboard.matchRule='Host(`dashboard.docker.localhost`)' \
  --set ingressRoute.dashboard.entryPoints={web} \
  --set ports.web.nodePort=30000 \
  --set ports.websecure.nodePort=30001 \
   traefik/traefik

💡

Did you know that Traefik comes pre-packaged as the default Ingress Controller for Rancher's popular Kubernetes distribution? So, if you want to replay the commands in k3d, you first need to create a cluster with the default Traefik Ingress Controller turned off:

k3d cluster create traefik-hub --port 80:80@loadbalancer \
--port 443:443@loadbalancer --port 8000:8000@loadbalancer \
--k3s-arg "--disable=traefik@server:0"

Once it's installed, we can access the local Proxy dashboard: http://dashboard.docker.localhost/

Traefik Proxy Dashboard

Now, it’s time to deploy the backend of the Weather API using a simple JSON server written in Go and expose it with an IngressRoute object:

kubectl create namespace apps
kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/weather-app.yaml
kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-no-auth.yaml

The Traefik-native IngressRoute is chosen here for simplicity. Still, Traefik Proxy can also expose applications using the old-standard Kubernetes Ingress or Kubernetes’ new Gateway API HTTPRoute objects. In fact, we have first-class production-ready Kubernetes Gateway API support starting from Proxy v3.1!

The Weather API can be accessed now; let’s try it using curl:

$ curl http://walkthrough.docker.localhost/no-auth
{
  "public": [
    { "id": 1, "city": "GopherCity", "weather": "Moderate rain" },
    { "id": 2, "city": "City of Gophers", "weather": "Sunny" },
    { "id": 3, "city": "GopherRocks", "weather": "Cloudy" }
  ]
}

Let’s not keep the API unsecured; protect it quickly with Basic Authentication.

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-basic-auth.yaml
# To generate the password, we used `htpasswd -nb foo bar | openssl base64`
# The endpoint path has also moved to /basic-auth

Basic Authentication was widely used in the early days of the web. However, it also has a security risk: credentials can be visible to any observer when using HTTP. Additionally, it uses hard-coded credentials, potentially giving more authorization than required for a specific use case.

Nowadays, those issues are addressed when using JSON Web Tokens (JWT). A JWT can be cryptographically verified; it detaches authentication from user credentials and has an issue and expiration date.

Even in this simple use case, this is how we reach the need for a sophisticated API gateway that has enterprise-level authentication and authorization, among many other new capabilities, on top of Traefik Proxy.

Traefik Hub API Gateway: Level Up Your API Security

While the exact percentage may vary depending on the data source and methodology, recent reports consistently show that API traffic makes up most of the web traffic. Imperva found that 71% of internet traffic in 2023 was API calls. The API Security and Management Report by Cloudflare indicates that APIs accounted for over 57% of dynamic internet traffic processed by their platform in the past year. And according to Akamai, 83% of web traffic consisted of API calls.

The growth in API usage is driven by factors like digital transformation, cloud migration, and the rise of the microservice architecture. As APIs proliferate within organizations, the need for an API Gateway becomes increasingly critical, particularly as centralizing common functionalities to streamline operations and enhance security becomes a priority.

Traefik Hub API Gateway offers a natural evolution from Traefik Proxy, addressing this need by offering a range of enterprise-grade features specifically targeting the use case of exposing and securing APIs. Building upon the core strengths of Proxy, our API gateway equips you with a comprehensive suite of features designed to safeguard your APIs, including centralized authentication and authorization, distributed security features, and various vendor integrations.

Centralized security and control: Imagine managing authentication and authorization across your entire API portfolio from a single location. Traefik Hub API Gateway empowers you to do just that. It supports a wide range of industry-standard protocols, like OIDC and OAuth token introspection, allowing you to define and enforce access policies for all your APIs centrally. This eliminates the need for repetitive implementation and configuration on the microservice level and ensures consistent security practices across your deployments.
Distributed security features: Traefik Hub API Gateway takes security a step further by offering features like distributed rate limiting and distributed TLS certificates. Rate limiting ensures that no single API backend becomes overwhelmed by excessive traffic spikes. Rate-limiting quotas are distributed across your Gateway instances, providing a robust defense against denial-of-service attacks. Additionally, API Gateway integrates seamlessly with security tools like Let's Encrypt, allowing you to provision and manage TLS certificates for your APIs effortlessly, even in a distributed environment.
Enhanced vendor integrations: The modern API landscape often involves a complex ecosystem of tools and services. Traefik Hub API Gateway recognizes this by integrating with leading security vendors like HashiCorp Vault and Azure Key Vault. This enables you to centralize your secrets management and leverage existing security infrastructure within your organization.
Native Web Application Firewall (WAF): In our API Gateway, the recently announced Coraza WAF integration is implemented natively as part of the Gateway’s code base. The native integration provides more than 23 times (Yes, you read that right!) performance improvement over the plugin-based WAF available in Proxy.
Open-source with enterprise benefits: Traefik Hub API Gateway retains the open-source core of Traefik Proxy, allowing you to benefit from continuous innovation from the whole open community. However, API Gateway also unlocks additional enterprise-grade features on top of Proxy without requiring excessive migration.

Upgrading from Traefik Proxy to API Gateway is a smooth and straightforward process. The seamless upgrade path protects your investment in existing configurations and ensures minimal disruption to your ongoing operations. All your current configurations can stay in place, and the API Gateway continues to honor them without any change. While building upon the foundation you've established with Traefik Proxy, Traefik Hub API Gateway empowers you to manage access centrally, enforce security policies, and leverage industry-standard best practices.

Journey Lab 2: Upgrade Traefik Proxy to Traefik Hub API Gateway

Let’s continue the upgrade journey in this next lab, leveling up the Weather API game from Proxy to API Gateway capabilities. The upgrade is as easy as setting up a license key and executing a simple Helm chart upgrade. Fill out this form to get a trial or production license key.

Now, open a terminal and save the license as a Kubernetes secret:

kubectl create secret generic license \
  --namespace traefik \
  --from-literal=token=$TRAEFIK_HUB_TOKEN

Then, upgrade Traefik Proxy to Traefik Hub using the same Helm chart we used previously since Traefik Hub API Gateway is 100% compatible with Traefik Proxy v3 and up. All settings are retained; the only command you need to execute just replaces the running binary with a new one:

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set hub.token=license \
  --set image.registry=ghcr.io \
  --set image.repository=traefik/traefik-hub \
  --set image.tag=v3.2.0 \
   traefik/traefik

The upgrade finishes in seconds, and the dashboard is still reachable (http://dashboard.docker.localhost/), but it now shows the Traefik Hub API Gateway logo in the top left corner.

Traefik Hub API Gateway Dashboard

You can also confirm that Basic Auth is still here if you execute the same curl commands against the Weather API. All other configurations are also kept and served.

Let's secure the weather API with something more professional, such as an API Key, a new middleware to which the API Gateway gives access (among many other things).

First, we generate the hash of our API key: “Let's use API Key with Traefik Hub”. Using the key hash in the configuration ensures that the key itself is not leaked, even if it is accidentally pushed to a public Git repository.

htpasswd -nbs "" "Let's use API Key with Traefik Hub" | cut -c 2-

We can now put it into the API Key middleware and apply it:

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/weather-app-apikey.yaml
# The API path has also been changed to reflect the use case better, now it’s /api-key

Test it with the following commands:

# This call is not authorized => 401
curl -I http://walkthrough.docker.localhost/api-key
# Let's set API Key
export API_KEY=$(echo -n "Let's use API Key with Traefik Hub" | base64)
# This call with the token is allowed => 200
curl -I -H "Authorization: Bearer $API_KEY" \
  http://walkthrough.docker.localhost/api-key

The API is now secured using an API key. The API gateway also allows you to handle users with an identity provider using OIDC or OAuth, apply distributed rate-limiting, and implement other enterprise-grade configurations.

But what if we want to cover internal and external use cases and protect the Weather API on the HTTP verb level to apply further governance? Or test a new API version with part of the production traffic? And what about publishing all the managed APIs to a developer portal for easy discovery and reduced time to consumption?

You’ll need Traefik Hub with API Management features enabled to do this.

Traefik Hub API Management: Granular Control and Scalability

As your API ecosystem scales beyond security needs, managing APIs effectively becomes crucial. Building upon the robust features of Traefik Hub API Gateway, Traefik Hub API Management offers a powerful solution for comprehensive API lifecycle management, enabling you to govern your APIs, streamline development workflows, and cater to diverse API stakeholder needs.

API governance for complete control: Traefik Hub API Management provides a centralized platform for discovering, registering, and managing your entire API portfolio. Features like versioning and Kubernetes-native API labels and selectors allow you to organize your APIs logically and track their evolution over time. Additionally, API Management integrates with linters, which can automatically detect and flag potential errors within your API configurations before they impact production.
Advanced API security: API Management takes API security to the next level by offering granular access control mechanisms. You can define fine-grained permissions for individual users or groups, ensuring only authorized users can access specific API resources and functionalities. Furthermore, API Management integrates with industry-standard authentication protocols, allowing you to leverage existing identity and access management (IAM) solutions within your organization.
API monitoring and observability: Deep insights into API health and performance are crucial for maintaining a reliable and scalable API landscape. Traefik Hub API Management integrates seamlessly with OpenTelemetry, providing comprehensive metrics and tracing capabilities. This allows you to gain real-time visibility into API traffic patterns, identify potential bottlenecks, proactively address any performance issues, and troubleshoot effectively.
API Developer Portal: Create one or more dedicated API Developer Portals easily! The portal is a central hub for developers to discover, understand, and interact with your APIs. The portal can house API documentation, interactive consoles for testing purposes, and clear instructions on integrating with your APIs. You can also fully customize it to your liking. By providing a user-friendly developer experience, API Management fosters adoption and streamlines the development process for your APIs' internal and external consumers.

As with the previous steps, upgrading from Traefik Proxy or API Gateway to API Management is a similarly simple process. It involves a license upgrade and a Helm chart update, minimizing disruption to ongoing operations. API Management inherits all the functionalities of both Proxy and Gateway, ensuring a smooth transition and continued access to their features. API Management also treats configuration through declarative CRDs as a first-class citizen, but also offers a user-friendly UI for managing API configurations and their current state.

Journey Lab 3: Upgrade Traefik Hub API Gateway to API Management

In our last lab, we assume that the license of the weather provider company has been updated with enabling API Management capabilities. After that, we enable the API Management feature on the Traefik Hub deployment using the same Helm chart:

helm upgrade traefik -n traefik --wait \
  --reuse-values \
  --set hub.apimanagement.enabled=true \
  traefik/traefik

The upgrade was as easy as executing this one command. The local dashboard is still reachable on http://dashboard.docker.localhost/ and we can also confirm that the API is still secured using an API Key.

Now, let's promote the Weather API Ingress to be a managed API with Traefik Hub API Management:

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/api.yaml

What did we just do here?

The applied manifest has created a managed API CRD describing the properties of the Weather API, like the location of its OpenAPI Specification.
It also created an API Access CRD describing who can access the Weather API.
It removed the API Key authentication middleware, as we'll use Traefik Hub's built-in identity provider for user and credential management. The API is still secured, as we'll see it shortly.
It added a simple one-line annotation on the existing IngressRoute CRD with a link to the API CRD created in the 1st point, effectively promoting the ingress to be now a Traefik Hub-managed API.

hub.traefik.io/api: walkthrough-weather-api # <= Link to the API using its name

We can confirm this API is still not publicly exposed without authentication as it returns the expected 401 Unauthorized HTTP code. To be able to consume the API, let’s create a user for this API in the Traefik Hub API Management online dashboard.

The user created previously will connect to an API Portal to generate an API key, so let's deploy our API Portal!

kubectl apply -f https://raw.githubusercontent.com/traefik/hub/master/src/manifests/walkthrough/api-portal.yaml

After a couple of seconds, the API Portal should be reachable on http://api.walkthrough.docker.localhost. Log in with the admin user and create an API Key. The Weather API is now consumable with the new token, and it’s also visible on the portal:

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  http://api.walkthrough.docker.localhost/weather

Traefik Hub API Portal

The Weather API has documentation built from the OpenAPI specification, and the API can also be interactively tested right on the Portal with the try-out functionality.

And that's it! You’ve now seen the very short and quick upgrade steps of each tool and also got a glimpse of using them.

Wrapping It Up

The seamless migration journey across Traefik Labs' products ensures that everything built remains intact while adding new capabilities as needed. Investing time and resources into Traefik Proxy, Traefik Hub API Gateway, or API Management is a smart choice for businesses looking to future-proof their infrastructure. There is no rip-and-replace—just an in-place upgrade!

Consider Traefik’s cloud-native, fully declarative, and GitOps-driven approach to ensure efficient and effective API operations in your organization. The functionally additive upgrade path, robust feature set, and ease of management make it the best choice for handling the complexities of modern cloud-native environments. Scale your operations confidently with Traefik as your business grows and evolves.

]]>

How the Latest API Gateway Innovations Are Reshaping Operational Efficiency and Error Metrics

Immánuel Fodor — Thu, 15 Aug 2024 23:04:24 GMT

Downtime is not just an inconvenience—it can be a critical threat to business operations. The cost of downtime extends far beyond immediate financial losses; it can damage your company's reputation, erode customer trust, and, in some industries, such as healthcare and chemical processing, can lead to dangerous situations.

Modern, declarative API gateways are pivotal in managing the complexities of microservices architectures. They provide GitOps capabilities that reduce human error, allow for immediate rollback to a stable state, and make things repeatable and scalable even for a small staff. They can also provide advanced observability and monitoring capabilities, automating key tasks and offering real-time insights that let DevOps teams quickly identify and resolve issues.

And as you already know, issues are going to happen. The question is how much they’ll cost you.

The Cost of Downtime

Downtime can affect everything from financial performance to customer trust, so understanding and minimizing downtime is crucial for maintaining operational efficiency and ensuring continuous service availability. To get a tangible feel for downtime costs, we can look at the Mean Time to Recovery (MTTR) and the associated cost per minute of a system that’s not operating properly. In some situations, even brief downtimes are unacceptable, showing the business benefits of implementing robust observability and monitoring tools combined with GitOps to mitigate these risks.

Mean Time to Recovery (MTTR)

MTTR is a critical metric in DevOps, representing the average time it takes to restore a system to operational status after a failure. A lower MTTR indicates a more resilient and efficient system, which is essential for minimizing downtime. Quick failure recovery can make a significant difference in environments where uptime is paramount, such as healthcare or financial services. Rapid issue resolution reduces the impact on users and helps maintain trust in the system's reliability.

Cost per Minute of Downtime

The financial impact of downtime can be staggering. According to a 2016 study, the average cost of IT downtime was as much as $9,000 per minute, which translates to $540,000 per hour—and that was almost 10 years ago. (You can get a visceral sense of how quickly this adds up here.) Even for smaller businesses, where the cost per minute can be in the hundreds of dollars rather than thousands, it’s still significant.

The costs can be even higher in industries with critical dependencies on IT systems, such as healthcare, finance, and telecommunications. For example, when a bug turned the automatic update of CrowdStrike’s Falcon antivirus software into a catastrophic event, many hospitals found themselves unable to access diagnostic systems or even patient registration. All in all, it cost healthcare providers an estimated $1.94 billion, with an average of 64.6 million for each provider.

Even more important, however, is that when a hospital’s systems go down, critical care might be delayed, putting lives at risk. Therefore, minimizing downtime may not be just a technical or financial challenge, and instead may be far more high-stakes.

While a modern API gateway couldn't have helped companies avoid the CrowdStrike outage, they can be pinnacle in preventing other outages.

The API Gateway Innovations Reshaping Incident Prevention and Mitigation

The combination of GitOps and comprehensive observability is revolutionizing the way we manage and monitor API gateways.

GitOps leverages version control and automation to bolster the reliability, security, and scalability of these critical components, ensuring seamless, error-free operations. By defining infrastructure and API configurations declaratively and managing them in version-controlled repositories, GitOps enables rapid, consistent deployments while enhancing audit trails and compliance tracking.

Meanwhile, comprehensive observability extends monitoring beyond traditional methods, providing real-time insights through logs, metrics, and traces. This holistic approach enables teams to proactively identify and address potential issues, minimizing downtime and maintaining high system performance. Together, GitOps and observability form a powerful duo, empowering organizations to optimize their API gateway operations in today's demanding digital landscape.

The GitOps Revolution

GitOps is crucial for modern API gateways because it leverages the principles of version control and automation to enhance reliability, security, and scalability. Because API gateways act as the gatekeepers for all incoming and outgoing traffic in microservices architectures, the need for robust, error-free, and agile operations is paramount. GitOps addresses these needs by providing a framework where infrastructure and API configurations are defined declaratively and managed in version-controlled repositories.

This approach not only facilitates rapid, consistent deployments but also significantly improves audit trails and compliance tracking. By automating deployment processes, GitOps reduces human error and increases operational efficiency, making it an essential strategy for organizations looking to optimize team performance in a cloud-native environment.

But as we all know, errors will happen. With a GitOps strategy, code can be instantly rolled back to a stable state, creating the time necessary to diagnose and remedy issues without impacting stakeholders. For this reason, it’s important to choose an API gateway that supports GitOps end-to-end.

The Advent of OpenTelemetry and Comprehensive Observability

When it comes to mitigating the effects of downtime, in addition to GitOps, your best friend is “comprehensive observability,” or the ability to fully monitor and understand the state and behavior of a system across its entire stack by collecting, analyzing, and visualizing data from its various components. This concept extends beyond traditional monitoring to include three key types of telemetry data:

Logs: timestamped records of events
Metrics: quantitative data that measure the performance and health of the system
Traces: which track the journey of requests as they flow through the various components of the system

By implementing comprehensive observability solutions, your organization can gain real-time insights into your systems' health and performance. This proactive approach enables teams to identify and address potential issues before they escalate into significant problems, minimizing downtime and associated costs and, using GitOps, to roll back if necessary.

Observability tools that can monitor your API gateway offer detailed metrics, logs, and tracing capabilities that help DevOps teams maintain high system performance. These tools allow for monitoring key performance indicators (KPIs) such as request latency, error rates, and system throughput. By closely watching these metrics, teams can quickly detect anomalies and take corrective actions, ensuring systems remain operational and performant.

One thing that makes this kind of comprehensive observability possible is the advent of OpenTelemetry, an open-source standard for observability. Any tool that reads and/or writes it can share information with any other tool that understands it. By choosing an API Gateway that supports OpenTelemetry, you’re providing yourself with much greater possibilities.

As you might imagine, the observability strategies you have available to you are going to depend on the richness of logs, metrics, and traces your API gateway supports.

How to Reduce the Impact of Downtime

While businesses can implement several measures to minimize downtime, advanced observability and monitoring strategies are essential for proactively managing system health. This includes a combination of general best practices, GitOps, and specific observability-focused strategies to help you reduce downtime. DevOps teams can ensure higher system availability and performance by leveraging tools, such as their API gateway, for real-time monitoring, detailed metrics, and distributed tracing.

Generic Measures for Avoiding Downtime

While specific strategies vary, there are several generic measures are fundamental to the cause:

Regular System Updates and Maintenance: Regular maintenance schedules help ensure that all components function optimally and that known vulnerabilities are addressed promptly.
Robust Disaster Recovery Plans: This plan should include regular backups, clear recovery procedures, and periodic testing to ensure that systems can be quickly restored in case of a failure.
Redundant Systems and Failovers: Implementing redundancy at various levels—such as servers, storage, and network connections—ensures that a failure in one component does not bring down the entire system. Failover mechanisms should be in place to automatically switch to backup systems if a primary system fails.

Now let’s explore the more innovative measures.

GitOps-Focused Strategies

GitOps, by integrating core practices of DevOps with Git-based workflows, offers several strategic approaches to enhancing reliability, preventing downtime, and accelerating recovery in system operations. They include:

Automated Rollbacks and Progressive Delivery: It takes a bit of extra setup, but by using automated rollbacks, teams can rapidly address issues without manual intervention, drastically reducing potential downtime. Furthermore, progressive delivery techniques such as canary releases or blue-green deployments can be managed through GitOps workflows, allowing for a small subset of users to be exposed to new changes, limiting the impact of potential failures and ensuring that any critical issues can be identified and rectified before they affect the entire system.
Declarative Configuration Management: With GitOps, all system configurations are defined declaratively and stored in version control systems. This approach ensures that the configurations are reproducible and consistent across all environments, which eliminates common causes of downtime such as configuration drift or manual errors in setup. If a new configuration update causes a problem, teams can quickly revert to a previous configuration, ensuring continuous system availability.
Continuous Validation and Testing: Continuous integration tools integrated within GitOps workflows can automatically trigger tests and validation checks every time a change is made to the repository. This means that code and configurations are continuously tested, and only changes that pass predefined checks are deployed to production environments. This reduces the risk of introducing errors that can lead to downtime, ensuring that only stable, thoroughly tested updates are released.
Collaboration and Quick Response: GitOps fosters a culture of collaboration by ensuring that all changes are visible and traceable through the Git platform. Transparency helps teams to quickly identify and respond to issues, minimizing response times and reducing downtime. Additionally, having a centralized platform for all operations simplifies communication among team members, which is critical during incident response.

Observability-Focused Strategies

Observability is key to minimizing downtime. It involves monitoring systems comprehensively to proactively detect and address issues:

Real-time Monitoring: Using an API gateway with a monitoring dashboard, such as Grafana, for continuous health checks and performance monitoring ensures that any deviations from normal operation are immediately detected.
Detailed Metrics and Logging: For example, monitoring CPU usage, memory consumption, and network latency helps identify performance bottlenecks. Logs offer a detailed record of system activities, which is useful for diagnosing and troubleshooting issues.
Distributed Tracing: Some API gateways (e.g., Traefik’s) offer distributed tracing capabilities for tracking request flowing across various services. Tracing provides a holistic view of how different components interact, making pinpointing and addressing issues easier.
Proactive Issue Resolution: For example, if an observability tool indicates an increase in error rates, the team can investigate and address the underlying cause before it affects users.

Capabilities of an Effective Observability Solution

An effective observability solution should provide the following capabilities:

API Gateway Status Monitoring: Most API gateways provide status monitoring and alerts, enabling teams to detect and address issues with the gateway itself.
Tracing and OpenTelemetry: Tracing helps in understanding the flow of requests through the system. For example, Traefik supports OpenTelemetry, an open-source format for observability, enabling integration with any monitoring tool that supports it. This standardization allows for real-time troubleshooting and comprehensive system analysis.
Distributed Tracing: Distributed tracing is essential for understanding interactions across different services. Traefik, for example, helps tracing tools make correlations between various sources, such as the gateway, database, and microservices, providing a detailed view of the request lifecycle, and helping teams understand performance issues and optimize the system. The end result is greater transparency rather than the proverbial “black box.”

It’s important to realize that the level of observability in your API Gateway is more than just a formality. Not all gateways are created equal, and your ability to avoid and quickly bounce back from downtime will be directly related to your observability capabilities.

Conclusion

In today’s fast-paced and highly competitive digital landscape, minimizing downtime and enhancing system performance are critical to business success. Through the use of GitOps and integration with monitoring and observability capabilities, modern API Gateways provide robust solutions to these challenges. By reducing MTTR and its associated costs and leveraging real-time insights, organizations can swiftly address issues, ensuring continuous service availability and optimal performance.

Additional Resources

Traefik’s Blog on Observability and Tracing: Visit Traefik’s blog to learn more about how its observability features can enhance your API gateway management.
DORA Reports and Metrics: Understand the key metrics for evaluating DevOps performance and how Traefik can help improve them. Access the 2023 DORA Report.
Google Cloud’s ROI Whitepaper: Discover the financial benefits of investing in robust observability and monitoring tools in the Google Cloud ROI Whitepaper.

]]>

Strategic API Gateway Migration: A Comprehensive Blueprint

Immánuel Fodor — Fri, 02 Aug 2024 21:19:31 GMT

You’ve had it.

As the head of your company’s DevOps process, you’ve been frustrated with the web of APIs your company has created for some time. There’s no centralized governance; there’s no service discovery so developers are constantly searching for the right APIs to use; load balancing is a nightmare … but last night’s security breach tears it. Those credentials should have been deactivated months ago.

It’s time for a real, honest-to-goodness API gateway.

Or maybe you already have an API gateway, but it’s inconsistent across different environments and it hurts team efficiency by requiring constant situational manual interventions. Either way, you’re going to need to move your workloads to a new system that solves these problems.

Of course, that makes you cringe, too. The migration will be a nightmare.

But it doesn’t have to be.

By doing a strategic migration of your systems and moving a few APIs at a time, you can minimize risk and increase your agility during the process. Yes, it will take a bit longer, and there will be more moving parts, but a solid GitOps strategy will smooth the transition, and it will definitely be worth avoiding the stress that’s churning in your stomach right now as you have another cup of coffee to make up for not sleeping last night

Let’s look at specifically what it takes to perform a strategic migration to an API gateway model.

Phase 1: Planning and Preparation

Just as with everything else in software, the foundation of a successful API gateway migration lies in meticulous planning and preparation. This initial phase sets the stage for a smooth transition by addressing all critical aspects, from gathering detailed requirements to engaging stakeholders and evaluating potential solutions.

By taking the time to thoroughly prepare, you can identify and mitigate risks, align the migration with business objectives so the execs are on your side, and make sure that all the resources and tools you’ll need are in place. Effective planning not only reduces the likelihood of unforeseen issues but also enhances the overall efficiency and success of the migration process.

Let's look at the key steps involved in this phase.

Step 1: Requirement Gathering and Analysis

The first step in a successful API gateway migration is thorough requirement gathering and analysis. Identify and document both business and technical requirements, considering current pain points and future scalability needs. Conduct a detailed analysis of the existing API infrastructure, evaluating functionalities, dependencies, and potential integration challenges. By establishing a comprehensive understanding of requirements, you lay a solid foundation for a strategic and well-executed migration process.

Step 2: Stakeholder Engagement

In this phase you’ll want to closely collaborate with stakeholders to make sure you understand their needs and expectations so the new API gateway aligns with organizational goals.This kind of effective stakeholder engagement is crucial for a successful API gateway migration.

Start by identifying key stakeholders across departments, including development, operations, security, and business units. Engage with these stakeholders early to gather their input and address their concerns. You want to clearly communicate the benefits of the migration, such as improved security, scalability, and centralized management, but don’t forget to highlight potential risks and the mitigation strategies in place.

Regular updates and transparent communication throughout the process will help promote trust and collaboration, ensuring all parties are aligned and supportive of your migration efforts. What’s more, engaged stakeholders are more likely to contribute valuable insights and resources, helping move toward the migration's success.

Step 3: Create a Migration Plan

Now that you’ve spoken to stakeholders it’s time to create a detailed migration plan. Make sure that you:

Outline each phase of the migration, specifying tasks, timelines, and responsible parties. Include key milestones, risk assessments, and contingency plans to address potential challenges.
Define clear objectives for each phase, from initial setup to full deployment, making sure that you align with business and technical goals.
Incorporate best practices and lessons learned from previous projects to optimize the migration strategy.

A well-structured migration plan provides a roadmap for the project, enabling smooth execution, effective resource management, and timely delivery of the new API gateway solution.

Step 4: Assessment and Selection

Now that you know what you need, you can choose a new API gateway. Don’t skimp on the selection process, because you’ll likely live with your choices for a long time. The selection process consists of several steps:

Start by evaluating various API gateway options against your documented requirements, focusing on scalability, security, ease of integration, cost, and support.
Make sure the gateway you choose supports GitOps so you can easily make and rollback changes to your configuration.
Consider the community and vendor support available for each option.
Conduct proof-of-concept tests to validate the gateway’s performance and compatibility with your existing systems.
Involve stakeholders in the evaluation process to gather diverse perspectives and ensure all needs are addressed.

Selecting the right API gateway lays the groundwork for a successful migration and long-term operational efficiency (not to mention fewer sleepless nights) so take all the time you need (within reason).

You can learn how to evaluate modern API gateways in our Buyer’s Guide for Modern API Gateways.

Step 5: Define Success Criteria

Now that you know what you’re trying to do, you need to decide how you’ll know when you’ve done it. Defining clear success criteria is essential to measure the effectiveness of the API gateway migration. Good success criteria will:

Establish specific, measurable goals beyond just completing the migration from point A to point B.
Include key performance indicators (KPIs) such as improved latency, fewer security vulnerabilities, and increased scalability.
Set benchmarks for user satisfaction, system reliability, and overall performance improvements.
Ensure these criteria align with both business and technical objectives, providing a comprehensive framework for evaluating the migration’s success.

Clearly articulate what success looks like, so you can focus your efforts, track your progress, and demonstrate the value of the migration to stakeholders.

Step 6: Environment Setup

Now you’ll need the environment in which to perform the migration. Obviously you’ll need a testing environment, because you wouldn’t run this directly in production, right? Right???

You’ll want a testing environment that closely mirrors the production setup to identify and address potential issues early. Make sure all the necessary tools and resources, such as monitoring systems, logging mechanisms, and CI/CD pipelines, are in place and properly configured.

Basically, make sure you can do thorough testing and validation of the new API gateway before it goes live. You need to be able to mitigate risks, improve system stability, and ensure a smoother transition to the new gateway.

Phase 2: Initial Rollout

The initial rollout phase is where the groundwork you laid during planning and preparation begins to take shape. This phase focuses on carefully selecting a pilot group of APIs for migration, configuring and deploying the new API gateway, and conducting thorough testing to ensure everything operates smoothly. It also emphasizes the importance of monitoring and gathering feedback to refine the process before moving on to broader implementation. By starting with a controlled and manageable subset of APIs, you can identify and address any issues early on.

Step 1: Pilot Group Selection

Selecting the right pilot group is critical for a successful initial rollout of the API gateway migration. You want to identify a small, manageable set of APIs and functionalities to migrate first, prioritizing non-critical or low-risk APIs to minimize potential impact of any issues.

Choose APIs that represent a variety of use cases to comprehensively test the new gateway's capabilities, and involve a cross-functional team of developers, testers, and users to ensure diverse feedback. Having the proper team helps you validate the migration process, identify and resolve issues early, and build confidence for subsequent phases of the migration.

Step 2: Configuration and Deployment

The configuration and deployment phase is where everything comes to fruition. You’ll want to:

Begin by configuring the new API gateway according to the documented requirements, ensuring all settings align with your security, performance, and integration needs.
Deploy the selected APIs to the new gateway in the testing environment, meticulously following your migration plan.
Utilize automated deployment tools to streamline the process and reduce the risk of human error.

If this step fails, your migration fails, so be extra diligent.

Step 3: Testing

Now you need to thoroughly test to ensure the new API gateway functions correctly and meets all specified requirements. To make sure that everything is working properly:

Perform a comprehensive suite of tests, including functional, performance, security, and integration testing.
Use automated testing tools to enhance efficiency and coverage, ensuring all aspects of the gateway are evaluated.
Verify that the new gateway handles API requests correctly, maintains performance standards, and adheres to security protocols.

Testing should simulate real-world scenarios to uncover any potential issues so you can identify and resolve problems early, before they impact your users.

Step 4: Monitoring and Feedback

Once you’ve done the initial deployment of the API gateway, ongoing monitoring and feedback collection are crucial. Make sure that you:

Continuously track the performance and usage of the migrated APIs, comparing them against predefined success criteria.
Use monitoring tools to detect anomalies, measure response times, and assess overall system health.
Collect feedback from users and developers to identify any issues or areas for improvement.
Document lessons learned during this phase to refine the migration process.

At this stage, negative comments are actually a good thing, because they will help you make the system better. By actively monitoring and gathering feedback, you can promptly address any problems and optimize not only the gateway's performance, but also the overall process internally.

Phase 3: Gradual Expansion

Now that you’ve done the initial rollout and you know that everything works, it’s time to start giving users access to the new gateway.

This phase involves iteratively migrating additional APIs to the new gateway in small, manageable batches, doing regular testing, monitoring for issues, and keeping stakeholders in the loop. To ensure continuity and reliability, in this phase you’ll operate the old and new gateways in parallel.

Step 1: Parallel Operation

In this phase you’re making the new API gateway available, but you’ll still need the old one, both as a fallback for the migrated APIs and because it’s got the non-migrated API’s on it. In other words, operating the old and new API gateways in parallel ensures a safety net during the migration process.

This dual operation enables you to seamlessly switch between gateways if issues arise, minimizing downtime and service disruptions. Implement version control and routing strategies to efficiently manage API traffic between the two gateways and ensure smooth transitions and consistent performance.

This parallel setup also provides an opportunity to compare the performance and functionality of both gateways in real-time. By maintaining parallel operation, you can safeguard against potential problems, ensure reliability, and build confidence before fully decommissioning the old gateway.

Step 2: Iterative Migration

Iterative migration involves gradually moving additional APIs to the new gateway in manageable batches. When you started, you started with the least critical so that any issues that came up caused the least amount of disruption. Now it’s time to reverse that. Prioritize the business's most important APIs (e.g., the biggest volume, or the most complex) to move the most workloads to the new API gateway.

Another approach is to start with the APIs that hurt the most, then the rest is easy. If you leave the most painful APIs for last, there is a very good chance that they will not be migrated as the team's focus shifts over time.

After each batch, perform the same thorough testing, monitoring, and feedback collection as in the initial rollout phase. This incremental approach allows for continuous assessment and adjustment, reducing risks and ensuring stability at each step.

Step 3: Stakeholder Communication

Effective stakeholder communication is essential throughout the migration process. To keep in touch with stakeholders:

Regularly update users, developers, and business stakeholders on the progress of the migration, highlighting milestones and addressing any concerns.
Share insights from monitoring and feedback phases to demonstrate transparency and build trust.
Evaluate the migration's impact against success criteria and communicate these results to stakeholders, showing tangible benefits and improvements.

By maintaining open lines of communication and involving stakeholders at every stage, you can ensure alignment with organizational goals and get continued support for the migration initiative, which will be important at budget time.

Step 4: Documentation and Training

Comprehensive documentation and training help you make sure your transition to the new API gateway is smooth. In this step, do the following:

Update all API documentation to reflect the changes and new capabilities introduced by the new gateway.
Make sure that this documentation is clear, detailed, and easily accessible to developers and users.
Provide training sessions to familiarize the development team and end-users with the new system.
Offer ongoing support and resources, such as tutorials and FAQs, to assist with the transition.

By investing in thorough documentation and effective training, you can facilitate user adoption, reduce onboarding time, and enhance the overall success of the migration.

Phase 4: Full Migration and Decommissioning

You’re almost there! The final phase of the API gateway migration involves the complete transition of all remaining APIs and the careful decommissioning of the old gateway. This phase focuses on ensuring that all functionalities and dependencies are successfully replicated or enhanced in the new gateway.

For example, post-migration monitoring is crucial to verify stability and performance, while continuous improvement efforts ensure the gateway remains optimized and aligned with business needs. Additionally, the decommissioning process involves securely phasing out the old infrastructure, ensuring no residual traffic or dependencies remain. By executing a thorough and methodical migration and decommissioning, you can achieve a successful transition and position your organization for ongoing API gateway success.

Let's delve into the specific steps involved in this final migration and decommissioning phase.

Step 1: Final Migration

The final migration phase involves moving the remaining APIs to the new gateway, ensuring that all functionalities of the old gateway are replicated or enhanced. You’ll want to:

Conduct a thorough review to confirm that all dependencies are addressed and that no critical elements are overlooked.
Execute the migration carefully, following established procedures to minimize disruptions.
Once the final APIs are successfully migrated, perform comprehensive testing to verify that the new gateway operates smoothly and meets all performance, security, and functionality criteria.

Be meticulous in this step; you’re almost finished, it would be a shame if everything fell apart now, just when you were starting to believe you were going to be rid of all this pain.

Step 2: Decommissioning

Decommissioning the old API gateway is the final step in the migration process, coming after all workloads are being executed on the new gateway. To get there:

Verify that all dependencies are removed and that no residual traffic is being routed through the old gateway.
Conduct a thorough audit to confirm that all functionalities have been successfully transferred and are operating as expected on the new gateway.
Once you’re comfortable that everything has been moved to the new system and is running properly, you can proceed with securely shutting down and dismantling the old gateway infrastructure.

Proper decommissioning ensures a clean transition, reduces maintenance overhead, and eliminates potential security vulnerabilities associated with the outdated system. So, tempting as it may be to leave the old system in place “just in case,” don’t.

Step 3: Post-Migration Monitoring

Now that you’ve shut down the old system you’ll be tempted to consider the project finished, but it’s not. You need to continuously monitor the new gateway to track its operation, identify any issues, and assess its performance against predefined success criteria, using monitoring tools to gather real-time data on API usage, latency, and error rates.

Should any post-migration issues pop up (as they almost definitely will) make sure to address them immediately to maintain service reliability and user satisfaction. Regularly review and analyze the monitoring data to identify trends and areas for further optimization.

Effective post-migration monitoring helps maintain the integrity of the new gateway and supports ongoing improvements.

Step 4: Continuous Improvement

Speaking of ongoing improvements, continuous improvement is key to maximizing the benefits of your new API gateway. After the migration, regularly evaluate the gateway's performance, security, and scalability to identify areas for enhancement and implement iterative updates and optimizations based on user feedback and monitoring data.

Stay informed about emerging technologies and best practices to keep your API gateway up-to-date. The idea is to create a culture of continuous learning and adaptation within your team to proactively address challenges and leverage new opportunities. By committing to continuous improvement, you ensure that your API gateway remains robust, efficient, and aligned with evolving business needs.

Additional Considerations

Now, while all of that may cover the actual migration, there are a few more things you need to consider, both positive and negative.

Security

Ensuring robust security measures is essential when migrating to a new API gateway, so make sure that you:

Implement strong authentication and authorization protocols to protect your APIs from unauthorized access.
Use encryption to safeguard data in transit and at rest.
Regularly conduct security audits and vulnerability assessments to identify and address potential threats.
Integrate security best practices into your development and deployment processes, ensuring that security is a continuous focus.
Stay updated on the latest security trends and threats to proactively defend against emerging risks.

Remember, security shouldn’t be a “bolt-on.”

Scalability and Performance

Designing for scalability and performance is crucial for the long-term success of your new API gateway, and is probably one of the reasons you decided to upgrade in the first place. To do that:

Ensure the gateway can handle increased traffic and grow alongside your business needs by implementing scalable architecture and infrastructure.
Optimize performance by fine-tuning configurations, employing efficient load balancing, and utilizing caching strategies to reduce latency.
Regularly conduct performance testing to identify bottlenecks and areas for improvement.
Monitor the gateway's performance metrics to ensure it meets or exceeds current capabilities and adapts to varying loads.

By focusing on scalability and performance, you can deliver a reliable and responsive API experience to your users.

Easy Rollbacks

When you’re selecting a new API gateway, you want to make sure to implement easy rollback mechanisms to ensure that configuration changes and deployments can be quickly reversed in case of issues.

The best way to do this is to use a system that is based on version control, tracking changes, and maintaining previous versions of configurations and code. This enables you to automate rollback procedures within your CI/CD pipeline to enable rapid and reliable reversion to stable states.

This architecture is called GitOps, because you are essentially controlling your system by making changes to a git repository and merging them. The changes are then propagated to the API gateway platform’s configuration. (Remember that not all API Gateways support this capability. Traefik API Gateway is one that does.)

Regularly test rollback processes in your testing environment to ensure they function as expected. By planning for easy rollbacks, you can quickly address unforeseen problems, maintain service continuity, and reduce the impact of potential disruptions during the migration process.

Backup and Recovery

As with anything else in software, implementing a robust backup and recovery plan is critical for safeguarding your data during the API gateway migration. Make sure to:

Regularly back up all configurations, data, and critical system components to secure storage locations.
Ensure that backups are comprehensive and include all necessary elements to restore operations in case of data loss or corruption.
Test recovery procedures frequently to verify that backups can be restored quickly and effectively.
Implement automated backup solutions to maintain up-to-date copies of your data.

All of this is in order to ensure business continuity, protect against data loss, and quickly recover from any unexpected issues during the migration.

Team Collaboration

Strong team collaboration is vital for a successful API gateway migration, and fortunately, the way GitOps works fosters that spirit of collaboration, as everyone can work together without stepping on each other. You can further this goal by making sure to:

Promote open communication and collaboration among all team members, including developers, operations, security, and business stakeholders.
Use collaborative tools and platforms to share information, track progress, and coordinate tasks effectively.
Conduct regular meetings and update sessions to keep everyone aligned and informed about the migration’s status and any emerging issues.
Encourage a culture of teamwork and collective problem-solving to leverage diverse expertise and perspectives.

By enhancing team collaboration, you can ensure a smoother migration process.

Compliance and Governance

Ensuring compliance and robust governance is essential when choosing and migrating to a new API gateway. Some things to consider:

Align the new gateway with relevant regulations and industry standards, such as FIPS-140-3, GDPR, HIPAA, or PCI-DSS, to maintain compliance and avoid legal issues.
Implement governance policies to manage the API lifecycle, including versioning, deprecation, and documentation standards.
Regularly review and audit the gateway to ensure ongoing adherence to compliance requirements and internal policies.
Establish clear roles and responsibilities for governance to maintain accountability and transparency.

By focusing on compliance and governance, you can protect your organization from regulatory risks and ensure the API gateway operates within established guidelines.

Conclusion

Migrating to a new API gateway is complex but highly rewarding. It requires strategic planning, meticulous execution, and continuous improvement. However, by following a phased approach, you can minimize risks, ensure a smooth transition, and achieve significant improvements in API management, security, and performance.

Each phase, from initial planning and preparation through to full migration and decommissioning, plays a crucial role in the overall success of the project. Additionally, addressing key considerations such as security, scalability, compliance, and team collaboration further strengthens the migration process.

As an IT Director, leading this strategic migration not only enhances your organization’s technological capabilities but also positions you for long-term success in managing and optimizing your operations.

Embrace the journey, leverage the insights gained along the way, and transform your API infrastructure for the future.

... And finally get some sleep.

]]>

Getting started with Kubernetes Gateway API and Traefik

Nicolas Mengin — Tue, 16 Jul 2024 17:43:58 GMT

We're continuing our in-depth series on Traefik 3. If you missed it, be sure to read the previous articles on migrating from Traefik v2, WASM support with Coraza WAF, Open Telemetry, and SPIFFE, Tailscale, and HTTP/3. This article dives into how to get started with GatewayAPI and Traefik.

In 2015, when Traefik was just born, Kubernetes released the first version of its Ingress specifications. The goal of Ingress was to provide a straightforward, vendor-neutral method for exposing Kubernetes services. However, the premise has quickly been broken.

While Ingress could handle basic service exposure, adding functionalities like rate-limiting or IP checking required numerous vendor-specific annotations.

To address these limitations, a new standard began to take shape in 2019: GatewayAPI was born. Officially released in its v1.0 form at the end of 2023, the GatewayAPI aims to be the new standard for exposing services in Kubernetes. It offers a set of core rules designed to meet most needs while allowing the Gateway Controller to extend these rules with their features.

Traefik has been a Kubernetes first-class citizen for a long time, initially supporting Ingress and later expanding with our IngressRoutes to surpass Ingress limitations, adding features like TCP, UDP, and structured options declaration.

We quickly embraced the big changes in Kubernetes with the Gateway API, offering early experimental support since 2022.

With the release of Traefik v3.1, we’ve taken a major step forward: our GatewayAPI provider is now production-ready. Traefik v3.1 today meets and exceeds 100% of the core requirements, as shown in our SIG network conformance tests report.

Let’s see together how you can start with GatewayAPI and Traefik v3!

Install Traefik as GatewayController

To begin, ensure you have installed the most recent version of Traefik on your Kubernetes cluster, utilizing the Helm Chart for streamlined deployment. This setup enables the GatewayAPI provider to effectively discover and expose your applications.

Define your installation configuration

First, customize your values to enable Traefik to discover GatewayAPI objects on your Kubernetes cluster. Create a values.yaml file with the following minimal configuration:

## File values.yaml ##
providers:
  # Disable the Ingress provider (optional)
  # We do not want to use Ingress objects anymore!
  kubernetesIngress:
    enabled: false
  # Enable the GatewayAPI provider
  kubernetesGateway:
    enabled: true
# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  namespacePolicy: All

Next, execute the following commands to deploy Traefik in the traefik namespace using the previously described configuration:

# helm commands to execute:

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ kubectl create namespace traefik
$ helm upgrade --install --version 0.29.1 --namespace traefik traefik traefik/traefik -f values.yaml

And voilà! You have deployed Traefik on your Kubernetes cluster:

$ kubectl describe deployments.apps traefik --namespace traefik
Name:               	traefik
Namespace:          	traefik
…
  Containers:
   traefik:
	Image:   	docker.io/traefik:v3.1
	Ports:   	9100/TCP, 9000/TCP, 8000/TCP, 8443/TCP
	Host Ports:  0/TCP, 0/TCP, 0/TCP, 0/TCP
	Args:
  	--entryPoints.web.address=:8000/tcp
  	--entryPoints.websecure.address=:8443/tcp
  	--api.dashboard=true
  	--ping=true
  	--providers.kubernetescrd
  	--providers.kubernetesgateway
  	--entryPoints.websecure.http.tls=true
	…

As you can see, both kubernetescrd and kubernetesgateway providers are enabled. The kubernetescrd provider allows you to define Traefik specific resources like Middlewares.

The entrypoints web and websecure will be used to expose your applications.

Additionally, when Traefik is installed with the GatewayAPI provider enabled, it automatically creates a default GatewayClass named traefik:

$ kubectl describe GatewayClass traefik
Name:     	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	GatewayClass
…
  Controller Name:  traefik.io/gateway-controller

As you can see above, the option Controller Name is set to traefik.io/gateway-controller. Thus, it allows you to expose every Gateway attached to the traefik GatewayClass using your Traefik Gateway Controller.

Additionally, the Gateway traefik-gateway is also deployed:

$ kubectl describe Gateway traefik --namespace traefik
Name:     	traefik-gateway
Namespace:	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	Gateway
…
Spec:
  Gateway Class Name:  traefik
  Listeners:
	Allowed Routes:
  	  Namespaces:
    	    From:  All
    Name:  	web
    Port:  	8000
    Protocol:  HTTP

This Gateway allows you to expose HTTPRoute in HTTP on the port 8000 of your Traefik Gateway Controller.

Let’s now expose your first application using a GatewayAPI HTTPRoute.

Deploy your first HTTPRoute

A GatewayAPI HTTPRoute enables you to expose an application through Gateways using HTTP(S).

To do this, reference the Services to reach (using the backendRefs option) and the Gateways (using the parentRef option) as shown in the snippet below:

# Application to expose
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
    app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
---
# Service to reach the application on the cluster
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: whoami
  selector:
    app: whoami
---
# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80

This example demonstrates how to expose a web server that displays some headers when Traefik is accessed using the hostname whoami.myexample.io.
You can use the following curl command to verify that the application is correctly exposed:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-7nqmf
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.9
IP: fe80::e8c0:86ff:feba:5e06
RemoteAddr: 10.42.0.14:59316
GET / HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Real-Ip: 10.42.0.1

Congratulations, you've successfully deployed your first HTTPRoute with Traefik! Now, let's take it a step further by adding a few operations to the requests.

Go beyond with Filters

GatewayAPI Filters enable Traefik to perform various operations on requests and responses.

There are three types of filters:

Core Filters: Mandatory filters for every GatewayController, such as requestHeaderModifier and requestRedirect.
Extended Filters: Optional filters for GatewayControllers, such as responseHeaderModifier and requestMirror.
ExtensionRef Filters: Additional filters provided by the GatewayController. In Traefik, these are the HTTP middlewares.

Let's modify the HTTPRoute to:
Add the header x-post-topic using the core filter.
Add the prefix gatewayapi to the request using the middleware AddPrefix.

# HTTPRoute with the Filters
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80
    filters:
    # Core filter which adds a header
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: x-post-topic
            value: GatewayAPI
    # ExtensionRef filter to use the Traefik Middleware AddPrefix
    - type: ExtensionRef
      extensionRef:
        group: traefik.io
        kind: Middleware
        name: addprefix
---
# Traefik Middleware AddPrefix
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: addprefix
  namespace: whoami
spec:
  addPrefix:
    prefix: /gatewayapi

You can check the modification using a curl command:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-2xkpr
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.10
IP: fe80::b8b5:6fff:fe48:19e9
RemoteAddr: 10.42.0.14:40044
GET /gatewayapi/ HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Post-Topic: GatewayAPI
X-Real-Ip: 10.42.0.1

As we have seen, the GatewayAPI integration in Traefik simplifies the initial setup significantly. We successfully exposed our first HTTPRoute and added a couple of operations to the requests. Traefik has many more GatewayAPI capabilities, such as TCPRoute, TLSRoute, and ReferenceGrant that are not covered in this blog post, but we highly encourage you to explore them further.

Conclusion

GatewayAPI is undoubtedly the new standard for exposing your resources in a Kubernetes cluster. Its specifications already surpass those of Ingress, and with a strong community backing it, GatewayAPI will continue to grow.

As a Kubernetes first-class citizen, Traefik is of course strongly supporting this effort and will continue to contribute to this community project.

We've improved status management, updated route priority rules, and introduced the ReferenceGrant feature to make Traefik even more robust and flexible for your Kubernetes setup.

But it’s just the beginning! We plan to further advance this integration by adding support for features like GRPCRoute, UDPRoute, and BackendTLSPolicy.

Our goal is to provide comprehensive support, enabling you to fully leverage GatewayAPI in your Kubernetes journey. Stay tuned for more updates!

Useful Links

]]>

Announcing Traefik Proxy v3.1

Emile Vauge — Tue, 16 Jul 2024 17:40:52 GMT

Traefik v3.0 was released less than 3 months ago with key new features introduced like WASM, Open Telemetry, and Kubernetes Gateway API support. The feedback from the community members about the v3 has been overwhelmingly positive (thank you!) and extremely motivating to continue improving Traefik Proxy. Today we are thrilled to release the v3.1 which further enhances WASM and Gateway API integrations.

With Gateway API now poised to become the new standard for exposing resources within a Kubernetes cluster, we are proud to announce that Traefik v3.1’s Kubernetes Gateway API is ready for production use 🎉

Let’s jump in!

Gateway API now production ready

Traefik has been a GatewayController since the early days of the Gateway API specification, but needed some adjustments to pass the conformance tests and fully meet the specification. Traefik v3.1 now meets and exceeds 100% of the core requirements, as shown in our SIG network conformance tests report. Our GatewayAPI provider is now ready for production use!

Oh, by the way, we just released a deep dive into Gateway API & Traefik, we strongly invite you to check it out.

Let’s see what’s new in v3.1 with Gateway API. First things first, If you were already using Gateway API in Traefik, you can now remove the experimental option from the helm chart:

## File values.yaml ##
experimental:
  kubernetesGateway:
    enabled: true

As usual, you can enable Gateway API by simply enabling kubernetesGateway:

## File values.yaml ##
providers:
  # Disable the Ingress provider (optional)
  # We do not want to use Ingress objects anymore!
  kubernetesIngress:
	enabled: false
  # Enable the GatewayAPI provider
  kubernetesGateway:
	enabled: true
# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  namespacePolicy: All

Now that you are up and running, let’s dig deeper into the changes made in Traefik.

Improved Status Management

Status management is an important piece of the Gateway API, enabling real-time monitoring of your infrastructure to ensure your GatewayAPI objects are ready to manage traffic. In Traefik v3.1, we've improved support for status management across all GatewayAPI objects, especially HTTPRoutes. Our status calculation now aligns perfectly with the GatewayAPI specification, providing you a ready-for-production status monitoring of your objects.

Route Priority Updates

Traefik has always used a route priority mechanism based on the rule length to prevent overlaps.

However, the GatewayAPI specification has its own priority rules, which differ from Traefik. In Traefik v3.1, we've updated our priority calculation for HTTPRoutes. This ensures that you can switch to Traefik from any other GatewayController without changing your route-matching system.

Introducing ReferenceGrant

The Ingress specification has always struggled with cross-namespace references because of security issues. However, in some cases, like multi-tenant environments, cross-namespace references are essential. Traefik has previously allowed such references through our custom IngressRoute, by adding a specific option to allow it (though security concerns remain).

With GatewayAPI's ReferenceGrant object, these security issues are addressed.

Using ReferenceGrant, you can now declare a Gateway in Traefik that serves a TLS certificate stored in a secret from another namespace or an HTTPRoute targeting a Service in a different namespace. This new feature makes cross-namespace referencing secure and straightforward.

# HTTRoute in the default namespace.
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-http
  namespace: default
spec:
  parentRefs:
    - name: traefik
      kind: Gateway
  rules:
     - backendRefs:
        - name: whoami
          namespace: whoami
          port: 80

# ReferenceGrant and Service in the whoami namespace.
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: whoami
  namespace: whoami
spec:
  from:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      namespace: default
  to:
    - group: ""
      kind: Service
      name: whoami

---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
spec:
  selector:
    app: whoami
  ports:
    - port: 80

Beyond the core features

In addition to the core features required to meet the specification, GatewayAPI offers optional extended features that a GatewayController can implement. Traefik v3.1 brings several of these extended features, including HTTPURLRewriteFilter, HTTPRouteRedirect, and support for method and query parameter matching.

More than the specification

When we started our journey with GatewayAPI, our goal was not just to meet the specification but to bring the same robust feature set as our own Kubernetes provider. That's why Traefik v3.1 also includes support for TCPRoute and TLSRoute, as well as the ability to add Traefik middlewares to your HTTPRoutes using the ExtensionRef mechanism. These features provide even more flexibility and control over your traffic management.

# HTTRoute in the default namespace.
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-http
  namespace: default
spec:
  parentRefs:
    - name: traefik
      kind: Gateway
  rules:
    - backendRefs:
       - name: whoami
         namespace: default
         port: 80

      filters:
       - type: ExtensionRef
         extensionRef:
           group: traefik.io
           kind: Middleware
           name: add-prefix


# Traefik Middleware.
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: add-prefix
  namespace: default
spec:
  addPrefix:
    prefix: /prefix

WASM better than ever

Traefik v3.0 introduced support for WASM plugins along with the original Yaegi plugins. However, the WASM support had some limitations. One of the limitations was the inability to make HTTP calls using the Go standard library from plugins. The technical reason for this is that a function export mechanism is needed for this, and it’s not yet implemented in Go (but is an accepted proposal).

The team found a workaround to simulate WASM exports in a WASM compiler in Go. It means, as of today, it is possible to create a full featured WASM plugin in Traefik which does HTTP calls (through the host), import any Go library, etc.

On top of that, we added the possibility to mount shared directories in plugins and also to configure environment variables.

experimental:
  plugins:
    example:
      moduleName: github.com/traefik/plugindemowasm-http-call
      version: v0.0.2
      settings:
        mounts:
          - /path:/path:ro # Read only mount
          - /tmp/test:/tmp/test # Read Write mount
        envs:
          - TEST_ENV_1
          - TEST_ENV_2

Here is a simple example of a plugin in wasm that makes HTTP calls through the host.

# Static configuration
experimental:
  plugins:
    example:
      moduleName: github.com/traefik/plugindemowasm-http-call
      version: v0.0.2
---
# Dynamic configuration
http:
  routers:
    my-router:
      rule: host(`demo.localhost`)
      service: service-foo
      entryPoints:
        - web
      middlewares:
        - my-plugin

  services:
   service-foo:
      loadBalancer:
        servers:
          - url: http://127.0.0.1:5000
  
  middlewares:
    my-plugin:
      plugin:
        example:
          headerName: X-World-Time

Thanks to this innovative and unique approach, WASM is now a powerful plugin platform in Traefik that we will continue to improve upon moving forward.

Other Improvements

Several contributions were made on Kubernetes:

By Marc Mognol who brought health checks to ExternalName services
By Jesper Noordsij who migrated the Kubernetes provider to the EnpointSlices API
By Joris Vergeer who added the possibility to use Node IPs for NodePort services

Another great contribution came from Antoine Aflalo who added Zstandard to the compress middleware (in addition to Gzip & Brotli). This algorithm is much faster, especially at decompressing.

Support for Content-Security-Policy-Report-Only was added by Roman Donchenko to the headers middleware.

Finally, we added support to systemd socket activation, which allows systemd to listen on socket and dynamically start the associated service. Simply use the same name for your entrypoint and file descriptor, and Traefik will start on systemd demand.

The full release note is available here.

Conclusion

Traefik release notes are usually pretty packed and this one is no exception. Traefik v3.1 brings critical features to the project with state-of-the-art WASM support that makes Traefik’s plugin platform one of the best in the industry. Being a Kubernetes native product, it goes without saying that Traefik closely follows the latest evolutions of the platform. Traefik is now getting full support of Gateway API v1.1.0 and is ready for production use.

Traefik is almost 10 years old, but like a good wine, is still getting better and better 🙂.

Lastly, a huge thank you to all contributors. Your assistance is invaluable.

See you on GitHub!

Useful Links

Traefik 3.1 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

The API Gateway Model: Centralizing Control in Complex Microservices Architectures

Immánuel Fodor — Wed, 26 Jun 2024 22:24:48 GMT

Imagine an organization with hundreds or even thousands of microservices, each responsible for a specific function or feature. Routing requests across these services, managing authentication and authorization, load balancing traffic, and ensuring system resilience can quickly become a daunting task.

Mortgage provider Amerisave, with a presence in 49 states, 5,000 employees, and on-prem legacy infrastructure with more than 150 servers handling up to 40,000 requests per second, had to consider that in its migration to microservices using Docker Swarm. Without centralized control of routing, security, and other distributed functions, Amerisave and organizations like it risked introducing silos, inconsistencies, and potential security vulnerabilities, ultimately making it more difficult to scale and adapt to changing business needs.

One such solution that exemplifies the principles of centralized control of distributed functions is the use of an API gateway.

The API Gateway Model

The API gateway model provides companies with a centralized way to manage routing, authentication, and authorization. This model enables developers to offload the complexities of routing and security to DevOps, platform, or security teams, allowing developers to focus on building the applications. Let’s discuss the architecture of this model.

But first, what is an API gateway?

An API gateway is a single entry point into your microservices that routes your clients’ API requests to the backend microservices in your network. An API gateway functions very similarly to a reverse proxy for standard user-facing applications in that it routes incoming requests to the correct API in the backend.

Considering we have covered this already in our glossary, I won’t be going in more depth here. If you need a refresher on the basics, please see this API Gateway glossary article.

API gateway in modern infrastructure

Because API gateways live in front of all services and DevOps teams are able to centralize control, the API Gateway model is far more efficient than models that require, for example, authentication and authorization to be configured in each service.

There are numerous benefits to the API gateway model, a few of which we’ll explore here.

Improved Routing, Load Balancing, and System Resilience

At the core of centralized control lies the ability to consolidate routing logic and load balancing mechanisms. Because they’re the single entry point for all external traffic, API gateways simplify routing in complex architectures and eliminate the need for individual microservices to handle routing logic independently, ensuring consistency, reducing redundancy, and facilitating easier maintenance and updates.

Additionally, modern, cloud native API gateways offer dynamic service discovery with automated default configurations, advanced load balancing (blue/green, canary, etc.), and intelligent routing based on factors such as resource utilization, response times, and health checks. Add rate limiting and circuit breaking to this list, and you can see why API gateways are effective tools for streamlining the management of complex architectures, fostering scalability, and enhancing system resilience.

Better Authentication and Authorization Management

A core reason companies move towards an API gateway model is security. Not only do API gateways tend to be compatible with modern authentication and authorization protocols (OIDC, JWT, etc.), but they also streamline security management by centralizing access control and even helping to automate it.

Managing access control at the service level, in contrast, requires tedious manual configuration, increases the risk of human error, and opens you up to attacks. Add to this that most companies have hundreds, if not thousands of microservices and APIs to expose, and you can see how embedding access control into each service is not scalable and overly burdensome on developers.

Managing access control on service level

By enforcing authentication and authorization policies at the API gateway level, you’re able greatly simplify the configuration and maintenance of new and existing access control policies. And with everything in one place, managed by one team, policies can be enforced consistently (and even automatically with dynamic service discovery and configuration). This leaves less room for human error, reduces development overhead, and minimizes vulnerabilities from inconsistent implementations across different services.

Managing access control via API gateway model

If you choose a modern API gateway for your stack, you’ll also be able to take advantage any authentication and authorization protocol—including LDAP, JWT, OAuth, OpenID Connect, HMAC, and API keys—as well as direct integrations with identity providers (IdPs). The best gateways also support advanced authorization mechanisms, such as Open Policy Agent (OPA), which allow organizations to implement fine-grained access control policies based on complex rules and contextual data.

Streamlined Change Management

As new instances of microservices are introduced or existing ones are scaled up or down, they have to be configured. Whether it’s registering, routing, load balancing, or securing the service, change management is no small feat.

And the more decentralized and manual this process, the more likely you are to have issues. With the API gateway model, however, this can all happen at a single point, ensuring optimum consistency and control.

The best API gateways take this even further and automate service discovery and default configuration. As your team spins up new services or makes changes to existing ones, the API gateway gets to work. New services are registered and the default routing, load balancing, rate limiting, access control, and other defaults are applied dynamically. The best solutions are able to deploy these changes dynamically without restarts or service disruptions.

This dynamism ensures teams remain agile and responsive to changing business needs, which is crucial in today's fast-paced business environment.

Comprehensive Monitoring and Observability

Monitoring and observability are essential. Whether it's identifying performance bottlenecks, detecting issues before they happen, or ensuring compliance with service-level agreements, the ability to monitory your system is a must.

As architectures become more complex, having a centralized tool that gathers and monitors system data becomes all the more important. This is where an API gateway model can help.

Monitoring traffic flow, load balancing, and overall system health is much more challenging without a centralized point through which all this data flows. And considering how API gateways help you centralize routing, security, etc., observability is no different.

API gateways offer some built-in metrics that cover a wide range of dimensions. These can include request rates, response times, error rates, and more. They may also provide built-in dashboards or the ability to visualize data using third-party observability tools, such as Prometheus, Grafana, and Elasticsearch.

This is nice, but modern API gateways take observability to the next level. They are increasingly aligned to OpenTelemetry (OTel), a standard for dynamic systems telemetry data that replaces multiple vendor-specific solutions. OTel unifies disparate data, ensures consistent instrumentation across various programming languages, and offers enhanced insights into application use, health, and performance.

For example, API gateways with best-in-class OTel support can illuminate request patterns, authentication and authorization events, system-level events, and issue correlations. You may even have the ability to remove or mask GDPR-related Personally Identifiable Information (PII) in logs to comply with security requirements. This level of visibility and control enables organizations to troubleshoot issues more effectively, identify potential security threats, and optimize their applications performance.

Integration with Orchestration Tools

The last benefit of the API gateway model we’ll discuss here is the stitching together of complex, often disparate, environments. As organizations move from monolithic to microservices architectures, they often need to manage routing, security, et al. across hybrid cloud, multi-cloud, and mulit-orchestrator deployment models.

This is no easy feat. And the larger the company, the more you’re likely have have different teams working in different environments. So how can you tame such a complex beast?

API gateways can be used as a unifying layer, ensuring consistent control and governance over distributed systems, regardless of the underlying infrastructure or orchestration platform. One consideration here is that not all API gateways are natively compatible with all orchestrators.

This is another situation where modern, cloud native API gateways shine. Regardless of your orchestrator or deployment model, the best gateway solutions will behave the same in all instances. They don’t require you to operate them differently depending on the environment, and they might even be able to communicate with each other to further centralize your operations.

With this level of flexibility, your organization can grow and change, being responsive to it’s needs today and in the future, without the pressures of vendor lock-in or imminent rip and replace headaches.

It’s Time to Centralize Control

The API gateway model has clear advantages, particularly as infrastructures scale and become more complex. Centralizing critical functions, such as routing, security, load balancing, and observability can have profound effects on your organization. Efficiency, consistency, agility, and scalability can all increase dramatically when an API gateway model is implemented well (and when the API gateway you choose is right for you).

Modern, cloud native API gateways magnify all the benefits of the API gateway model. From dynamic configurations to native support for all orchestrators and deployment models, the best of the best API gateways tame complexity and make it possible to scale how and where you need.

Are you ready for that?

]]>

5 Essential Qualities All API Gateways Need in 2024

Immánuel Fodor — Thu, 06 Jun 2024 22:49:07 GMT

In the constantly evolving digital landscape, APIs are the connective tissue that powers modern applications. A well-designed API gateway acts as the central nervous system for your API ecosystem, streamlining access, management, and security. But with the rapid pace of innovation, simply having an API gateway isn't enough. In 2024, you need a modern API gateway to stay ahead of the curve and not be left behind in the digital stone age.

Let’s explore the five essential features your API gateway must possess to empower your developers, elevate security, and maximize business agility. We'll explore simplicity and ease of use, container orchestration integration, security features, extensibility, and embracing GitOps principles for declarative configuration. With these at the top of your evaluation criteria, you'll be better equipped to select the perfect API gateway for your organization's needs.

#1 Simplicity and Ease of Use are The Cornerstone of Agility

Developers are your key partners. A modern API gateway should prioritize developer experience (DX) by offering a clean, intuitive interface and a robust set of tools, ensuring they can navigate, configure, and deploy APIs and microservices without unnecessary friction. This translates to:

Reduced learning curve: A convenient user interface and clear documentation enable developers to grasp the API gateway's functionalities quickly, minimizing onboarding time and maximizing productivity. Note that a user interface doesn’t necessarily mean a clickable UI; it can also be any interface that allows configuration.
Streamlined development: Powerful tools for defining routes, security policies, and request or response modification should be readily available along with automation, allowing developers to focus on business logic rather than getting bogged down in configuration.
Rapid prototyping and iteration: Simplicity fosters agility. The ease of publishing and modifying APIs within the gateway empowers developers to prototype and iterate, accelerating innovation and time-to-market rapidly.

Here are two real-world examples: Axione, a French telecommunications company, leveraged a user-friendly API gateway to save countless hours every month by reducing configuration updates in multiple clusters to just a few minutes with zero downtime. ABAX, a multi-national IoT (internet of things) company, spends very little time training people. They give them the basic understanding they need and then move on. Saving so much time, like these companies, can also allow you to capitalize on a critical market window that might never come again.

But simplicity extends beyond developer efficiency. A modern API gateway should also champion:

Faster time to change: Configuration changes should be straightforward and easily rolled out, minimizing downtime and disruption during updates. Imagine a scenario where a critical bug fix needs immediate deployment. A modern, streamlined API gateway enables this with minimal hassle.
Rapid disaster recovery: In the event of an outage, a modern API gateway with predictable configuration rollback features allows for swift restoration, ensuring business continuity and minimizing revenue loss. According to the trusted DORA metrics, companies with high, on-demand deployment frequency can minimize the change-failure rate to only 5% and recover from a failure in less than 1 hour (which can still cost businesses an average of $9,000 per minute, so you should cut it down as much as possible). Since the industry average for recovery is between a day and one week, a speed like this can put you at the top performing 20%, minimizing customer impact and protecting your brand reputation.

Finally, a modern API gateway shouldn't burden developers with tedious configuration tasks. Here's where features like the following help:

Sane defaults: Out-of-the-box configurations catering to common use cases minimize manual setup and avoid divergence from industry best practices.
Auto-discovery of services: Modern API gateways can automatically detect and integrate with backend services, saving developers valuable configuration time.
Scalability and high availability: The gateway should seamlessly scale to accommodate traffic spikes and ensure continuous API access, freeing developers from infrastructure concerns and allowing them to focus on building exceptional applications.

While many API gateways exist, some fall short on simplicity and ease of use. Avoid UI-heavy solutions because a complex, cluttered user interface can hinder developer productivity. Some API gateways on the market even require navigating a labyrinth of menus and options just to perform basic configurations.

Don’t frustrate your developers, nor waste their time! Also, beware of manual drudgery and gateways without auto service discovery. Manually configuring every backend service integration can be a tedious and error-prone process, especially for large deployments.

#2 Embrace the Container Revolution with Native Integrations

The landscape is shifting. Bare-metal deployments and virtual machines (VMs) are giving way to cloud-native architectures built on containerization. But considering cloud migration doesn’t happen overnight, modern API gateways need to be able to handle both legacy and cloud-based workloads.

Deep container orchestration integration, first and foremost, means Kubernetes. Kubernetes reigns supreme in the container orchestration realm. A modern API gateway should seamlessly integrate with Kubernetes, allowing you to leverage its powerful features for service discovery, load balancing, and health checks. This simplifies API deployment and management within your containerized environment.

But what about beyond Kubernetes?

A truly modern API gateway shouldn't limit you. It should offer first-class support for other cloud-native orchestrators like Docker Swarm and HashiCorp Nomad, ensuring flexibility for your specific infrastructure needs.
Modern applications often leverage a diverse set of backend services; furthermore, many organizations operate in hybrid environments, with a mix of on-premises infrastructure and cloud deployments. A modern API gateway should bridge this gap and be able to serve APIs from many sources simultaneously. It should seamlessly expose APIs residing on-premises and across various cloud providers and Kubernetes distributions. This flexibility empowers you to manage your entire API ecosystem from a single pane of glass.

Effective traffic routing is crucial for application performance and user experience to support all this complexity. A modern API gateway acting as the central hub for all your API traffic should offer intelligent routing capabilities, allowing you to route requests based on various factors like hostname, path, HTTP method, request headers, JWT claims, or even a mix of all these. This granularity empowers you to deliver the right service to the right user at the right time, optimizing performance and security.

To give you a real-life example, AmeriSave, a mortgage company in the financial services industry, faced similar challenges as they were migrating from an on-premises monolith to a cloud-based microservices architecture. They needed to keep APIs online from both systems during the transition. A flexible, multi-orchestrator API gateway helped them to make this happen and have a smooth and consistent process.

However, not all integrations are created equal, you should avoid pitfalls while working in the field. While container orchestration offers numerous advantages, some API gateways fall short in this area. Watch out for:

Limited Kubernetes capabilities: While some API gateways might offer a basic Kubernetes integration or can be run inside Kubernetes, true interoperability is key. Avoid API gateways that lack robust Kubernetes features like service discovery, health checks, and Kubernetes-native CRDs (Custom Resource Definition) to declare configuration, as these will only increase the manual configuration burden and potential for human error. Many solutions were born before Kubernetes or even the cloud-native era, so they lack this critical capability.
Cloud vendor lock-in: Beware of API gateways restricting you to a specific cloud provider's (Kubernetes) offering. A modern API gateway should be cloud-agnostic, supporting various Kubernetes distributions across multiple cloud providers. This flexibility ensures you're not locked into a single vendor and can adapt to your evolving needs.
Orchestration singularity: While Kubernetes is the leader, some situations can leverage other orchestrators, and vice versa, one single non-Kubernetes orchestrator support is also not enough. Make sure your API gateways don’t limit you to just one or two environments. Instead, seek one that offers broader support for various container orchestration platforms to make your containerization strategy future-proof.

These are just a few cautionary tales. Remember, a modern API gateway should empower you to build and deploy APIs that align with your unique infrastructure and business situation.

#3 Protecting Your APIs in a Hostile World

Security is non-negotiable in a world marked by relentless cyber threats and strict regulatory requirements. A modern API gateway must serve as the vanguard of your digital fortress, offering a comprehensive suite of security features to safeguard your data, services, and reputation.

Secure a wide range of protocols easily: Modern APIs leverage diverse protocols. A robust API gateway should secure these protocols uniformly at a centralized point. This includes common protocols like HTTP, TCP, UDP, WebSockets, and even gRPC. Centralizing security policies can streamline management and ensure consistent protection across your entire API ecosystem.
Authentication and authorization: The bedrock of any secure API ecosystem lies in robust authentication and authorization mechanisms to control who can access what. A modern API gateway should offer a robust arsenal of options, including industry standards like API keys, LDAP, JWT (JSON Web Tokens), OAuth2, OIDC (OpenID Connect), and HMAC (Hash-based Message Authentication Code). Implement the most suitable authentication mechanisms for each API, ensuring untrusted actors can’t misuse them.
Effective traffic management: This is essential for optimizing performance, enhancing resilience, and ensuring a superior user experience. A feature-rich API gateway facilitates advanced traffic management capabilities, including:
- Traffic mirroring—troubleshooting and security analysis
- Blue/green and canary deployments—safely roll out new API versions with minimal risk
- Session stickiness—maintain user sessions with specific upstream backends
- Active health checks—continuously monitor the health of backend services and ensure API uptime SLAs
- Circuit breakers—protect against cascading failures
- Retries and buffering—enhance API resilience towards temporary network issues
- Distributed in-flight request and rate limiting—evade denial-of-service attacks.
Built-in Let's Encrypt support: Securing communication channels with Transport Layer Security (TLS) is a non-negotiable security best practice. Let’s Encrypt is the most well-known ACME (Automated Certificate Management Environment) compatible TLS certificate authority. A modern API gateway should offer built-in ACME/Let's Encrypt support, simplifying the process of obtaining and managing TLS certificates. This aligns with the esteemed Electronic Frontier Foundation's (EFF) recommendation to leverage tools like Traefik that offer integrated support for certificate automation, moving away from external tools like Certbot.

Without such robust features, Leroy Merlin, a home improvement and gardening retailer, couldn’t have made it "effortless to manage" 50 clusters in parallel securely. With other API gateways, the team would expend time building and maintaining certificates with CertManager or other external tools requiring maintenance, expertise, and attention.

Beware of API gateways that offer a limited security feature set. While they might provide basic functionalities, advanced capabilities like fine-grained security policy enforcement might be missing. This can leave your APIs vulnerable to sophisticated attacks. You can also encounter API gateways that offer robust security features but are limited to a specific orchestration platform. You must not compromise on security and must be able to leverage best-of-breed security solutions across your entire infrastructure.

Remember, security is an ongoing process that requires constant adaptation. Stakes are high, and threats are widespread—avoid gateways that restrict customization and extensibility options to tailor security measures to your specific needs.

#4 Embrace Extensibility for a Future-Proof API Gateway

Meeting your current needs is nice, but a modern API gateway must also evolve alongside your organization's growth and changing requirements.

Imagine you're stuck in a walled garden. You can still move around, but it's hard to get out. Vendor lock-in is similar. The vendor creates a system where switching to another provider is inconvenient and costly, and you must move at the pace the vendor lets you by shipping new features according to their roadmap, not yours. For this reason, it’s best to avoid API gateways that lock you into a specific vendor's feature set. Extensibility empowers you to integrate best-of-breed solutions and customize your API gateway to meet your unique needs when needed.

A rich plugin ecosystem is a hallmark of a modern API gateway, amplifying its versatility. Built-in plugins can extend core functionalities, while the ability to create custom plugins allow you to address specific use cases. Security is a prime example. Custom plugins can be leveraged to fortify the gateway with advanced security features like a Web Application Firewall (WAF).
WebAssembly (Wasm) has emerged as a game-changer in extensibility. By supporting WASM-based plugins, an API gateway opens the door to a world of possibilities, enabling portable, lightweight, high-performance customization without sacrificing security or efficiency. This opens the door to a vast array of innovative plugins that can extend the gateway's capabilities in unprecedented ways.
Open-source development fosters innovation and rapid improvement. An API gateway and a plugin ecosystem built on open-source foundations driven by a passionate community ensures continuous feature enhancements and a future-proof platform. You can also contribute and receive feedback and improvements to your added functionality from the community.

Did you know that Traefik has recently joined OWASP, a well-known non-profit organization focused on improving software security, particularly web application security? OWASP is endorsing the use of the Coraza WAF, which is the successor to ModSecurity-based WAF solutions. Traefik v3 adds support for Coraza as a Wasm-based WAF plugin, created by a community member who is, in fact, the author of the Coraza WAF. What a great example of the opening three points working together!

A modern API gateway with WAF integration empowers you to leverage industry-leading protection mechanisms to safeguard your APIs against sophisticated attacks. This is made possible by the power of extensibility, allowing you to integrate best-in-class security solutions as they become available on the market, not by relying on the vendor's mercy to add them when they feel like it.

Vendor lock-in is also a barrier to observability. A modern API gateway should embrace OpenTelemetry, an industry standard for collecting and analyzing API metrics, traces, and logs. This empowers you to get comprehensive insights into your API ecosystem's health and performance not just from one vendor-specific tool but any compatible tool, even your existing monitoring tools if they support the standard.

What are the signs that your API gateway is failing in these areas?

Some gateways rely on Lua or C-based extensions. While these programming languages have their place, working with them can be cumbersome. Compiling the gateway from source code for even basic extensions adds unnecessary complexity and hinders development agility.
Also, beware of gateways that lock essential features behind a paywall, such as analytics, which should be core offerings in a modern API Gateway.
While WASM promises extensibility and allows you to leverage the language that best suits your development team's expertise, resource consumption remains a concern. A modern API gateway should balance extensibility with resource efficiency, ensuring smooth operation even with a robust plugin ecosystem.
Deep integrations can be convenient, but they can also lock you into a single vendor and hinder your ability to leverage best-of-breed solutions from other providers. Some gateways are so deeply intertwined with a specific cloud vendor's ecosystem or offered by the cloud provider itself that extensibility becomes an afterthought.

And last but not least, your needs might evolve as your API ecosystem matures. A modern API gateway should make migrating to a more comprehensive API Management solution smooth. Extensibility is key here, ensuring a seamless transition and minimal disruption to your existing API infrastructure.

#5 Taming Complexity with Declarative Magic

We've explored four out of the five essential features of a modern API gateway: simplicity, container orchestration integration, robust security, and extensibility. But how do you harness these capabilities and streamline your API development workflow? Enter GitOps, the unifying force that brings them all together.

GitOps embraces a declarative approach to configuration management. Instead of writing custom, complex scripts procedurally, developers simply define the desired state of the API gateway in a Git repository. They can then focus on configuring security settings, traffic management policies, plugins, and more through a declarative, code-based approach, leaving the underlying infrastructure management to GitOps automation.

This beautifully fits with our emphasis on developer experience. By leveraging Kubernetes CRDs stored in Git, GitOps eliminates the need to learn complex configuration languages specific to the API gateway. It also makes auditing changes a breeze, enhancing security and deploying changes easily without manual tinkering. All these translate to faster onboarding, fewer errors, and a more productive development environment.
Remember our focus on simplicity? GitOps delivers on this promise by enabling you to deploy your API gateway configuration to any environment using the same GitOps workflows. Imagine seamlessly deploying your API gateway configuration from development to staging and then to production just by using simple Git commits, automatically checking for mistakes in the process. This consistency minimizes errors and streamlines the deployment pipeline across your entire infrastructure.
GitOps also empowers you to leverage Git's power for security and incident management. Version control ensures a clear audit trail of all configuration changes, allowing you to easily roll back to a previous version in case of an issue. Additionally, Git branching strategies can be used to implement secure development workflows, such as pull requests for reviewing changes before deployment. This fosters a culture of security by design within your API development lifecycle.
Managing API versions has never been easier; GitOps excels at it. Each commit in your Git repository represents a specific API gateway configuration. Developers can experiment with new API versions in isolation before deploying them to production, minimizing risk and ensuring a smooth evolution of your API ecosystem.

For a deeper look at how GitOps empowers platform teams, check out this other article on the Traefik Labs blog: How Modern API Gateways Make DevOps Engineers More Efficient.

While GitOps offers a compelling approach to managing API gateways, some fail to deliver in this crucial area:

Avoid gateways that require additional tools, pretending they comply with GitOps-based workflows. A modern API gateway should natively embrace GitOps for declarative configuration management. However, some gateways require convoluted GitOps pipelines that become cumbersome to manage, especially when scaling to many APIs. This defeats the purpose of GitOps—it should simplify, not complicate.
Relying on an imperative (and not declarative) administrative API approach can also be inflexible and error-prone. Declarative GitOps empowers you to define the desired state, leaving the "how" to the automation tools. Rely on it for its clarity, version control benefits, and ease of rollback.
Modern gateways embrace configuration hot reloading, minimizing downtime during configuration updates, which can be frequent with GitOps. Avoid gateways that require full proxy reloads after every configuration change, losing active connections and hurting uptime.
And finally, if GitOps compatibility isn't a major focus for the API gateway vendor, it might not be the future-proof solution you seek.

Wrapping It Up

These days, applications often rely on many microservices, each with its own API, effectively increasing complexity to unprecedented levels. The API gateway has emerged as the cornerstone of this, now, often virtualized infrastructure. It is the gatekeeper between internal systems, external partners, and end-users, orchestrating the data flow and interactions with great finesse and efficiency.

But not just any API gateway will do. Only a modern solution equipped with the discussed 5 essential capabilities empowers you to build exceptional APIs, accelerate development lifecycles, and confidently navigate the ever-evolving landscape of application development.

Want to know more about choosing an API gateway in 2024? Download our free API gateway Buyer’s Guide by clicking the promo below for more details.

]]>

Fortify Your Frontlines: Distributed Security with HashiCorp Vault, Let's Encrypt, and Traefik

Immánuel Fodor — Tue, 28 May 2024 18:47:00 GMT

Distributed systems have become the backbone of the digital economy, enabling the seamless operation of everything from cloud services to mobile applications. These systems have specific requirements when it comes to encryption, which is provided by Transport Layer Security (TLS). With Traefik's API Gateway, TLS support can be handled by a combination of a Certificate Authority (CA) aligned to the ACME protocol (e.g., Let's Encrypt) and HashiCorp Vault, which collectively secure your distributed systems and provide key management while maintaining high availability.

In this article, we'll explore how these tools combine at a high level to provide encryption for distributed architectures and why that's important.

Note: For a deep dive into how to configure this in your environment using Traefik and HashiCorp Vault's PKI Engine, Nomad, and Consul, see this post by Open Source Consultant and Trainer Chris van Meer.

Why a Distributed System Requires TLS

A distributed system, at its core, is a collection of independent computers that appear to its users as a single coherent system. This architecture is designed to manage tasks that are too complex for a single computer by dividing them across multiple machines, thus leveraging the combined processing power, storage capacity, and specialized functionalities of all of them. The distributed nature of these systems offers several advantages, including scalability, fault tolerance, and resource sharing. However, these benefits also introduce unique challenges, particularly in the realm of security, that require the use of an encryption protocol such as TLS:

Decentralized Infrastructure and Data Transmission Across Networks: In a distributed system, components communicate over a network, potentially between different geographical locations. They exchange data that can range from sensitive user information to critical operational commands. This transmission occurs over channels that could be accessible to attackers, making the data susceptible to eavesdropping and interception. TLS secures these communications by encrypting the data, ensuring that even if data packets are captured, they cannot be deciphered by unauthorized parties.
Complex Interactions Among Components: Distributed systems often involve complex interactions between various applications, services, and databases, each possibly having its own security mechanisms and vulnerabilities. Consistently implementing TLS across all these interactions ensures a baseline level of security, providing a unified approach to protect against man-in-the-middle attacks and ensuring that data remains secure in transit between components.
Dynamic Scalability: One of the hallmarks of distributed systems is their ability to scale dynamically, adjusting resources in response to fluctuating demand. This scalability often involves the automatic deployment of new instances or services, which must immediately communicate securely with existing components. TLS certificates can be automatically managed and deployed to new instances, ensuring that they are immediately secured and can be trusted by other parts of the system.
Authentication: Beyond encrypting data, TLS also facilitates authentication through the use of certificates, enabling both servers and clients to verify each other's identity. This is particularly important in distributed systems where services may be interacting for the first time or where components are provided by different vendors.
Regulatory Compliance and Trust: Distributed systems, especially those handling personal data or sensitive information, must comply with a myriad of regulations regarding data protection and privacy, such as GDPR or HIPAA. TLS not only helps in complying with these regulations by securing data in transit but also bolsters trust among users and stakeholders by demonstrating a commitment to security.

In other words, the intrinsic characteristics of distributed systems—their networked, decentralized, and dynamic nature—significantly increase their vulnerability to security threats. TLS addresses these vulnerabilities by providing a robust mechanism for encryption, authentication, and data integrity across all communications. In other words, implementing TLS is not just about protecting data; it's about ensuring the resilience and reliability of the distributed system in the face of evolving cyber threats.

But in order to make this happen in an automated environment, we need a way to programmatically request and manage the TLS certificates that form the foundation of the protocol. Fortunately, we have Let’s Encrypt. Let's Encrypt is a non-profit CA that plays a pivotal role in securing distributed systems by offering free, easily managed TLS/SSL certificates. Its use of the ACME protocol automates certificate management, reducing the administrative burden and minimizing the risk of security lapses. By leveraging Let's Encrypt (or another ACME aligned certificate authority), organizations can ensure their components communicate securely, contributing to the safety and reliability of their services and the internet at large.

What is Vault and How is It Involved in This Process?

HashiCorp Vault stands out as a pivotal component in securing distributed systems, primarily due to its comprehensive approach to managing secrets, such as API keys, passwords, and TLS/SSL certificates. Its role in these systems extends beyond mere storage, encompassing secret management, data encryption, and access control, all of which are crucial for maintaining the integrity and confidentiality of communications within distributed architectures. Vault is involved in this process in several ways:

Centralized Secrets Management: In distributed systems, the need to securely manage a multitude of secrets across various services and environments is paramount. Vault centralizes the storage of these secrets, offering a single point of control and auditing. This centralization simplifies the secrets management process, reducing the risk of leaks or unauthorized access that could compromise the system's security.

Vault's dynamic secrets system is particularly beneficial for distributed systems. Unlike static secrets, which remain the same until manually changed, dynamic secrets are generated on-demand and are valid for a strictly limited duration. This means that even if a secret were to be exposed, its short lifespan significantly limits the potential for misuse. This feature is invaluable for TLS, where temporary credentials can be used for encrypted sessions, minimizing the risk associated with long-lived certificates.

Encryption as a Service: Vault provides encryption services, enabling applications to encrypt and decrypt data without managing encryption keys directly. This functionality supports the secure transmission of data across distributed systems by ensuring that sensitive information remains encrypted both at rest and in transit. When combined with TLS, which encrypts data during transmission, Vault's encryption services offer an additional layer of security for data stored within the system.
Automating Certificate Management: Vault integrates seamlessly with the TLS process by automating the management of TLS/SSL certificates. It can issue certificates directly using its internal CA, or it can act as an intermediary with external CAs, including those compatible with the ACME protocol, such as Let's Encrypt. This automation extends to the renewal and revocation of certificates, ensuring that distributed systems consistently use valid certificates without manual intervention.

Managing the lifecycle of TLS/SSL certificates is a critical aspect of maintaining secure communications. Vault automates the renewal process, issuing new certificates before the old ones expire, and can revoke certificates that are no longer needed or have been compromised. This capability is crucial for distributed systems, where outdated or revoked certificates can lead to security vulnerabilities or system outages.

In essence, Vault's comprehensive suite of features for managing secrets and certificates plays a vital role in the security and operational efficiency of distributed systems. By automating the creation and revocation of TLS/SSL certificates, Vault acts as a middleman that enables Traefik's API Gateway to renew certificates, and to ensure that secure, encrypted communication is a standard practice, not an afterthought. Its involvement in the TLS process not only bolsters security but also contributes to the reliability and resilience of the distributed system as a whole.

What is ACME and How Does It Help Secure Distributed Systems?

The Automated Certificate Management Environment (ACME) is a protocol designed to automate interactions between CAs and web servers, streamlining the process of obtaining, renewing, and revoking digital certificates. ACME has several goals:

Simplifying Certificate Issuance: ACME standardizes the process for verifying domain ownership and automating certificate issuance. Systems can programmatically request and receive certificates, ensuring that secure, encrypted communication channels can be established without manual intervention.
Automating Renewals and Revocation: Certificates have a limited validity period and need to be renewed regularly to maintain secure connections. ACME automates this renewal process, allowing systems to automatically request new certificates as expiration dates approach. Similarly, if a security breach occurs or a certificate is otherwise compromised, ACME can facilitate the rapid revocation of these certificates, helping to minimize potential security risks.
Domain Validation: ACME includes mechanisms for automated domain validation, a prerequisite for issuing a certificate. This process verifies that the requester has control over the domain for which the certificate is being requested. ACME automates this verification through several methods, such as placing a specific file in a predefined directory on the web server or making certain DNS changes. For distributed systems, this automation removes a significant barrier to securing communications, ensuring that validation and certificate issuance can occur seamlessly as new services are deployed or as systems scale.

Vault as an ACME Client

At the heart of the Vault and ACME integration is Vault's ability to function as an ACME client. This capability enables Vault to directly interact with ACME-compliant CAs, such as Let's Encrypt. Vault can automatically request, renew, and revoke TLS/SSL certificates based on predefined policies and the needs of the system it secures. This process is managed through Vault's interface, leveraging ACME to handle the operational details with the CA.

This integration provides centralized management of certificates: integrating Vault with ACME provides a centralized platform for managing all aspects of TLS/SSL certificates within a distributed system. Administrators can define policies within Vault that dictate how certificates are issued, renewed, and revoked. This centralized approach not only simplifies management but also provides a clear audit trail of certificate-related activities, enhancing security and compliance.

The integration of Vault and ACME transforms the management of TLS/SSL certificates from a complex, manual task into a streamlined, automated process. This integration not only enhances security and compliance but also simplifies the operational aspects of managing certificates in distributed systems, enabling organizations to focus on their core objectives while maintaining a strong security posture.

How Traefik’s API Gateway Optimizes Distributed Systems

In distributed systems, the API gateway serves as a critical intersection, managing incoming traffic to various application components securely and efficiently. Traefik's API Gateway’s design optimizes this process by minimizing latency and ensuring that connections are always directed to the optimal endpoints, thereby enhancing overall system performance and reliability.

In other words, Traefik's API Gateway simplifies the routing of client requests to the appropriate backend services, ensuring efficient load balancing, traffic management, and network resilience. Key features include:

Dynamic Configuration: Unlike traditional API gateways, Traefik automatically detects changes in service configurations within a cluster, adapting routes without requiring manual updates or restarts. This dynamic response is essential for environments where services are frequently scaled or updated.
Built-in Security with TLS Management: Traefik integrates seamlessly with Let’s Encrypt to automate TLS certificate generation and renewal, ensuring encrypted and secure communications without manual intervention. This automation is critical for maintaining continuous security compliance and protecting data integrity across services.
Middleware Customization: Traefik allows the use of various middlewares that can modify requests and responses, implement additional security checks, or manage access control, providing enhanced flexibility to meet diverse operational requirements.
Observability and Monitoring: With native support for various monitoring tools, Traefik can enable these tools to provide detailed insights into API traffic patterns and health metrics, facilitating proactive management and optimization of network resources.

Traefik's API Gateway stands at the forefront of network management solutions by providing an adaptive, secure, and efficient routing mechanism that supports the complex demands of modern distributed systems. Its ability to integrate advanced security protocols, coupled with dynamic configuration capabilities, makes it an indispensable tool for developers and enterprises aiming to optimize application delivery and performance.

Traefik vs Traditional API Gateway Considerations

While Traefik offers several enhancements over traditional API gateways like NGINX, there are challenges to consider:

Complexity in Initial Setup and Learning Curve: The dynamic and flexible nature of Traefik's configuration might present a steeper learning curve compared to others with a history predating the cloud-native era. Organizations might find the initial setup of Traefik to be a bit more complex due to its abstracted and automated mechanisms but the initial investment pays off in the long run. Traefik also offers easy to understand video courses to cover from the basics to advanced load balancing in the free Traefik Academy.
Dependency on External Services: Traefik’s effectiveness, especially in certificate management, often hinges on seamless integration with external services such as Let’s Encrypt. This can introduce a dependency that might affect gateway functionality in the unlikely event that these services go down, but this is the case with any other gateways that integrate with external services. However, more manual control over certificate management usually isn't worth the effort. Even the Electronic Frontier Foundation (EFF) argues that Traefik should replace certbot, an auxiliary certificate automation tool, as built-in certificate automation is the future.
Resource Usage: Traefik’s advanced features and continuous monitoring for changes can lead to higher resource usage than others with minimal static setups. This can be a challenge for environments where strict resource optimization is critical. However, there are always trade offs, and Traefik’s added capabilities and automation can bring many times more value to the table than the saved resource costs.
Mature Ecosystem and Community Support: Users of traditional API gateways might benefit from a longstanding user community and a wealth of knowledge accumulated online. Although there are many plugins that can extend traditional gateway functionality, these often require recompilation or are written in embedded simple languages. Traefik still has some way to go in growing its ecosystem and knowledge base, but it’s growing fast, and has recently added WebAssembly (Wasm) support for its plugin system, besides the existing Go-based solution, to make plugin development even simpler.

However, where Traefik's API Gateway shines, it really shines.

Automated certificate discovery: Navigating the complexities of TLS/SSL certificate management in distributed systems, particularly the issue of applying new or updated certificates without causing service downtime, is a formidable challenge. For example, traditional proxy servers often require restarting or reloading the proxy instances to apply these certificates, leading to potential availability issues and the disruption of existing sessions. By contrast, Traefik's API Gateway provides automatic discovery of new certificates, which offers a sophisticated solution, eliminating the need for manual intervention and significantly enhancing system resilience and uptime.
Plug-in Ecosystem: While still growing, Traefik’s plug-in ecosystem is tailored towards modern, dynamic environments and includes plug-ins for authentication, security enhancements, and traffic management tailored to the needs of microservices architectures.

Traefik's API Gateway’s ability to apply TLS/SSL certificate updates without requiring proxy restarts represents a significant leap forward in achieving high availability for distributed systems. This capability not only ensures continuous operation and enhanced security but also aligns with the operational demands and expectations of modern digital services. By adopting technologies and practices that support this capability, organizations can deliver more reliable, secure, and user-centric services.

Conclusion

In conclusion, the seamless integration of Vault and ACME with Traefik's API Gateway, coupled with innovative solutions like automatic certificate discovery, represents a significant advancement in managing TLS certificates for distributed systems. By addressing the challenges of certificate management and avoiding the need for proxy restarts, these technologies ensure that distributed systems can maintain high availability without compromising on security. This blend of security and availability is essential for the modern digital landscape, where the reliability and integrity of distributed systems are paramount.

]]>

Traefik 3.0 With SPIFFE, Tailscale, and HTTP/3

Emile Vauge — Thu, 23 May 2024 17:17:25 GMT

We're continuing our in-depth series on Traefik 3.0. If you missed it, be sure to read the previous articles on migrating from Traefik v2, WASM support with Coraza WAF, and Open Telemetry. Today, we'll be exploring state of the art technologies added to Traefik: SPIFFE, Tailscale, HTTP/3.

SPIFFE

Deploying dozens or hundreds of applications in production, in a distributed environment, comes with many challenges. One of them is network security, i.e. how to ensure application A is authorized to request application B? One of the historical solutions is to protect private networks with security controls like firewalls and VPNs. But many companies have adapted their security architectures to revolve around a concept known as Zero Trust networking, based on the paradigm that even private networks are untrusted. This means that applications need to authenticate themselves with other services.

In an effort to address this problem, the community created SPIFFE, the Secure Production Identity Framework For Everyone Project, which defines a framework and set of standards for identifying and securing communications between applications. The runtime counterpart, SPIRE, is a toolchain of APIs for establishing trust between applications.

Thanks to Julien Levesy, Traefik now supports SPIFFE mTLS with its backend servers! Here is a quick overview of the setup. First of all, you need to enable it in the static configuration:

## Static configuration
spiffe:
  workloadAPIAddr: localhost

The workloadAPIAddr configuration defines the address of the SPIFFE Workload API.
Now that SPIFFE is enabled globally, you need to configure your routes, at the ServersTransport level, for example, using SPIFFE trust domain (which makes Traefik allow any SpiffeID that comes from the given trust domain):

## Dynamic configuration
http:
  serversTransports:
    mytransport:
      spiffe:
        trustDomain: spiffe://trust-domain

Traefik is now able to connect to the Workload API to obtain an x509-SVID used to secure the connection with SPIFFE enabled backends 🎉. Yes, it’s as simple as that!

Tailscale

Tailscale is a zero-config VPN for building secure networks. If you are not familiar with this technology yet, I strongly recommend you to have a look, it is probably the simplest and most convenient way to set up servers in a VPN network. If you need to protect websites within a Tailscale network, Traefik can now request TLS certificates from the Tailscale API.

This new feature allows you to access HTTPS-enabled services on your tailnet behind Traefik Proxy without the hassle of managing certificates or exposing an endpoint for TLS challenges from Let’s Encrypt. Instead, Tailscale handles your certificate lifecycle, automatically renewing your Let’s Encrypt certificate, and then hands over to Traefik to handle the TLS secured requests.

Let’s see how to enable this feature. To obtain a TLS certificate from Tailscale, a Tailscale certificate resolver needs to be configured in the dynamic configuration:

certificatesResolvers:
  myresolver:
    tailscale: {}

Now all you have to do is reference myresolver from any router or entrypoint that is part of your tailnet. Here is an example using Docker labels:

## Dynamic configuration
labels:
  - traefik.http.routers.blog.rule=Host(`monitoring.yak-bebop.ts.net`) && Path(`/metrics`)
  - traefik.http.routers.blog.tls.certresolver=myresolver

Here is another example in a Kubernetes ingress resource:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: blogtls
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`monitoring.yak-bebop.ts.net`) && Path(`/metrics`)
      kind: Rule
      services:
        - name: blog
          port: 8080
  tls:
    certResolver: myresolver

Your apps are now able to access other apps securely within your tailnet, using Tailscale TLS certificates. Pretty straightforward right?

HTTP/3

HTTP is the backbone of the Internet. Since HTTP/2 was approved in 2015, significant progress has been made toward the next milestone: HTTP/3. This new version is a major shift, as it no longer relies on the traditional TCP, but instead uses UDP, leveraging QUIC—a transport protocol developed by Google in 2012. HTTP/3 offers numerous benefits, such as improved performance, faster connection establishment, and simpler encryption. Now, HTTP/3 is out of the experimental stage and fully production-ready in Traefik!

Let’s enable HTTP/3 on an entrypoint:

entryPoints:
  foo:
    http3: {}

Keep in mind that, as HTTP/3 actually uses UDP, when traefik is configured with a TCP entryPoint on port N with HTTP/3 enabled, the underlying HTTP/3 server that is started automatically listens on UDP port N too. As a consequence, it means port N cannot be used by another UDP entryPoint. Since HTTP/3 requires the use of TLS, only routers with TLS enabled will be usable with HTTP/3.

Wrapping Up

Simplicity and outstanding user experience has always been at the core of Traefik. Those new features packed in v3 follow strictly this principle. How straightforward it is to add Tailscale TLS certificates support, how elementary Traefik can be set up to retrieve SPIFFE x509-SVIDs, and how effortless you can migrate to the latest major revision of the Hypertext Transfer Protocol HTTP/3, all those added capabilities are great examples of the main core value of Traefik: keep it simple, yet state of the art.

The Traefik community has been instrumental in implementing those key features. We can't praise the outstanding work of Traefik contributors enough ❤️

Stay tuned for more deep dives on Traefik 3.0 key features!

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

Useful Links

]]>

Monitor Your Production at a Glance With Traefik 3.0 and OpenTelemetry

Nicolas Mengin — Thu, 16 May 2024 17:49:45 GMT

Today, we’re continuing our blog series on Traefik v3 (checkout our previous posts on migration path from Traefik v3 and the Wasm support with Coraza WAF plugin). This article will focus on how to leverage OpenTelemetry in Traefik v3.

Monitoring and observability have always been a priority for Traefik, consistently following industry standards such as OpenTracing integration starting with Traefik v2. That's why incorporating OpenTelemetry, the latest emerging standard, into our toolkit was an easy decision.

How Observability Ensures Infrastructure Robustness

In the microservices era, monitoring and observability—with rich metrics, tracing, and logs—are fundamental necessities in order to maintain the reliability and efficiency of our systems.

Tracing allows you to track the flow of operations within your system. Using traces, and spans, you can identify performance bottlenecks and pinpoint applications causing slowdowns, so you can optimize response times effectively.

Logs provide real-time insight into the health of your system. It enables swift error detection and intervention through alerts. By centralizing logs, you streamline the debugging process during incident resolution (say goodbye to grep).

Metrics offer a comprehensive view of your infrastructure's health. They allow you to monitor critical indicators like incoming traffic volume. For instance, Traefik enables you to dynamically scale your infrastructure based on observed traffic patterns. Additionally, metrics graphs and visualizations are helpful during incident triage to understand the causes and implement proactive measures.

OpenTelemetry: One Standard to Rule Them All

There are many tools in the market from a wide variety of vendors. Sifting through all the choices can be difficult, and picking the wrong one can be a huge pain as switching later is extremely cumbersome.

Part of what makes it difficult in deciding is because each vendor does things differently. Some scrape data, some pull it. Some are cloud-based, whereas others are on-premises. Switching between them is complex and time-consuming, if even feasible at all.

That's where OpenTelemetry comes in. It's a fusion of two older standards, OpenTracing and OpenCensus. OpenTracing focused on tracing, while OpenCensus focused on metrics and tracing. OpenTelemetry brings them together and adds logging specifications.

OpenTelemetry is an open-source project that belongs to the CNCF since 2019. Unlike with OpenTracing and OpenCensus, OpenTelemetry provides many implementations in different languages, like a Go SDK for example.

Open Telemetry makes life easier for everyone. For vendors and data providers like Traefik, having one standard obviously makes it simpler to support. For users, it ensures that the observability APIs will stay consistent regardless of the tools chosen.

OpenTelemetry in Traefik

While Traefik v2 supports many metrics and tracing vendors, it doesn’t support every solution in the market.

Traefik v2 observability solutions support

Now, with Traefik v3, we've made it possible to export metrics and traces in the OpenTelemetry format (logs support will be available as soon as the Go SDK is ready).

This allows us to ensure we support every OpenTelemetry consumer with no further implementation changes, making Traefik easy to integrate into your infrastructure.

To make sure we're fully compliant with the OpenTelemetry standard, Traefik adheres to the specified semantic conventions. These conventions tell us what data to provide based on what our software does in your setup.

Since Traefik Proxy is a reverse proxy, we've included metrics and spans information as expected in the OpenTelemetry framework.

Now, let's take a look at how easy it is to set up OpenTelemetry in Traefik.

How to install OpenTelemetry with Traefik v3?

Let's walk through the straightforward steps to install OpenTelemetry with Traefik.

OpenTelemetry Architecture

First off, it's worth noting there are two main ways to integrate Traefik (or any OpenTelemetry data provider) with an OpenTelemetry data consumer:

Vendor Direct Access: Traefik can directly connect with a consumer-like Prometheus or Jaeger to send OpenTelemetry info. It's typically used for development purposes.
OpenTelemetry Collector: This software acts as a middleman, importing data from Traefik, potentially transforming it, and then exporting it to the consumer. This approach offers centralized management of OpenTelemetry data, and scalability, and is the recommended method for handling OpenTelemetry information in large-scale applications.

Traefik v3 Configuration

Regardless of which method you choose, the cool thing is that the configuration stays the same.

First, you deploy the OpenTelemetry collector (or the solution you need) as well as Traefik on your cluster.

Then, you just set up the collector endpoints, and you're set. Whether you're adding a collector or switching up the consumer, all you have to do is tweak the address in the configuration as described below (check the documentation for more options):

## Traefik v3 ##
################

# Send metrics data to a Prometheus instance 
metrics:
  otlp:
    http:
      endpoint: http://prometheus:9090/api/v1/otlp/v1/metrics

# Send tracing data to a Jaeger instance 
tracing:
  otlp:
    http:
      endpoint: http://jaeger:4318/v1/traces

And that's it! Simple, right?

Traefik v2 Migration

While this setup works smoothly for users starting fresh with Traefik v3, what about those looking to migrate from Traefik v2?

Concerning the metrics, the good news is you can migrate seamlessly—i.e., the vendor-specific integration still works. But do not hesitate to migrate to OpenTelemetry when you’re ready for it!

However, if you've been using Traefik v2 for tracing, it's important to note that you may need to migrate to OpenTelemetry in Traefik v3. Maintaining both vendor-specific implementations and OpenTelemetry wasn't feasible, hence the decision to break backward compatibility.

Conclusion

As you have seen, integrating Traefik with OpenTelemetry couldn't be easier.

Sure, OpenTelemetry is relatively new and still evolving (with ongoing work on logs integration, expanding semantic conventions, and more), but it's promising and undoubtedly the best standard for monitoring and observability.

So, why wait? Dive in, give it a try, and don't forget to share your feedback to aid in refining our integration.

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

See you on GitHub.

]]>

Traefik 3.0: Deep Dive Into Wasm Support With Coraza WAF Plugin

Emile Vauge — Thu, 09 May 2024 14:34:27 GMT

We’re continuing our deep dive series on Traefik 3.0 which was released a week ago. Make sure to check out last week’s article on the migration path from Traefik v2. Today, we will deep dive into WebAssembly support.

Custom plugins for Traefik are one of the most requested features going back to the early days of the project, starting with this issue, from back in 2017:

The author suggested using the brand new plugin feature introduced in go 1.8. The idea was to compile go code into dynamic libraries which could be loaded and executed at runtime. It looked very promising and we immediately started working on this. Sadly, some severe limitations were found and this pursuit was put on hold. A second attempt was made a few months later, but was eventually abandoned due to the incomplete plugin implementation provided by the go team (only on Linux with CGO_ENABLED).

After many discussions, we ended up with a different solution. If the go language ecosystem wouldn’t provide any solid solutions for building plugins, let’s implement our own (crazy, I know) fully compliant Go interpreter. In 2019, Yaegi—Yet Another Elegant Go Interpreter—was released. The project quickly got a lot of traction being the best Go interpreter, and middleware plugins became a reality within Traefik soon after. Yaegi enabled many to develop middlewares or providers for their context. To this day, we can count more than a hundred middleware plugins made available through the catalog, plus many more that are kept private.

However, Yaegi only provides the option to build plugins using go, and this can be a bit restrictive for some people. This is the reason why some community members started to work on the brand new plugin engine for Traefik in 2023: Web Assembly, abbreviated WASM.

WASM Overview

WebAssembly is a low-level assembly-like language with a compact binary format that runs with near-native performance. It provides languages such as C/C++, Go or Rust (and many more) with a compilation target so that they can run on the web, and in other runtimes thanks to WASI, the WebAssembly System Interface.

Announced in 2015, WASM evolved into an open standard and got support in all major browsers in 2017.

In a nutshell, WASM allows you to compile almost any language in a portable binary format that can run on any platform, with native performance.

It looks like we found the perfect plugin engine for Traefik 🙂

Traefik + WASM

The idea to bring WASM support to Traefik was appealing. However, many challenges had to be tackled. The first one was to write an interface between Traefik, the host, and WASM plugins, the guests. This is called an Application Binary Interface (ABI) and in the case of Traefik middleware plugins, we had to provide an interface to the net/http HTTP server library. Writing and maintaining a fully compliant ABI for the net/http library is not an easy task but luckily, talented developers had already released different options, two of which stood out: proxy-wasm and http-wasm. After many discussions with community members, the clear winner was http-wasm, thanks to its much simpler and straightforward integration within a go codebase.

Then everything accelerated. A community contributor—Jesse Haka—opened a pull request, and a month later, after careful reviews from maintainers and external contributors like José Carlos Chávez, Traefik had support for WASM middleware plugins!

Let’s see how to write a WASM plugin for Traefik that customizes the routing pipeline that handles requests and responses. The Traefik routing pipeline basically splits into 3 main components: entrypoints, routers and services. Entrypoints define the network entry points into Traefik. Routers are in charge of connecting incoming requests to the services that can handle them. In the process, routers may use a chain of middleware to do some pre- and/or post-processing of the request. Our goal here is to write a custom middleware that will be added to the chain in a Traefik router. To simplify, we will use go as the plugin langage, but we could also use C or Rust, as soon as the code implements the http-wasm interface.

Ultimately, the only thing you have to do is implement 1 function: handleRequest which handles the pre-processing or handleResponse, the post-processing. You can get a simple example of a WASM plugin in the repository. Here is boilerplate:

package main

import (
	"encoding/json"
	"fmt"
	"os"

	"github.com/http-wasm/http-wasm-guest-tinygo/handler"
	"github.com/http-wasm/http-wasm-guest-tinygo/handler/api"
)

// Config the plugin configuration.
type Config struct {
	Headers map[string]string `json:"headers,omitempty"`
}

func main() {
	var config Config
	err := json.Unmarshal(handler.Host.GetConfig(), &config)
	if err != nil {
		handler.Host.Log(api.LogLevelError, fmt.Sprintf("Could not load config %v", err))
		os.Exit(1)
	}

	mw, err := New(config)
	if err != nil {
		handler.Host.Log(api.LogLevelError, fmt.Sprintf("Could not load config %v", err))
		os.Exit(1)
	}
	handler.HandleRequestFn = mw.handleRequest
}

// Demo a Demo plugin.
type Demo struct{}

// New created a new Demo plugin.
func New(config Config) (*Demo, error) {
	return &Demo{}, nil
}

func (d Demo) handleRequest(req api.Request, resp api.Response) (next bool, reqCtx uint32) {
	return true, 0
}

Then load your compiled plugin into the static configuration:

# Static configuration
experimental:
  localPlugins:
    example:
      moduleName: github.com/traefik/plugindemowasm

```

And finally, customize a router with your new plugin:

# Dynamic configuration
http:
  routers:
    my-router:
      rule: host(`demo.localhost`)
      service: service-foo
      entryPoints:
        - web
      middlewares:
        - my-plugin

  services:
   service-foo:
      loadBalancer:
        servers:
          - url: http://127.0.0.1:5000
  
  middlewares:
    my-plugin:
      plugin:
        example:
          headers:
            Foo: Bar
```

As you can see, this is pretty straightforward to customize the routing pipeline in Traefik thanks to WASM plugins. Let’s tackle a production use case. How about embedding a Web Application Firewall into the middleware chain?

Traefik + Coraza

Traefik is a critical piece of many companies’ infrastructure and is playing a great role in securing connectivity with applications and APIs. It's impossible for any single security measure to cover every potential attack angle. Security operates in layers, each with its own focus, sometimes with areas of overlap. A global approach, acknowledging the diverse threat landscape, is the way to go. One of the main links in the security chain are Web Application Firewalls. They look for malicious and unwanted content within incoming requests and mitigate those potential threats. Having a WAF integrated to Traefik as a plugin is clearly a game changer and adds a first line protection layer to your infrastructure.

Coraza is an open source, go based, high performance, Web Application Firewall. It’s an OWASP project, understands Modsecurity’s seclang language, and it’s able to enforce OWASP core rule sets.

Of course, when Coraza’s leaders proposed to provide an integration of Coraza in Traefik within a WASM plugin, we were all pretty excited. José Carlos Chávez quickly came up with a working solution now available on the Traefik Plugins Catalog!

First of all, let’s load the Coraza plugin into the static configuration:

# This config was generated by "mage updateVersion". DO NOT EDIT.

entryPoints:
  web:
    address: :80

providers:
  file:
    filename: /etc/traefik/config-dynamic.yaml

experimental:
  plugins:
    coraza:
      moduleName: github.com/jcchavezs/coraza-http-wasm-traefik
      version: v0.2.1

Let’s update the middleware section for a router in the dynamic configuration:

http:
# ...
  middlewares:
    waf:
      plugin:
        coraza:
          directives:
            - SecRuleEngine On
            - SecDebugLog /dev/stdout
            - SecDebugLogLevel 9
            - SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,log,deny,status:403"
```

Then curl -I 'http://localhost:8080/admin' will return a 403 error as specified by the configuration rule SecRule REQUEST_URI "@streq /admin.
curl -I 'http://localhost:8080/anything' will return a 200 as there is no matching rule.

A more advanced example can be found here, where we attempt to use the log4shell attack on a vulnerable application. Log4shell is a zero-day vulnerability in Log4j involving arbitrary code execution. This attack can be mitigated by simply enabling CRS rule 932130 to look into REQUEST_HEADERS:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: config
  namespace: traefik
data:
  config.yaml: |
    http:
      middlewares:
        waf:
          plugin:
            coraza:
              directives:
                - Include @recommended-conf
                - Include @crs-setup-conf
                - Include @owasp_crs/*.conf
                - SecRuleUpdateTargetById 932130 "REQUEST_HEADERS"
                - SecRuleEngine On
EOF

As you can see, the WASM plugin feature in Traefik provides a state-of-the-art extension technology and covers the most advanced use cases.

Takeaways

Traefik already had an extension engine based on the Yaegi Go interpreter. It is extremely easy to set up and provides a powerful foundation for customizing Traefik. However, it requires writing plugins in Go, and some people could find this too restrictive.

WebAssembly has gained huge popularity lately and provides exactly what is needed to build a perfect plugin engine: portable binaries, open standard, fast, multiple languages support, and multiple host environments. Traefik 3.0 adds full support of WASM plugins for middlewares and allows to run binary code compiled from many different languages like C, Rust, or JavaScript.

Finally, to show the full potential of this new plugin engine, we have integrated the powerful Web Application Firewall Coraza, as a WASM plugin. This will bring new critical capabilities to Traefik, adding an additional layer of security to your environment.

The Traefik community has played a central role in brainstorming and implementing those major features. We can’t say enough good things about the exceptional work made by Traefik contributors ❤️

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

Stay tuned for more deep dives on Traefik 3.0 key features!

]]>

Traefik 3.0 GA Has Landed: Here's How to Migrate

Emile Vauge — Tue, 30 Apr 2024 12:43:09 GMT

This year will mark the 9th anniversary of Traefik. Back in 2015, while I was writing the first lines of code for a cloud native reverse proxy, I couldn’t imagine how big this would go. From a side project, Traefik became one of the most deployed modern gateways on earth, with more than 3 billion downloads and 750+ contributors. It is ranked in the Top 15 on DockerHub and has 47,000 GitHub stars. Insane.

Traefik 1.0 was released in 2016. Three years later, Traefik 2.0 was born. Today, after 5 years of development, the wait is finally over! We are proud to announce that Traefik 3.0 is generally available 🎉

This major release is a huge step forward in the cloud native world, adding support for the latest technologies like WASM, OpenTelemetry, Kubernetes Gateway API, and SPIFFE.

After many years of effort, the changelog is pretty wild, with more than 200 pull requests merged, each coming with new features. The list of new possibilities is so big that we want to provide a series of blog posts, each deep diving into one feature.

Today’s article will focus on the beginning, the elephant in the room 🥁🥁🥁: the migration.

Migration Under Control

A new major version is always something eagerly awaited: new design, new features, better user experience … But the counterpart is usually the migration aspect. A major version often means breaking changes, but that shouldn’t imply a painful migration experience, and with Traefik v3 it doesn’t.

With Traefik v3, we are introducing a streamlined transition process from v2. As a reminder, Traefik has 2 kinds of configurations:

the static configuration which is loaded when Traefik starts up and manages global options
the dynamic configuration can be updated while Traefik is running and contains all the routing information

Minimal breaking changes have been made on specific options on the static configuration, and we are ensuring backward compatibility with v2 syntax on the dynamic configuration. This will offer a gradual path for adopting the v3 syntax, allowing users to progressively migrate their Kubernetes ingress resources, Docker labels, etc. to the new format.

Let’s deep dive into the migration path with a typical use case: you have to migrate 100 Traefik instances from v2.11 to v3.0 in Kubernetes.

Step 1: Prepare and Test

The first thing to do is to identify how your static configuration is impacted by the changes made in v3. The breaking changes are minimal and target very specific options, in 90% of use cases, this process should take a couple minutes only. As an example, Docker and Swarm are now 2 providers, HTTP/3 is no longer an experimental option, Rancher v1 has been dropped as the project is no longer being actively maintained, etc. Please refer to the static configuration section of the migration documentation, as well as the operations section, to get the full list and prepare your new v3 static configuration.

Add the following snippet to your static configuration, this will use by default the v2 syntax for your routing rules:

# static configuration
core:
  defaultRuleSyntax: v2

When you are ready to test it, start Traefik v3 with this new configuration. If you don’t get any error log, you are good to go! Otherwise, no problem, the remaining migration options are highlighted in the logs, you can simply apply them.

Once your Traefik test instances are starting and routing to your applications, you can move to the next step.

Step 2: Rolling Update

Now that you have tested your updated static configuration, it is time to progressively migrate your production instances to Traefik v3. Use the Kubernetes rolling update mechanism to incrementally replace the current Pods with new ones.

⚠️ Before triggering any change on your production, ensure you to have a working real time monitoring solution on your ingress traffic that will allow you to instantly detect any issue. Traefik provides support for many observability solutions.

While the rolling update is in progress, constantly monitor your ingress traffic looking for unexpected (and unusual) errors, and be prepared to rollback to a working state. Then, leverage debug and access logs provided by Traefik to understand and fix the issue before upgrading again. In case you're not sure, jump onto the community forum.

Once every pod is updated, congratulations, you are now on Traefik v3!

You can move to the last step.

Step 3: Progressive Ingresses Update

Now that you run Traefik v3, start migrating your ingress resources to the new format.

📍 Keep in mind that this step can be done later, as Traefik v3 is compatible with the v2 format for dynamic configuration. But of course, you can start using new features right away in new ingresses, and migrate older ingresses later.

The dynamic configuration in v3 has undergone a few changes. For example, the Router Rule Matchers have an updated syntax, the Kubernetes Ingress API Group has been changed, and the TCP LoadBalancer terminationDelay option has been removed. The full list can be found in the dynamic configuration section of the migration documentation.

Progressively, switch each router to the v3 syntax, test & update each ingress resource and check that the ingress traffic is not impacted. Once a v3 ingress resource migration is validated, you can delete the v2 ingress resource and deploy the v3 version. Repeat those steps for every ingress resource.

At the end of the process, you can safely remove the snippet added at step 1:

# static configuration
core:
  defaultRuleSyntax: v2

Et voilà!

You are now fully migrated to Traefik v3 🎉! And you did this progressively, keeping control during the whole process, with the option to rollback any change at any time.

This example with Kubernetes can be transposed to any orchestrator or environment, the process stays the same.

In the coming weeks, we will continue this series, and release deep dive articles on WASM support (and Coraza Web Application Firewall plugin), OpenTelemetry, SPIFFE/Tailscale/HTTP/3, and Kubernetes Gateway API.

Ultimately, I'd like to express immense gratitude to all contributors. Your support is incredibly valuable.

To learn more about v3, watch the recording of our recent Traefik v3 Online Meetup.

See you on GitHub!

Useful Links

Traefik 3.0 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Level Up API Operations: Granular API Access Control Meets GitOps

Immánuel Fodor — Thu, 18 Apr 2024 19:47:17 GMT

We're thrilled to announce the release of a powerful new security feature for managing APIs with Traefik Hub: granular API access control with operation filters. This feature empowers you to control who can access your APIs and what they can do with them with unprecedented precision.

Modern applications rely heavily on APIs, as evidenced by the Cloudflare 2024 API Security report, which found that APIs comprise over half (57%) of internet traffic. As the building blocks of digital infrastructure, APIs play a pivotal role in organizational success. However, securing these critical resources can be challenging. Traditionally, access control options have been limited to all-or-nothing approaches, leaving organizations vulnerable to security risks and compliance gaps.

With Traefik Hub's new granular API access control, you can finally move beyond this restrictive approach. You now have the fine-grained control you need to secure your APIs at the HTTP method and URI path level, ensuring only authorized users can access specific functionalities within your APIs.

The Growing Challenge of Securing APIs in Modern Applications

The expanding landscape of APIs presents a significant challenge for organizations seeking to maintain robust security measures. According to Cloudflare, nearly 60% of organizations grant "write" access to at least half of their APIs, and the prevalence of shadow and zombie APIs further complicates the picture:

Shadow APIs, essentially unmanaged and unsecured application programming interfaces, pose a severe threat due to their inherent lack of oversight and potential access to sensitive data. Cloudflare’s machine learning algorithms found ~33% more API endpoints vs. what customers self-reported. These rogue APIs can lead to data breaches and expose critical information when compromised.
Zombie APIs were once active but have been deprecated or abandoned. They may already be managed but not used actively. Likely, these APIs are not well secured or left unsecured, so they impose an ever-growing risk of becoming compromised or being used to carry out malicious activities.

Additionally, the rise of generative AI introduces new security concerns. While AI holds great promise, its integration also brings potential vulnerabilities, as both AI models and AI-generated code can be susceptible to attack. Forrester predicts that by 2024, a lack of proper security measures could lead to several high-profile data breaches originating from AI-generated code. However, they also believe that up to 90% of data breaches will include a human element.

Furthermore, APIs used for business to business (B2B) communication, while often perceived as more trustworthy due to their authenticated nature, can make organizations fall into a false sense of security. Because these APIs typically serve a smaller, known set of partners, security teams might underestimate the potential for abuse. However, malicious actors can exploit APIs in unforeseen ways, accessing confidential data and restricted functionalities or compromising previously trusted partners, infiltrating their systems to search for more vulnerable APIs.

Finally, the OWASP Top 10 API Security Risks reveal a worrying trend, with over half of the listed points focusing on API consumption and authentication flaws. These vulnerabilities—such as broken authentication, unrestricted resource consumption, and improper access controls—can be exploited by attackers to gain unauthorized access to sensitive data and functionalities.

Granular API Access Control to the Rescue

Traefik Hub now empowers you to move beyond the limitations of traditional, all-or-nothing API access control. Our new granular API access control feature allows you to precisely define who can access your APIs and what actions they are authorized to perform on a per-endpoint basis. This fine-grained control, achieved through HTTP method and URI path filtering, ensures that only authorized users can access specific functionalities within your APIs, significantly enhancing your overall security posture.

We've taken a modern and efficient approach, leveraging the power of GitOps and Kubernetes Custom Resource Definitions (CRDs). GitOps, a popular methodology for infrastructure automation, allows you to manage your access control rules in Git repositories, ensuring version control, collaboration, and easy rollbacks. CRDs, on the other hand, provide a Kubernetes-native way to define custom resources specific to your needs, such as API access control rules. This combination offers several benefits:

Easy to read and understand: CRDs are written in a human-readable format, making them easier to understand and manage than complex configuration files.
Validation and consistency: CRD validity is enforced through specific schema validation, ensuring the consistency and integrity of your access control rules.
Version control and auditability: Git provides built-in version control capabilities, allowing you to track changes to your access control rules and easily roll back to previous versions if needed. Additionally, Git enables comprehensive auditability of access control policy changes.
Automated policy application: By leveraging Kubernetes-native labels and selectors, Traefik Hub can automatically apply your API access control policies to newly published APIs based on defined criteria, streamlining the security configuration process.

Key Features of Traefik Hub's Uniquely Flexible API Security

The flexibility of Traefik Hub's granular API access control sets its apart from similar solutions. Here are the key features and functionalities that define our unique approach:

Support for all standard HTTP methods: Our solution empowers you to define access control rules for all commonly used HTTP methods, including GET, POST, PUT, DELETE, PATCH, and more. This fine-grained control allows you to grant or deny access based on the specific actions users attempt to perform on your APIs.
Versatile URI path definition: You can define URI paths with various options, including:
- Exact paths: Grant or deny access to specific API endpoints
- Path prefixes: Control access to a group of related endpoints sharing a common prefix
- Regular expressions: Define complex patterns for matching multiple URI paths under a single rule, simplifying access control management.
Powerful rule combinations: Go beyond simple allow/deny rules and leverage AND/OR logic to create sophisticated access control scenarios. This allows you to combine path and method restrictions to define precisely who can access what and under what conditions.
Simplified rule management with operation sets: Introduce reusable sets of path and method combinations, called "operation sets," to streamline rule creation and management. This approach promotes code organization and reduces redundancy, especially when defining access control rules for multiple APIs with similar functionalities.
Granular control at both API and API version levels: Enforce access control not only at the API level but also at the individual API version level. This enables you to adapt access policies based on specific functionalities or security requirements introduced in different API versions.

Business Benefits: Efficiently Elevate Your API Security

By utilizing Traefik Hub’s novel operation filtering, you can unlock a plethora of benefits for your organization:

Craft highly customized rules tailored to your specific API security needs with streamlined access control policy management leveraging GitOps and CRDs for efficient access control rule creation, version control, and auditability.
Mitigate security risks and limit data exposure by granting access only to authorized users and actions on your APIs. Implement zero trust principles by restricting API access based on specific needs, minimizing the potential for data breaches and unauthorized exposure.
Improve developer experience by enabling them to easily understand and manage access control policies, thanks to clear and concise rules.
Facilitate adherence to industry regulations and compliance requirements. Many industries have regulations governing data access and security. Granular access control simplifies compliance by enabling you to demonstrate control over who can access your APIs and the specific data they can access.
Improve governance by enforcing consistent access control policies across your API landscape. Adhering to the same security standards can reduce confusion and potential vulnerabilities.
Keep your agility! Quickly define and modify access rules, and reuse existing access policies across APIs and API versions.

Traefik Hub supports the growing complexity of your API and access control needs. As your API landscape evolves, you can manage the increasing complexity of access requirements without sacrificing security or efficiency.

In the next section, we'll showcase example scenarios demonstrating how granular API access control can be applied to address specific security challenges.

Example: Protecting Sensitive Data in a Healthcare API

Imagine a healthcare provider utilizing an API to manage patient data. This API likely contains sensitive information, such as medical records, prescriptions, and insurance details. While specific doctors and nurses require access to this data to provide care, it's crucial to restrict access to unauthorized individuals.

Traditionally, without granular access control, the provider might have only two options:

Grant all-or-nothing access: This approach exposes all data to all authorized users, posing significant security risks if unauthorized access occurs
Implementing complex custom business logic: This can be time-consuming to develop and maintain, especially as the API and its users evolve.

With Traefik Hub's granular API access control, the provider can achieve precise control by defining fine-grained API access rules:

Limit access to specific patient data API endpoints with URI path rules based on user roles. For example, doctors can access detailed medical records, while nurses can only view basic information.
Restrict access to specific HTTP methods. For example, allow only GET requests for read-only access and disallow modification methods like PUT or DELETE for nurses.
Mix and match these to create more advanced rules.

Let’s see the following configuration example we want to express for an Admin API that is versioned with two versions and an unversioned Patient API.

Rule Placement	Group	URI Path	HTTP Method
API Version	admin	/v1/admin (prefix)	*
API Version	admin-beta	/v1/admin (prefix) /v2/admin (prefix)	*
API	nurse	/patients (strict) /patients/[0-9]+$ (regex)	GET GET
API	doctor	/patients (prefix) /.* (regex)	GET, POST, PUT, DELETE GET

The Admin API has two versions defined using URL path-based versioning:
- The regular admin group can access everything on the current 1st version.
- The beta tester admin group can access everything on both the current and the upcoming v2 version.
The patient API exposes a /patients endpoint with a sub-path /patients/:id to access basic info of specific patients and a /patients/:id/details endpoint to get all the details. All of these support modifying and deleting the available data and creating new records.
- Nurses can only view basic information. They can list patients with /patients using strict URI path matching to limit access only to the list endpoint and can access basic info with /patients/:id.
- However, doctors can access full medical records, meaning everything under /patients like /patients/:id and /patients/:id/details using URI path prefix matching.
On the other hand, besides the Patient API, doctors can access all Admin API versions with read-only permissions to retrieve basic information about the hospital administration.

Let’s see the examples expressed as Traefik Hub CRDs. The relevant additions of granular API access control are highlighted.


---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: admin-api
  namespace: apps
  labels:
    area: health
spec:
  pathPrefix: /admin
  currentVersion: admin-api-v1
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: admin-api-v1
  namespace: apps
  labels:
    area: health
spec:
  apiName: admin-api
  release: v1.0.0
  title: "Title of Admin API v1"
  routes:
    - pathPrefix: /v1
  stripPathPrefix: true
  service:
    name: admin-service-v1
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: admin-ops-v1
          matchers:
            - path: /admin
        - name: read-all
          matchers:
            - methods:
                - GET
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: admin-api-v2
  namespace: apps
  labels:
    area: health
spec:
  apiName: admin-api
  release: v2.0.0
  title: "Title of the beta Admin API v2"
  routes:
    - pathPrefix: /v2
  stripPathPrefix: true
  service:
    name: admin-service-v2
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: admin-ops-v2
          matchers:
            - path: /admin
        - name: read-all
          matchers:
            - methods:
                - GET
---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: patient-api
  namespace: apps
  labels:
    area: health
spec:
  pathPrefix: /patients
  service:
    name: patient-service
    port:
      number: 8080
    openApiSpec:
      path: /openapi.json
      operationSets:                  # A new block from here
        - name: nurse-ops
          matchers:
            - path: /patients
              methods:
                - GET
            - pathRegex: "/patients/[0-9]+$"
              methods:
                - GET
        - name: doctor-ops
          matchers:
            - pathPrefix: /patients
              methods:
                - GET
                - POST
                - PUT
                - DELETE
        - name: read-all
          matchers:
            - methods:
                - GET

First, we define the Admin API and its two versions with operation sets specific to that version. Note that you don’t need to add the version matcher to the operation set, it will automatically apply—i.e., we only define /admin, and depending on the version, it will apply to /v1/admin and /v2/admin. There is also an operation set allowing read access on both versions, using the same operation set name to be reusable across even multiple APIs when we give access to users. On the Patient API, we define the nurse and doctor operation sets with explicit rules as to who can access what. The generic read-all set is also present here, extending to multiple APIs.

Then we define our API Access policies to match the user groups with the APIs, API Versions, and filter for the operation sets.


---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: admin-access
  labels:
    area: health
spec:
  groups:
 - admin                             # Note the admin group
  apis:
    - name: admin-api                # Has Access to the Admin API
      namespace: apps
  operationFilter:
    include:
      - admin-ops-v1                 # With the v1 operation set
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: admin-beta-access
  labels:
    area: health
spec:
  groups:
    - admin-beta                     # Note the beta tester admin group
  apis:
    - name: admin-api                # Has access to the Admin API
      namespace: apps
  operationFilter:
    include:
      - admin-ops-v1                 # With the v1 operation set as regular admins
      - admin-ops-v2                 # And the new beta features of v2
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: nurse-access
  labels:
    area: health
spec:
  groups:
    - nurse                          # Note the nurse group
  apis:
    - name: patient-api              # Has access to the Patient API
      namespace: apps
  operationFilter:
    include:
      - nurse-ops                    # With the restricted operations allowed to
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIAccess
metadata:
  name: doctor-access
  labels:
    area: health
spec:
  groups:
    - doctor                         # Note the doctor group
  apiSelector:
    matchLabels:
      area: health                   # Can access APIs (current+future) with this label
  operationFilter:
    include:
      - doctor-ops                   # But restricted to operations defined for doctors
      - read-all                     # and read-all operations over multiple APIs

What stands out here is that the beta admin group can access two operation sets across the two API versions. In contrast, the regular admin group will only be granted access to the first version of the API. Nurses can only access the nurse operations, that’s simple. However, the policy for doctors is defined using a Kubernetes-native label selector to select all APIs that match the health label. This way, the filter for the doctor-ops operation will apply to the Patient API, and the read-all operation filter will be used throughout the two API versions of the Admin API. Since the read-all operation is also present on the Patient API, but the doctor-ops provide extended access to doctors, the broader rule overrides it.

When the Admin API v2 becomes stable, we could deprecate the v1 version to have only read-only access by modifying the related operation set and the regular admins’ access policy. Showing only the changes below:


# Admin API v1
operationSets:
  - name: admin-ops-v1
    matchers:
      - path: /admin
        Methods:                    # New line
          - GET                     # New line

# Regular Admin Access
operationFilter:
  include:
    - admin-ops-v1
    - admin-ops-v2                  # New line

In case a new API is added later with the same health label, that API will not grant automatic access to doctors to prevent data leaks, even though the label selector applies. Doctors can only gain access if the doctor-ops is also defined on that new API or if it defines some other operations that will be explicitly added to the doctors’ API access policy.

We hope these examples shed light on the flexibility of Traefik Hub’s new granular API access control solution with operation sets and filters.

Conclusion: Take Control of Your API Security

In today's API-driven landscape, robust security measures are essential. Traditional, all-or-nothing access control approaches leave organizations vulnerable to data breaches, compliance gaps, and operational inefficiencies. Traefik Hub's granular API access control empowers you to address these challenges head-on. Gain the control you need to secure your APIs, optimize developer workflows, and build a foundation for secure and thriving digital interactions.

Ready to learn more? Dive deeper into our documentation of operation set definitions for API & API Version and use them in operation filters in the API Access CRD.

]]>

How Modern API Gateways Make DevOps Engineers More Efficient

Immánuel Fodor — Wed, 03 Apr 2024 19:06:40 GMT

Imagine, for a moment, that you are in charge of a large chain of home improvement stores. You have hundreds of stores and tens of thousands of employees spread across 12 countries. You receive tens of thousands of orders every single day spread over 100,000 requests every single second. You’ve got four different IT platforms, all on Kubernetes. Latency is unstable. Debugging is time-consuming. Developers are frustrated.

And then the pandemic happens, sending the whole world online.

That’s the situation home improvement and gardening retailer Leroy Merlin found themselves in in 2020. Like so many companies, they knew they needed to face that “digital transformation” buzzword before they lost their developers to The Great Resignation. And in this case, that meant integrating an API gateway into their processes and architecture.

It’s a problem you may also be facing.

Why Consider an API Gateway Model?

An API gateway serves as an intermediary between clients and backend services, streamlining request routing, enforcing security measures, managing APIs and microservices, and optimizing the communication flow within complex software architectures. That’s a mouthful, but the important thing is that it provides a centralized and unified way for developers to interact with the backend services that power your applications and infrastructure.

An API gateway also provides several other features that an enterprise with significant infrastructure needs.

Traffic Management

Whether you have just a few lightly trafficked services or you’re dealing with 100K requests per second, it’s obviously crucial to make sure those requests are routed properly, but that involves much more than simply sending them to the right service.

The API gateway has to provide efficient routing, of course, but it also has to provide load balancing; it would be simple for a single service to not just bog down and stop responding, but also bring the entire system to a standstill, particularly during traffic spikes. The API gateway has to handle these spikes gracefully.

To do all this, the API gateway has to optimize service communication, making sure that data and requests take the most efficient paths through the system.

Authentication and Authorization

Just because an API gateway makes it possible to more easily access services doesn't mean that anyone and everyone should be able to access every service within the system. An API gateway needs to ensure that non-public resources can only be accessed by those to whom they should be available, whether that’s a user or another service.

To make this happen, you need to be able to verify digital identity through the most appropriate means. In some cases that means issuing, managing, and verifying tokens. In others, it may mean implementing multi-factor authentication (MFA). The API gateway also has to ensure that that particular identity actually has the privileges it’s trying to exercise so that sensitive areas are reserved for those with the right privileges, keeping customer data and critical services safe and sound. In other words, you need to make sure you’re wrapping API endpoints in layers of security that deter would-be digital trespassers.

Caching and Rate Limiting

Speed is the lifeblood of the digital experience. For example, Leroy Merlin’s site required multiple service calls, so to ensure it felt responsive, each call could take no more than 20ms. The API gateway, with its caching capabilities, stores copies of frequently requested data so that future requests can be fulfilled faster. It can also use smart caching strategies, anticipating the customer’s next move and ensuring the most relevant and up-to-date information is always at the ready, minimizing wait times.

Sometimes, however, the API gateway has to limit requests from a specific user or service in order to prevent overuse and preserve resources for the rest of the system.

Log Management

In modern applications and infrastructure, every action, every request, and every error leaves a trace in the form of logs. The API gateway acts as the central library for these vast volumes of data, collecting and organizing them so that they can be easily accessed, studied, and acted upon.

But a library is only as good as its librarians. By integrating with external monitoring—for example, Leroy Merlin needed a connector to send their logs to Datadog—the API gateway ensures that the logs are not just collected but analyzed, turning raw data into actionable insights that can guide the evolution of digital services.

Of course, this is probably most useful when things go awry, as they inevitably do. In this case, the collected logs become the map that leads developers and operators back to the source of the problem.

Finally, in the realm of digital commerce, compliance with laws and regulations is not just a necessity but a cornerstone of trust. Through meticulous log management, the API Gateway helps safeguard customer data and maintain the integrity of digital operations.

Certificate Management

Of course, it’s impossible to talk about digital security without talking about SSL/TLS for end-to-end encryption. The API gateway not only handles encryption, but also tackles certificate renewal, not just taking care of the actual renewal, but ensuring that all servers and services in the system received the new certificates—preferably without having to perform a full reload, resulting in the loss of current sessions.

For years functions like this have been handled by the Electronic Frontier Foundation’s Certbot, but more modern applications have been using open source projects like Traefik.

All of these capabilities can be helpful for any business, Leroy Merlin included. But API gateways that are aligned to modern operations, namely GitOps, are more than just helpful. They're transformative.

Why Choose a GitOps-Driven API Gateway

While the capabilities of a traditional API gateways bring a number of benefits, many were created before the cloud native and operations revolutions. To maximize the benefits of switching to an API gateway model, however, pairing a GitOps strategy with a GitOps-driven API gateway is a must.

GitOps is a set of practices that uses Git as the single source of truth for declarative infrastructure and applications. Essentially, it applies the principles of version control, collaboration, compliance, and CI/CD (Continuous Integration/Continuous Deployment) automation to infrastructure and application management.

Everything in the system—infrastructure, networking, applications—is described in a declarative manner. This means you specify the desired state of your system, and automated processes work to maintain that state. When you want to make a change, you change that stored configuration and commit the change to the repository, and the system makes the change when it’s merged. The process enables code review, automated testing, and collaboration before changes are merged and applied, which improves the stability and security of the system.

This way, the actual state of the system matches the declared state in the repository. If discrepancies arise, the system can automatically correct itself, or alert an operator to intervene.

This process provides a number of benefits for developers.

Automation

Because changes are applied automatically, developers can roll out a new configuration by simply making a single change, and as soon as it’s approved and applied, it can be deployed everywhere it needs to go. This way developers can make changes quickly without risking the inevitable human error that comes with manual changes.

That enables developers to work more quickly and confidently, knowing their changes can be tested and reviewed before they roll out, and that every change is tracked, versioned, and most of all, reversible.

Easy Roll Backs

If a problem does develop, recovery from failures becomes a streamlined process with GitOps. Instead of frantic fixes, teams can methodically revert to a stable state, analyze what went wrong, and apply lessons learned. All of this lets developers keep downtime to a bare minimum.

Knowing they can easily undo changes encourages experimentation and innovation, so developers can try new features and approaches with the safety net of easy rollbacks, fostering a culture of innovation.

Increased Reliability & Repeatability

Consistency is key to reliability. GitOps ensures that every environment, from development to production, is a mirror image, eliminating the "it works on my machine" syndrome and bolstering developer confidence in every release.

It also gives developers the ability to reproduce environments and deployments at the push of a button, which means the power to quickly spin up new instances, test environments, or recover from (the inevitable) disasters. Cloning existing environments is as simple as cloning a repository and applying the configuration elsewhere.

What’s more, because all of these environments are based on the same images, developers get the standardization that is the foundation of efficiency. GitOps enforces a uniform setup across all infrastructure, simplifying management, reducing complexity, and ensuring that best practices are universally followed.

Better Collaboration

GitOps provides a common framework for operations, establishing a unified language and approach that streamlines workflows and harmonizes efforts across the developer landscape.

It creates a collaborative environment where developers, operations, and even security teams work from the same playbook, enhancing communication and reducing friction. Every change is tracked, every decision logged, making sure all team members are on the same page, fostering trust and helping create smoother operations.

With infrastructure and configuration codified, contributing changes becomes as simple as contributing code, lowering barriers for team members to suggest improvements or fixes, and the GitOps workflow encourages dialogue across departments, making sure that everyone's voice is heard and that decisions are made with a holistic view of the organization's objectives.

Putting it all together

By opting for a modern, GitOps-based API gateway (in this case, Traefik’s API Gateway), Leroy Merlin solved a number of issues. This approach creates significant benefits that improve workflows and productivity, streamline deployment processes, and make them more automated, consistent, and error-free. All this drastically reduces the time and effort Leroy Merlin and companies like them need for manual configurations and deployments.

The clear, version-controlled history of infrastructure changes improves collaboration between team members, allowing for easier code reviews, quicker onboarding of new developers, and a better development environment. Additionally, the ability to quickly rollback to previous states in case of errors or unforeseen issues significantly lowers the risk associated with new releases, fostering an environment where innovation and experimentation are encouraged without the fear of catastrophic failures.

All of this can not only boost the morale and efficiency of your development team, but also accelerate the pace of digital innovation, just as it did for Leroy Merlin, ensuring they could rapidly respond to market demands and maintain a competitive edge in the retail space.

]]>

We Built The Most Modern API Management from Scratch. Here’s How.

Emile Vauge — Thu, 21 Mar 2024 14:40:29 GMT

Complaining about things that don't work well is easy. What's hard is coming up with a better solution.

That statement is true of many things in life, including one we know well at Traefik: API management. After all, there is plenty to criticize about traditional API management — such as poor integration with cloud-native technologies, inefficient incident management, challenging release management and poor governance strategies.

But what does a better approach to API management look like? How, specifically, do you implement a tool that enables cloud native-friendly, scalable, collaborative API management? Those are not simple questions to answer.

But these are the questions that we had to solve in order to build Traefik Hub. We had to make careful design choices to determine how API management workflows based on our tool should work. We also had to analyze a variety of technical considerations, such as how best to implement tunneling and a high-availability data plane.

We're proud of the solutions we arrived at. We're so proud that we'd like to share them — and the rationale behind them — in this article, with the goal of offering our community a peek into the design process and thinking that led to Traefik Hub.

Why We Decided to Reboot API Management

Before delving into the choices we made when designing Traefik Hub, let's talk about why we decided to build the solution in the first place.

The reason was simple: we believe traditional API management is badly broken. Most conventional API management tools were designed and built before cloud-native architectures became widespread, and before the widespread adoption of collaboration-centric, DevOps-based cultures. As a result, they lack the scalability, modularity and efficiency to drive optimal results in today's scale-out, distributed world.

We conceived of Traefik Hub as a way to reboot API management, so to speak. By building a new solution from the ground up — instead of trying to extend or overhaul an existing API management tool — we gave ourselves total freedom to reinvision what modern API management should look like.

Our Design Principles

Starting from scratch meant we had virtually unlimited leeway to design Traefik Hub in whichever ways we thought best. But it also meant we had to make a ton of design choices, since there was no pre-existing foundation to guide us or get us started.

We settled on an approach oriented around the following design principles.

The Unix Philosophy

For starters, we wanted to adhere to the so-called Unix philosophy, which emphasizes designing a tool that does one thing and does it well. That's why Traefik Hub doesn't try to be a Swiss army knife of API-related functionality. Instead, it focuses on API "day 2" operations — API deployment, runtime management, observability, and security.

Other tools are available to assist with processes like API design and documentation, and we don't think it's healthy to try to pack too many features into one tool.

Declarative Management

In the cloud-native era, engineers have become accustomed to declarative management. That means writing code that describes how something should operate, then applying the desired configuration automatically.

Traefik Hub does this by enabling a GitOps-based approach to API management: by using files that you can manage through Git repositories, you describe what should happen to APIs, then Traefik Hub makes it happen automatically. In addition to helping to keep configuration data centralized and accessible, this approach enables scalability and automation in the realm of API management.

Kubernetes-Native

We decided not just to be cloud-native in general, but to focus on creating a truly Kubernetes-native solution, because Kubernetes has become the de facto solution behind cloud-native environments.

By "truly Kubernetes-native," we mean that Traefik Hub is CRD-driven and can be managed directly through standard kubectl commands, rather than requiring a proprietary tool.

Intuitive and Rapid Time-to-Value

We don't think anyone should have to take a course or spend months reading documentation and following tutorials to manage APIs with Traefik Hub. Instead, we prioritized a simple, intuitive design that minimizes the learning curve for anyone who is already familiar with cloud-native concepts and tooling.

Lightweight, Composable Architecture

Most cloud-native architectures are composable, meaning they involve a variety of loosely coupled components. Traefik Hub adopts the same type of design by giving users freedom to pick and choose from a variety of modern solutions when building their stacks. In other words, Traefik Hub doesn't force you into a particular stack or platform; you can run it alongside other tools of your choice.

Data Plane and Control Plane Separation

Finally, we decided that it was important to keep the data plane and the control plane separate — partly because we believe in modular and composable design, as we just noted, but also to enhance the security and reliability of our API management solution. We'll say more later about exactly how we implemented the data plane with this goal in mind.

Building Traefik Hub: Challenges We Faced and Lessons We Learned

Deciding on high-level design principles is one thing. Actually implementing them is another. And while we don't have room in this article to discuss every technical decision we made as we built Traefik Hub according to the design concepts described above, we'd like to highlight a couple key choices we made as we worked through implementation challenges.

Tunneling

One was tunneling. Since we had to expose some resources publicly, creating tunneled connections to them was an elegant solution. We wanted the tunnels to be:

Highly available
Encrypted to protect data
Compatible with full-duplex mode (so that data could flow from either plane to the other plane simultaneously)
Transparent to firewalls, so users would not require public IP addresses to expose the data plane
Capable of supporting multiplexed data transfer

There are a variety of tunneling technologies that have the potential to support at least most of these goals. At first, we considered using TCP and SSH, but that would have required a public IP address, and the connection would not have been reliable because there is no true fallback solution on SSH disconnect. We also thought about using WebSocket and SSH, which doesn't require a public IP, but is still subject to the disconnect issue.

Since SSH-based tunneling didn't seem ideal, we began exploring approaches based on Yamux, an open source multiplexing library. Initially, we considered using Yamux alongside GRPC, but that felt too hacky and the data envelope was too large. TCP and Yamux also proved suboptimal because it would have required a public IP.

Finally, we settled on WebSocket and Yamux — which doesn't need a public IP, allows firewall passthrough, provides built-in encryption via TLS, and is more reliable following disconnects than an SSH tunnel.

Data Plane High Availability

When implementing the data plane for Traefik Hub, we wanted to make sure our solution would be resilient against regional network failures. In other words, we didn't want a problem in one part of the network to make the entire solution unavailable.

The first step in making this possible was to deploy dozens of brokers in different regions. But on their own, distributed brokers don't guarantee high data plane availability because a data plane would still fail if it's connected to just one broker and that broker goes down. So, instead, we designed our data plane to connect to at least three different brokers simultaneously — meaning that it can tolerate the failure of a minimum of two brokers before the data plane becomes unavailable.

In addition, we dynamically optimized load balancing for data plane connections. This ensures that if failures occur in some parts of the network, traffic is automatically rebalanced to keep data flowing as efficiently as possible.

Conclusion: A New Approach to API Management

Again, there's plenty more to say about how Traefik Hub works under the hood, and why we decided to make it work that way. But we hope that the details above have provided at least a basic sense of what our thought process was as we reconceptualized API management, as well as the types of technical iterations we worked through to get things just right — which means bringing scalability, simple incident and release handling, and intuitive collaboration to the realm of API management.

]]>

Traefik Hub - API Gateway Perfection

Emile Vauge — Tue, 19 Mar 2024 02:34:03 GMT

Traefik Hub has profoundly transformed API Management. It provides a fully declarative approach to define, manage and run APIs while offering modularity and freedom of choice for its users through deep integration with the Cloud-Native ecosystem.

API Management brings critical capabilities to help companies handle their API fleet. Runtime API Governance is the cornerstone of Traefik Hub API Management: dynamic API discovery, logical grouping through collections, change and incident management through versioning and linters, and ease-of-consumption through developer portals. Traefik Hub API Management also adds an advanced security layer on top of it with industry-leading Granular Access Control and provides rich awareness and control of the whole API pool with precise monitoring and state-of-the-art observability.

Delivering on this simple yet powerful vision, Traefik Hub now delivers essential capabilities to help companies seamlessly start their API journey. See this as the wildly popular and familiar Traefik Proxy plus core API-focused features like Access Control (OAuth, OIDC, etc) and native scalability capabilities like Distributed Rate Limiting.

To put it more simply: Traefik Hub seamlessly transforms Traefik Proxy into the perfect API Gateway!

Let’s dig deeper into it.

From Traefik Proxy To Traefik Hub API Gateway

One key element of Traefik Hub API Gateway is how seamless it is to upgrade from Traefik Proxy. Take an existing Traefik Proxy ingress controller installed on Kubernetes.

Step 1: Install the Traefik Hub agent with Helm, providing the same Traefik Proxy configuration options.
Step 2: Remove Traefik Proxy, et voilà, you have a fully functioning API Gateway, that will act as the ingress controller exposing the existing ingress ressources.

API capabilities are configured using the Traefik Proxy middleware well known mechanism.

To illustrate this, let’s take a rate limiter attached to an IngressRoute used with Traefik Proxy:

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: ratelimit
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ratelimit
spec:
  rateLimit:
    period: 1m
    average: 600

To add OAuth authentication to this existing IngressRoute, just add an OAuth middleware and Traefik Hub will enforce OAuth authentication on it!

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: ratelimit
      - name: oauth-client-creds
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ratelimit
spec:
  rateLimit:
    period: 1m
    average: 600
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-client-creds
spec:
  plugin:
    oAuthClientCredentials:
      url: https://tenant.auth0.com/oauth/token

How simple it is! As you can see, getting access to API Gateway capabilities when you already use Traefik Proxy is transparent.

Now, let’s explore all these new middlewares!

The many flavors of OAuth

OAuth 2.0 is an open standard dealing with resource access control and is the latest version of the authorization protocol OAuth. An OAuth client provides web, desktop, and mobile application authorization flows. We brought to Traefik Hub one of the most complete OAuth implementations which comes with two OAuth middlewares:

Token Introspection

Token introspection allows Traefik Hub to retrieve information about the user from an authentication server. Then, once the token is retrieved, it uses fine-grained configuration, including nested claims, to grant access or not to the route.

Let’s imagine that the token it retrieves is the following

{
  "active": true,
  "grp": "admin",
  "scope": "reader writer deploy",
  "referrer": "http://example.com/foo/bar",
  "areas": [
    "office",
    "home"
  ]
}

You can validate this token using many functions, for example:

Equals(‘grp’, ‘admin’) checks if the value of grp in the token is equal to “admin”
Prefix(‘referrer’, ‘http://example.com’) checks if the value of referrer in the token starts with http://example.com
Contains(‘referrer’, ‘/foo/’) checks if the value of referrer has the sub string foo. When used on an array of values, it will check whether the value is in the list or not.
And others…

Most importantly, you can combine any of these using operators (and, or, not) creating powerful ways to express who should have access to what.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oauth-oauth-intro
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-oauth-intro
spec:
  plugin:
    oAuthIntrospection:
      tokenSource:
        header: Authorization
        headerAuthScheme: Bearer

Client Credential

Client Credential allows Traefik Hub to secure routes using the OAuth 2.0 Client Credentials flow as described in the RFC 6749. Access tokens are cached using an external KV store.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oauth-client-creds
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oauth-client-creds
spec:
  plugin:
    oAuthClientCredentials:
      url: https://tenant.auth0.com/oauth/token

OIDC

OIDC is an authentication layer built on top of the OAuth 2.0 protocol. OpenID Connect allows an application to obtain user login information by exchanging cryptographic tokens with an identity provider and is often used to implement federated SSO between multiple applications. With OIDC, Traefik Hub allows you to delegate the authentication process to a third party (Google Accounts, LinkedIn, GitHub, etc.) to obtain the end-user information for authorization purposes, where you get the same set of functions / operators to define the rules.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: oidc
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: oidc
spec:
  plugin:
    oidc:
      issuer: "https://tenant.auth0.com/realms/myrealm"
      redirectUrl: "/callback"
      clientID: my-oidc-client-name
      clientSecret: mysecret

JWT

JWT is a popular tool for authenticating API calls and single sign-on (SSO) applications. It’s a method of digitally signing information as a JSON object. The JWT includes a set of claims, which typically describe what an authenticated user is allowed to do. With JWT validation, Traefik Hub is capable of verifying the validity of the token, extracting its information, and then as usual checking against the rules you want to define.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: jwt
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: jwt
spec:
  plugin:
    jwt:
      signingSecret: urn:k8s:secret:my-secret:signingSecret

API Keys

When you need a quick authorization mechanism, API Key is one of the best options. You configure the values that should allow consumption of the API, and where to retrieve it from (header, query param, cookie), and it’s as simple as that.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: apikey
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: apikey
spec:
  plugin:
    apiKey:
      keySource:
        header: Authorization
        headerAuthScheme: Bearer

Distributed Rate Limit

When scaling Traefik Hub with standard rate limiting, each replica computes its own rate limit, meaning that if you configure 10 requests per second, you’re actually allowing (instance number) * request limit over time. When reaching the limit, depending on how well traffic is load-balanced between the instances, there might be some (small) inconsistencies in the response.

The distributed rate limit solves this problem and eases the configuration, allowing each replica to be aware of the remaining allowed requests. When you configure a given request per time allowance, you’re guaranteed that whatever the number of replicas you run, this is what will go to your service.

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: service-foo
      port: 80
    middlewares:
      - name: distributedratelimit
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: distributedratelimit
spec:
  plugin:
    distributedRateLimit:
      limit: 100

Wrapping Up

An API journey is long and complex. The last thing you want is to choose the wrong product to support you. Start with an over-engineered product and your ability to quickly iterate under control will vanish. Start with a product you can’t upgrade and you will be locked in at an early stage, without the ability to evolve to more advanced practices and technologies.

Traefik Labs provides companies a very progressive stack to deploy and manage APIs. From the ingress controller use case with Traefik Proxy to route and load balance services, to the API Gateway with critical capabilities around security and centralized control, up to the full API Management solution. All this within Traefik Hub, the most modern and progressive API runtime platform for Cloud-Native environments.

]]>

Why does WAF matter in API security?

José Carlos Chávez — Mon, 18 Mar 2024 19:56:59 GMT

Nowadays, APIs power the internet. We see APIs (Application Programming Interfaces) everywhere: webshops, social networks, mobile apps, government sites and most of the media entertainment is delivered to us over APIs. This is why APIs are so important and also why protecting APIs is fundamental for every organization being on the internet.

According to OWASP Top 10 API Security Risks 2023[1], 4 of the top 5 security risks are related to authentication and authorization, however by October 2023, the U.S. SEC launched an investigation around the MOVEit mass-hack that has exposed the personal data of at least 64 million people[2]. Considered the biggest hack of 2023, it began by exploiting a previously unknown SQL injection vulnerability in MOVEit Transfer. On the same line, cvedetails.com reports that the majority of CVE security vulnerabilities in 2023 came as Cross-Site Scripting (XSS) attacks, followed by memory corruption and SQL injection [3].

While identity and permissions are the foundation of API security, footprint attacks (XSS, SQL injection, Remote Code Execution etc) are still leading the game when it comes to security breaches hence the importance of protecting our APIs looking at the footprints.

Web Application Firewall (WAF) as part of a wider solution

No security layer can address all possible attack vectors, there are different layers of security with different concerns which sometimes narrowly overlap. A multi-layered strategy that acknowledges the complexity of the threat landscape is preferred, hence the need for a diverse array of defenses where each layer is a barrier to attacks, and together, they create a resilient shield capable of adapting to the shifting tactics of adversaries.

WAFs are proxy-based tools that mainly inspect incoming HTTP(S) requests. Its capabilities vary, however the basic function is to provide an application layer filter for web and API traffic. This stage looks for malicious and unwanted content within incoming requests (headers and payloads) and acts accordingly as well as making sure that only allowed actions can be performed.

Some of the WAF capabilities that are crucial for API security are:

Known attack detection: Recognizing common attack strategies and blocking unauthorized access.
Malformed/anomalous request detection: Distinguishing legitimate API requests from those with malicious intent.
Virtual patching: Intelligently delivering security patches to early protect API endpoints until the root cause can be fixed.
Anti-bot automation: Identifying and blocking malicious bot traffic.
Audit logs: Producing audit information about matched rules and anomaly scores that can be aggregated to come up with actions.

Any system that receives traffic (internal or external) is a target for attacks and any system that is target for attacks can be protected by a WAF, from websites, blogs, government portals, e-commerces, forums. Some of the attacks categories a WAF can protect you against of is:

SQL Injection (SQLi): malicious traffic could manage to inject strings directly into SQL queries to perform undesired actions e.g. retrieve users data.
Cross Site Scripting (XSS): attacker injects malicious executable scripts into the code of a trusted application or website
Local/Remote File Inclusion: attacker includes local/remote files on the server to later execute them e.g. to expose server data.
Code Injection: attacker injects code into the application execution through a vulnerability e.g. to expose environment credentials in the output.
Session Fixation: permits an attacker to hijack a valid user session
Bot Detection: the process of identifying and distinguishing between human users and automated bots.
Metadata/Error Leakages: leaks server or implementation metadata or internal errors to the attacker e.g. to exploit known vulnerabilities.

In a multilayer security approach, a WAF sits in the first line, protecting our systems from traffic attacks, even before authentication or authorization as those could also be compromised by a footprint attack.

PCI DSS 4.0 has WAFs as a hard requirement

The Payment Card Industry Data Security Standard (PCI DSS) is an information security standard used to handle credit cards from major card brands. The requirements are specifically designed to protect against security breaches.

While merchants, e-commerce businesses, retailers, and financial institutions are common entities that must comply with PCI DSS, other industries and service providers also fall under its scope. For instance, healthcare organizations, hospitality businesses and in general any internet site that handle card payments for bookings and services must comply with PCI DSS.

Even an organization that does not process cardholder data could follow the PCI Standard to implement a robust cybersecurity program for any of its important data and gain the confidence of its customers.

In version 4.0 (deadline: March 2025) WAF takes a fundamental role. Requirement 6.4.2 [4] explicitly requires affected businesses to "Deploy an automated technical solution for public-facing web applications that continually detects and prevents web-based attacks" with compliance to at least the following criteria:

Is installed in front of public-facing web applications and is configured to detect and prevent web-based attacks.
Actively running and up to date as applicable.
Generating audit logs.
Configured to either block web-based attacks or generate an alert that is immediately investigated.

WAFs targeting deeper protection

While originally WAFs were created and designed to protect API gateways as a first line of defense from outer traffic (so called north-south), strategies like zero-trust tell us that we should be protecting the system components as if the attacker is already inside of the network.

Hence in the same way we leverage a WAF in the API gateway we should be protecting the individual components from the internal traffic (so called east-west) with a WAF using a refined set of configurations and more symbiotic deployments where WAF is in the ingestion path of the component e.g. in the sidecar as part of a service mesh.

This is specifically important when it comes to lift and shift. Legacy systems being onboard in the cloud, also onboards all its vulnerabilities. Sometimes patching these systems is not an option because the code is too old, it lacks tests or requires risky upgrades while being critical. A WAF can help to protect them in the cloud without the need of doing changes into its code but instead rolling out a reverse proxy in front e.g. as a sidecar intercepting all the incoming traffic looking for exploits.

WAFs as part of the development workflow

With the proliferation of microservices, macroservices, lift and shift and lambdas, teams became more independent in a fast changing architecture which means that security teams can roll out general purpose organization-wide policies but it is the individual development team’s responsibility to compliment the security measures with curated WAF rules more related to their components and its specific risks.

This is a shift in the security mindset which moved from an afterthought and only delegated to security teams into a shared responsibility between development and security teams. A cloud native WAF promotes the so-called “shift-left” which is the practice of moving security, testing, quality, and performance evaluation early in the development process and making development teams accountable for that.

WAF unblocking your security pipeline

In December 9th 2021 a new vulnerability was disclosed in the super popular Apache log4j library, affecting an estimated 10 percent of all digital assets at time of discovery [5], allowing remote code execution (RCE) on underlying servers that run vulnerable applications. The next day, Log4j 2.15.0 was released and security teams were urged to install the latest version. In the next seven days two more vulnerabilities were disclosed and by January 4th, US Federal Trade Commission told companies to patch Log4j vulnerability threatening legal actions.

Two years later, more than 1 in 3 applications using Log4j currently run vulnerable versions of this library [6]. Research found that, in general, once developers are alerted to a vulnerable library through a scan, 50 percent of vulnerabilities are fixed in 89 days overall, in 65 days for high severity vulnerabilities and in 107 days for medium severity vulnerabilities.

These numbers tell us that patching vulnerabilities does not occur as frequently and quickly as one might expect, however it is a negligence to leave systems vulnerable and hence WAF features like Virtual Patching [7] can chime in to protect those unpatched systems.

Virtual patching is a method of protecting new vulnerabilities in the short-term to keep hackers away from breaching systems until they can be patched in code. A newly discovered CVE in OSS triggers several processes both in the OSS code as well as proprietary code and a patch landing into users’ systems requires several other processes leaving the system vulnerable during that time unless a virtual patch has been applied.

Conclusion

WAFs play a critical role in API security, serving as an essential component of a multi-layered defense strategy. By detecting and mitigating a wide range of threats in real-time, WAFs help safeguard APIs, protect business-critical sensitive data, and ensure the integrity of online services.

WAF not only takes part of the operational stage of the systems bringing protection and compliance, it brings more benefits across the entire security strategy across the engineering teams that will translate into customer trust and good reputation.

References

]]>

Revolutionizing API Operations: A Dive into GitOps-Based API Management

Immánuel Fodor — Thu, 14 Mar 2024 22:26:21 GMT

APIs (application programming interfaces) are a hot topic these days with the rise of machine-to-machine communication and AI, microservice-based cloud native architectures. Add to that the never-ending need for efficiency to reuse existing software components and services to build up solutions faster and cheaper without reinventing the wheel.

Traditional API management solutions are often slow and suboptimal for efficiency, built with click-operations in mind, leading to stateful and procedural proprietary scripting practices, which result in a steep learning curve, preventing the fast onboarding of new team members, and making collaboration difficult at scale. Many existing tools are 10+ years old, built before the Docker and Kubernetes era, which makes them far from ideal for modern environments. Add all these up, and you get bottlenecks, long release times, faulty deployments, and outdated documentation.

Here, we arrive at the need for a more efficient and up-to-date central view of managed APIs, their configurations, usage, and health analytics, with the capability to audit changes and maintain an up-to-date single source of truth.

We invite you on a journey to unravel the power of GitOps in streamlining API management, offering transparency and reliability, and paving the way for a more agile and collaborative future.

GitOps: A Proven and Efficient Answer

GitOps, a methodology widely adopted by infrastructure teams operating applications and services, can bring an interesting, fresh approach to API management with a focus on operational efficiency.

Let’s see the principles behind GitOps:

Declarative configuration: Humans declare the desired state, and systems autonomously determine the steps to achieve it—similar to a GPS defining the route given a desired final destination. Automated early checks can ensure that the declared state is consistent, error-free, and compliant. The declarative approach promotes a shared understanding within teams, fostering collaboration by discussing freely and within the code itself.
Versioning and immutability: Committing every change to Git in an immutable fashion enables a clear audit trail and the ability to reproduce current and previous configurations. As a result, it ensures accountability and traceability throughout the configuration lifecycle. In case of issues, Git provides seamless rollbacks to any previous working state, minimizing downtime.
Automatic pulling: When Git is established as a single source of truth for configurations, automatic pulling mechanisms eliminate surprises, creating a predictable modern environment. Substituting human effort with automated pipelines guarantees that changes are deployed consistently and efficiently, reducing the likelihood of manual errors and simplifying compliance.
Continuous reconciliation: Running the reconciliation process in a continuous loop can automatically correct any unexpected deviations from the declared state, contributing to a reliable and resilient system architecture. It also ensures stability by preventing unintended changes from piling up.

GitOps aligns seamlessly with the Unix and Kubernetes philosophy of doing one thing well and being composable with other tools. By adhering to these principles, GitOps ensures simplicity, efficiency, and compatibility, making it a versatile and powerful methodology for managing infrastructure—and API operations, as we’ll see soon.

The Philosophy of GitOps for API Operations

The core of GitOps for API operations lies in treating APIs as infrastructure, represented by Custom Resource Definitions (CRDs) within Kubernetes. This allows API configurations to be managed declaratively, just like other Kubernetes resources, simplifying management and reducing configuration errors. See an API CRD below as an example:

apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: employee-api
  namespace: apps
  labels:
    area: employee
    module: crm
spec:
  pathPrefix: "/employees"
  service:
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000
    name: employee-app
    port:
      number: 3000

GitOps promotes human-readable and editable configurations stored in Git repositories. APIs, rate limits, security policies, and other constructs are clearly defined, enabling collaboration between developers, operations, and security teams.

GitOps-based API Operations provide full auditability with version control. Every change, comment, and approval is linked to its specific version, creating a clear historical record. Deployment status is continuously visible, ensuring transparency and accountability.

Tools that understand declarative API configuration can perform automated checks and analysis of configuration changes before deployment. These checks identify potential errors, security vulnerabilities, and performance impacts, allowing proactive action to address them before causing disruptions.

By leveraging GitOps, API-related deployments become fully automated. Changes are committed to Git and automatically applied to the API Gateway or API Management platform. This eliminates manual installations, reduces downtime, and minimizes human error by eliminating repetitive tasks.

API operations become agile and reversible with GitOps. With Git as the single source of truth, rolling back to previous versions becomes simple, offering a safety net for experimentation and updates. Continuous updates become the norm, ensuring APIs remain secure and aligned with evolving business needs.

Operational Benefits Adopted from GitOps to APIOps

Building on the core philosophy of GitOps for API Operations, let's dive deeper into the specific operational benefits this approach unlocks:

Separation of concerns: streamlined development and deployment. API code and configuration releases can be decoupled, allowing faster innovation and bug fixes on either side without impacting the other.
Transparency and auditability: trace every change with confidence. You can easily see who made what change, when it happened, and why. This empowers compliance and security teams to assess risks and identify potential issues. Debugging configuration issues also becomes a breeze accelerating problem resolution.
Authentication and authorization: control who can change what. This is a two-fold concept. You can implement role-based access control (RBAC) within your GitOps workflow, defining who can push changes to different parts of the API configuration. Also, you can define RBAC for APIs in Git, ensuring only authorized personnel have access to sensitive API paths and methods. Each commit is associated with a user, promoting accountability and preventing unauthorized modifications of API access policies.
Risk reduction: rollback and logging assurance. Experiment and make changes fearlessly with confidence, knowing you can easily revert if needed. Git’s comprehensive record-keeping helps identify the root cause of issues and facilitates post-mortem analysis for continuous improvement.
Security policy: code-enforced vigilance. By integrating automated security scans and checks early in the process, GitOps helps shift left security, catching and addressing vulnerabilities before they reach production. This proactive approach enhances the overall security posture of your APIs.
Scalability and consistency: a self-healing API infrastructure. As API deployments grow, GitOps scales efficiently, managing configurations consistently across environments. API configurations are automatically re-applied when deviations occur, ensuring the desired state is maintained.

By adopting GitOps principles for API management, you gain a streamlined workflow with declarative configuration, clear audit trails, granular access control, and the safety of effortless rollbacks. The transparency improves security through early vulnerability detection and promotes consistency in multiple environments for a highly scalable and manageable API infrastructure.

Traefik Hub: GitOps-Driven Kubernetes-Native API Management

Imagine an API Management solution seamlessly integrated into your Kubernetes environment, built with GitOps for rock-solid configuration and automation. We’ve built Traefik Hub with all these principles and benefits in mind on the well-known and highly performant open source Traefik Proxy to make this dream a reality.

No more complex configurations! Traefik Hub leverages familiar Kubernetes Custom Resource Definitions (CRDs) for publishing and managing APIs. Simply declare your APIs, their versions, access policies, and group them using standard Kubernetes labels and selectors. This intuitive approach grants you powerful control without sacrificing ease of use.

Traefik Hub is built up from the following architectural components:

The Traefik Hub Agent: The guardian standing at your cluster's door, securing and monitoring incoming traffic on the data plane, hosting your API gateways, and ensuring smooth operation.
The API Dev Portal: Empower your API consumers with a dedicated portal. They can explore API documentation, request API keys, and get the most out of your offerings. The portal UI is available as an open-source example on GitHub, which you can tweak and extend to your liking.
The Traefik Hub UI: Gain a bird's-eye view of your API kingdom. View connected agents, API configurations, and other resources across multiple clusters and users.
The Traefik Hub Static Analyzer: Shield your APIs from errors before they deploy! This proactive tool meticulously analyzes configurations, pinpointing potential issues before deployment. Its human-readable impact analysis, embedded directly in Git pull requests, offers clear insights into the consequences of every change, empowering informed decisions and ensuring smooth deployments.

In the next sections, we’ll see two use cases of utilizing GitOps-based API Management with Traefik Hub. We’ll focus on operational excellence in day 2 operations, when every architectural component is in place, and you need to make APIs run smoothly and securely.

API Change Management with GitOps

In the first use case, we look into configuration safety. Traefik Airlines, our fictitious company, is a modern airline company that offers various digital services to its customers and partners. To meet evolving needs while maintaining business stability, Traefik Airlines must ensure its API Change Management is robust to minimize service downtime caused by configuration errors. Let's explore how this can be achieved by using Traefik Hub’s GitOps-driven API management.

Traefik Airlines regularly publishes changes to its Git repository, where the configuration of all managed internal and external APIs is stored. However, they configured the Traefik Hub Static Analyzer to run automatically for every pull request to Git.

The analyzer has a linter functionality that acts as your first line of defense, scanning configurations for errors and preventing them from reaching production. The errors found are not tucked away in an obscure location but put into context next to the error.

In the above screenshot, the analyzer running as a GitHub Action catches a typo that would cause an outage in one of the Airline APIs by referencing a non-existent Kubernetes service underlying the API. The action prevents the pull request's merge and gives contextual help on the error to let it be corrected in a follow-up Git commit that is now safe to deploy into production.

The linter can check for various configuration errors, including duplicate, orphan, childless, and invalid resources; resource references; selector definitions; conflicting API paths; unknown operation sets; and service ports.

The analyzer can also do an impact analysis on the change set of a pull request preventing modifications with significant differences from being merged and ensuring the cluster remains stable.

The above screenshot shows the human-readable reports outlining the affected APIs and users of an API consumption rate limit change. It allows the pull request reviewer to confidently decide whether to merge the changes or refine them further.

API Incident Mitigation and Resolution with GitOps

In our second use case, we see how you can manage your APIs with the confidence that any misstep can be instantly undone.

With GitOps's magic touch to API management, rolling back a faulty update of an API to its previous, healthy state is a breeze. No more scrambling, no more downtime, no more complaints from API consumers—just a quick rewind via a click of a button and you're back in business.

But GitOps doesn't stop there. It empowers your developers to become debugging detectives. When an error pops up, the faulty state is preserved like a crime scene, waiting to be examined. This traceability buys precious time to delve into the root cause, craft a fix, and deploy it seamlessly through a new pull request.

Wait, there's more! OpenTelemetry metrics paint a vivid picture of your API landscape. They expose error rates, latencies, and request counts, all categorized by API, version, and user. Want to zoom in on a specific API or version? No problem! Drill down with ease and gain laser-sharp insights into user behavior.

And the best part? You don't need to rip and replace your existing tools. OpenTelemetry integrates seamlessly, acting as a common language for your event data. This opens the door to exciting possibilities—e.g., effortless event correlation with Git actions, AI-powered analysis, and a world of customizations tailored to your needs.

The above screenshot shows Traefik Airline’s monitoring dashboard with GitHub pull requests and Flux deployment events correlated to OpenTelemetry-based API metrics collected by Prometheus and visualized in Grafana. Whenever something goes bad, and the unexpected happens, it’s easy to see which change caused the problem and roll it back to restore healthy operations.

You can dive into the code and see how GitOps API management can transform your API Operations for yourself with our example repository.

Slash Costs, Boost Efficiency: GitOps API Management is Your ROI Goldmine

Renowned tech experts at Gartner have labeled GitOps as an emerging technology on the verge of widespread acceptance, marking its transition from early stages to the brink of mainstream adoption. Time is money, and GitOps saves both. Forget the days of API Management costs draining your budget. Get ready to unlock efficiency, agility, and massive cost savings with the revolutionary power of GitOps for your API Operations.

Accelerate releases: Get changes from dev to production faster than ever. Save precious time and human effort across your team.
Avoid slashing SLAs: No more costly downtime due to faulty updates. With a single click, rewind to a stable state and keep your APIs humming.
Solve problems efficiently: Drill down into performance metrics, pinpoint issues before they snowball, and avoid costly outages.
Work on what matters: Automate repetitive tasks and eliminate manual interventions, freeing up your team for higher-value work.
Pay only for what you use: Meet demand fluctuations effortlessly with dynamic infrastructure provisioning, scaling up and down gracefully.
Eliminate waste: Say goodbye to siloed data, gain a holistic view of your API landscape, and optimize resource allocation.

Reduced effort, faster cycles, and fewer outages translate to measurable financial benefits. GitOps-driven API management is not just technology, it's a philosophy and a financial game-changer.

Wrapping It Up

In conclusion, the GitOps approach represents a significant paradigm shift in API operations. Grounded in the declarative principles of Kubernetes, it offers a sophisticated framework that empowers a new era of API management focused on collaboration, transparency, and control. This evolution is particularly relevant in an era dominated by the widespread adoption of Kubernetes and the integration of machine-to-machine communications.

With the operational benefits of GitOps-based API operations, you can achieve faster iterations, enhanced security, and improved overall API governance, unlocking new levels of agility and efficiency for your organization.

]]>

Why Modern API Management Needs an End-to-End GitOps Strategy

Immánuel Fodor — Fri, 01 Mar 2024 19:44:10 GMT

GitOps isn’t just another IT buzzword; it’s quickly transforming how software development operates, making deployments more reliable, repeatable, and automated. GitOps is used to manage infrastructure and workloads, but it hasn’t taken hold in managing APIs. While this modern operations model is used in API development, API management has yet to benefit from GitOps-driven processes, and it’s sorely needed.

With the rise of cloud-native development and microservices, a growing majority of development involves APIs, and managing them is key to both development velocity and application security.

By including GitOps throughout the API management process, DevOps and platform engineers are able to enforce consistency and make it possible to scale operations, all while enhancing the developer experience, fostering collaboration, and bolstering security. Let’s take a deeper look at how GitOps enables better API management.

Streamlined Workflows: Orchestrating Deployments in the GitOps Era

In the GitOps landscape, every deployment is a reflection of the desired state defined within a Git repository. This declarative approach, where the desired state of infrastructure and applications is codified, demands seamless and efficient workflows.

GitOps-driven API management ensures that as changes are pushed to the Git repository, they are mirrored in real-time across the infrastructure. This real-time synchronization not only accelerates deployment cycles but also minimizes manual intervention, a cornerstone of the GitOps philosophy.

But it's essential to understand that the role of API management isn't limited to just facilitating deployments. It extends to cataloging and securing both internal APIs that drive microservices and external APIs that interface with third-party services.

Most of all, by automating tasks that were traditionally manual, GitOps-driven API management reduces the scope for errors. For example, when a new API endpoint is added to a service, the management tool can automatically catalog it, apply predefined security policies, and ensure it aligns with the desired state in the Git repository.

Consistency: Maintaining Equilibrium in the GitOps Landscape

When it comes to GitOps, consistency is more than a best practice—it's the point. With every piece of infrastructure and application configuration codified within a Git repository, GitOps is about ensuring that the live state mirrors this codified state.

One way in which GitOps enhances consistency in API management is through the use of validation of changes or “early checks.” These early checks can compare changes against a set of rules, failing violations to ensure that nothing questionable is implemented. If an issue should arise, you have the ability to add any knowledge gained through fixing the problem to the set of rules, preventing it from happening again.

API management tools should ensure that as configurations are updated or new ones are added to the Git repository, they are consistently applied across all environments. Whether it's a development sandbox, a staging area, or a production setup, the behavior and configuration of APIs remain uniform. This uniformity is achieved through centralized configuration management, where API configurations are stored, versioned, and synchronized with the Git repository, along with the stateless nature of Kubernetes and its underlying immutable infrastructure.

Such centralization eliminates the discrepancies that often arise between environments, ensuring environment parity—a core tenet of GitOps. With it, teams can be confident that an API's behavior in a development environment is replicated identically in production, reducing "it works on my machine" issues.

More than that, in the event of an outage, not only can changes be rolled back in a matter of minutes, but you can duplicate the environment in order to debug. So instead of your engineers getting jolted out of bed at 2am on a Sunday while you lose money every minute you’re down (the Ponemon Institute estimated the average cost of downtime at $9000 per minute!), the system can rollback automatically, and the engineers can come in on Monday to an error report they can debug at their leisure before reapplying.

Furthermore, in a GitOps-driven world, changes are continuously integrated and deployed. This ensures that every deployment, every change, and every API call aligns perfectly with the state defined in Git. And if the API management tool is based on Kubernetes, it reduces the learning curve, because developers are already (by and large) familiar.

Monitoring & Analytics: The Pulse of GitOps Operations

The continuous flow of changes within an environment makes it necessary to keep an eye on the system's health and performance. These observability capabilities are where API management's monitoring and analytics capabilities shine, acting as the pulse-check for GitOps operations.

API management tools provide real-time insights into the performance and health of APIs. They track crucial metrics such as response times, error rates, and throughput. In a GitOps-driven environment, where deployments can be frequent and automated, understanding these metrics becomes vital. It helps teams identify potential bottlenecks, optimize performance, and ensure that the system remains resilient even as changes are continuously integrated.

Beyond performance metrics, analytics delve deeper into API usage patterns. They reveal which APIs are most accessed, which ones might be underutilized, and even which ones could be potential points of failure. In the context of GitOps, where the desired state of the system is codified, these insights help in refining this state, ensuring it aligns with actual usage patterns and needs.

Additionally, in the event of discrepancies between the Git-defined state and the live state, monitoring tools can trigger alerts. These alerts can be invaluable in a GitOps setup, allowing teams to quickly identify and rectify deviations, ensuring that the system remains consistent and aligned with the repository.

It’s also important that the tools involved can easily integrate with other systems. For example, a tool that provides observability based on OpenTelemetry can send data to dozens of other tools that can easily ingest it.

Scalability: Growing with GitOps

As organizations evolve, so do their digital needs. The number of services, applications, and consequently, API calls can expand exponentially. In the GitOps paradigm, where infrastructure and application configurations are continuously synchronized with the state defined in Git, ensuring that this growth is managed efficiently is crucial.

API management is the linchpin in this growth story. It provides the tools and mechanisms to handle an increasing volume of API calls. For example, it can provide load balancing, which ensures that incoming API requests are appropriately distributed across servers so that no single server or service becomes a bottleneck and performance is optimized.

But sometimes it’s not about distributing numerous requests, but limiting them. In a GitOps-driven environment, where changes can be continuously deployed, it's vital to ensure that no service or consumer overwhelms the system with excessive requests. Rate limiting sets a cap on the number of requests a consumer can make in a given time frame, ensuring system stability.

Furthermore, as the infrastructure grows, the Git repository becomes a more extensive catalog of configurations and desired states. API management ensures that, regardless of the scale, every API call, every service, and every microservice aligns with this cataloged state, ensuring that scalability doesn't come at the cost of consistency or performance.

Using GitOps with API management also enables you to scale teams as easily as you scale your infrastructure, because the Pull Requests that enable GitOps to work, enable easy collaboration between multiple developers working on a single system.

Developer Experience: Thriving in a GitOps Ecosystem

In the GitOps landscape, where infrastructure and application changes are driven by Git repositories, the developer's role is pivotal. They're not just coding features or fixing bugs; they're actively participating in the deployment and operational processes. In this context, enhancing the developer experience becomes paramount, and API management plays a crucial role in this enhancement.

API management tools offer a suite of features tailored for developers. One of the standout features is the automatic generation and updating of API documentation. In a GitOps-driven environment, where changes can be frequent and continuous, having up-to-date documentation is invaluable. It ensures that developers, regardless of their familiarity with an API, can quickly understand its functionalities, parameters, and expected behaviors.

Moreover, API management tools often provide sandbox environments. These are isolated spaces where developers can test new API integrations or changes without affecting the actual production environment. In the GitOps world, where changes in the Git repository can trigger deployments, having a safe space to test and validate these changes before they're merged into the main branch can save a lot of time.

And GitOps-enabled API management provides benefits to both sides of the API equation. The API publisher/producer side gets a way to abstract the APIs themselves, as well as a declarative way to deploy them that includes error checking and code review. For the API user/consumer side, in addition to up-to-date documentation, the API developer portal shows what APIs that are available and how to call them.

Auditability and Security: Safeguarding the GitOps Pipeline

In GitOps, every change to infrastructure or applications is initiated through a Git commit, making the Git repository a chronological record of all changes. This inherently provides a level of auditability. However, when APIs come into play, especially in a microservices architecture, the complexity multiplies, and GitOps can help to manage that complexity.

API management tools provide comprehensive logging and tracking capabilities, tracking every API call, every data fetch, and every modification. This granular logging ensures that organizations have a clear trail of who accessed what data, when, and for what purpose. In the context of GitOps, where the desired state of infrastructure is defined in Git, having a clear understanding of how APIs interact with this state is invaluable. It provides insights into potential discrepancies and aids in forensic analysis.

Security, on the other hand, is non-negotiable. As the gateways to data and services, APIs are prime targets for malicious attacks, and centralized API management plays a pivotal role in uniformly enforcing security policies. But without the centralization provided by GitOps, there’s the opportunity for different teams to manage things in their own way, eliminating many of the benefits a GitOps-enabled system provides. Access controls can be defined and enforced, ensuring that only authorized entities can access specific APIs. Moreover, advanced threat protection mechanisms can identify and block malicious patterns or anomalous behaviors, safeguarding the system.

Incident Management: Navigating Challenges in the GitOps World

Even with the most meticulous planning and robust systems in place, incidents are an inevitable part of the software lifecycle. In the GitOps paradigm, where infrastructure and application states are continuously synchronized with Git repositories, the ability to respond swiftly and effectively to incidents is crucial.

GitOps-driven API management plays a pivotal role in this rapid response mechanism. One of the standout features of modern GitOps-driven API management tools is the capability for quick rollbacks. If an API change leads to unexpected behavior or a system outage, teams can use the same Git-based workflows to roll back the entire API platform to a previous, stable state. This rollback isn't just about reverting code; it's about ensuring that the entire ecosystem, from infrastructure to data flows to service integrations, is restored to a known good state.

Furthermore, in the unfortunate event of an incident, real-time monitoring and analytics provided by API management tools become invaluable. They offer insights into the root cause, be it an API overload, a data breach attempt, or a faulty integration. By pinpointing the issue, teams can not only address the immediate concern but also implement preventive measures for the future.

Another crucial aspect of incident management is communication. In the heat of a situation, effective communication with stakeholders, be it developers, operations teams, or end-users, is vital. API management tools often come equipped with notification systems that can alert relevant parties about the incident, its impact, and the steps being taken for resolution, but because GitOps tools already handle incident management for software, applications, and services, adding an additional layer is an unnecessary level of complexity. Ideally, your API management tool will integrate with GitOps to simplify matters.

Embracing the Future with GitOps and API management

In the evolving landscape of software deployment, GitOps has firmly established itself as the blueprint for operational excellence. Yet, it's the seamless integration with API management that truly amplifies its potential. Together, they form a powerful duo, streamlining workflows, ensuring unwavering consistency, and offering unparalleled insights into system performance.

As organizations continue to navigate the complexities of modern development, the harmony between GitOps and API management will be instrumental in driving success. By embracing this synergy, businesses are not only future-proofing their operations but also setting the stage for innovation, growth, and resilience in the face of ever-changing technological challenges.

]]>

Traefik Proxy v2.11 is Now Available! Here are the Latest Updates.

Emile Vauge — Thu, 22 Feb 2024 17:24:58 GMT

On the heels of our announcement last week regarding the new release candidate of Traefik 3.0, we are excited to bring to you Traefik Proxy 2.11, an important update to the 2.x branch underscoring our ongoing commitment to support this branch based on continuous feedback from the community and our customers.

In a nutshell, Traefik Proxy 2.11 extends Redis support to Redis Sentinel, is more efficient dealing with open connections, is safer with sticky cookies, and adds a migration path to Go 1.22, along with several bug fixes.

Let's dig deeper into this latest release and the details of your new features and how to use them.

Redis Sentinel Support

Traefik Proxy has a unique configuration model that allows runtime changes to its dynamic configuration, so you can maintain high availability without dropping open connections by an unnecessary full proxy reload. From the multiple providers available, the Redis provider has been around since Traefik Proxy v2.2.

This release adds support to Redis Sentinel (PR#10245), which provides high availability for Redis when not using Redis Cluster. One of our Traefik Enterprise customers uses it for storing OIDC tokens, which we thought could could benefit the wider community, so we brought it to the open-source version.

Open Connections Improved

Before Traefik Proxy v2.11, HTTP connections would stay open until clients closed them. This behavior could lead to more open connections and higher memory usage. Furthermore, when coupling Traefik Proxy with an upstream Load Balancer, this connection management method prevented an efficient traffic distribution when scaling up Traefik Proxy instances. The Load Balancer would stick to the same instance rather than ensuring a more even traffic distribution by sending requests to the newest instances.

To tackle this, we’ve introduced two new transport options for entrypoints (PR#10247). You can use both options together or choose one or the other:

keepAliveMaxRequests: limits the number of requests each client can make within one connection. Whenever a client sends a request within an open connection, if this request number exceeds the keepAliveMaxRequests threshold, Traefik Proxy will ask for a connection close (sends header Connection:close).
keepAliveMaxTime: limits the time a client can keep reusing the same connection for its requests. Whenever a client sends a request within an open connection, if the elapsed time between the first and current requests exceeds the keepAliveMaxTime threshold, Traefik Proxy will ask for a connection close (sends header Connection:close).

Load balancing is a big topic in networking, we even have a free Traefik Academy video course on it if you want to learn more about the ins and outs of load balancing. There are different ways to achieve it, Weighted Round Robin (WRR) is one of them. With the File or IngressRoute providers, you can assign weights to services and let Traefik Proxy load balance the requests between these available services based on the weights.

When using WRR, a sticky session cookie is used to let the client know which server handles the first response. On subsequent requests, the client should send the cookie with the value set to keep the session alive with the same server. However, the contents of this cookie were previously not encrypted, which could expose the internal IP address of the proxied service.

We’ve listened to the community's voice and provided a fix (PR#10243) to hash the contents of the WRR sticky cookies. This way, it’s not possible anymore to deduct information about the server handling the request.

Other Changes

The / character in router names can now be used through URL encoding/decoding the YAML keys. This change helps avoid 404 errors on the dashboard for aptly named routers (PR#10292).
The ReadHeaderTimeout has been fixed for PROXY protocol by using the ReadTimeout configured on the entrypoint (PR#10320).
We have fixed Kerberos and NTLM authentication with IIS server by “sticking” to the TCP connection with the backend for a single client TCP Connection when it’s necessary (PR#10405).
The middleware IPWhiteList has been deprecated in favor of the new IPAllowList middleware (PR#10249).
The Go library used has been updated to take advantage of the new features embedded in Go v1.22 as well as benefiting from security fixes. This update impacts both the cipher suites and TLS minimum version management, for more information, please read the migration notes.
Our documentation’s readability, wording, formatting, link accessibility, and examples have also been improved.

Wrapping It Up

With 2.11, Traefik is smarter, safer, more efficient, and future proof. We want to thank all community members who gave feedback, asked questions and challenged the team. This makes Traefik much better.

If you are interested in contributing to Traefik Proxy, join us on GitHub, where you can request features/enhancements, help us design the next version, or even get your hands dirty by creating or reviewing pull requests. For more information, check out the Release Notes and updated Docs. If you have questions, pop over to our Community Forum, where you can work with other community members to find your answers. And as always, we look forward to your feedback.

]]>

Announcing Traefik Proxy v3.0 RC1

Emile Vauge — Wed, 14 Feb 2024 17:10:51 GMT

Almost five years ago, in September 2019, we released the previous major version of Traefik: v2.0. Since then, Traefik has been more active than ever. It has achieved a unique, long term traction with more than 3 billion downloads and 700+ contributors. Traefik is ranked in the Top 15 on DockerHub, has 46,000 GitHub stars, and continues a five-year streak as the #1 API Gateway according to OSSInsight.io. What a ride!

We have designed this major release around a few key aspects. First of all, we added support for popular, emerging technologies—WebAssembly (Wasm), OpenTelemetry, and Kubernetes Gateway API. And then, of course, because the cornerstone of a proxy is the routing and the security, we revamped some key parts of the routing rules and added support for some leading edge technologies like HTTP/3, SPIFFE, and Tailscale.

Another critical aspect of any major release, which by design contains breaking changes, is to provide the smoothest user experience migrating from the previous version. Not only are we providing a complete migration guide, but we ensured that the transition process from the previous v2 to the new v3 has been streamlined, by ensuring backward compatibility with v2 syntax while offering a progressive path for adopting the v3 syntax.

Finally, Traefik is and has always been a community-driven project. A huge effort has been made on improving the user experience for contributors with the help of strong and clear processes within a welcoming and respectful environment.

I’m extremely proud to see that this project, born 8 years ago, has become such a critical piece of the modern cloud-native infrastructure stack, with the help of such an amazing community.

Alright, let’s jump into the details of v3 🙂

Ease of Migration

A few years back, when we introduced Traefik v2 to the world, the team was so excited to improve the project that we underestimated one thing: migration experience from v1.x to v2.x. Not that our migration plan was empty, far from that: we wrote an extensive migration guide, we even provided a migration tool to help users convert their configuration to the new format. But that wasn’t enough and let’s be honest, the adoption of this new version took longer than what we expected. Clearly, this wasn’t something we wanted to see again with Traefik v3. A major version means breaking changes obviously, but that shouldn’t imply a painful migration experience.

With Traefik v3, we are introducing a streamlined transition process from v2, ensuring backward compatibility with v2 syntax while offering a progressive path for adopting the v3 syntax, effectively overcoming the challenges encountered in previous migrations.

Here are the guidelines we followed:

Obvious first thing is to provide a detailed migration guide describing every change that need to be made from v2.x to v3
Breaking changes on the static configuration are OK as soon as we provide user friendly logs when v2.x options are still used and link to the new v3 format
The dynamic configuration will support both v2.x and v3 configuration formats. Imagine a scenario with 200 ingress resources to migrate. You will be able to use Traefik v3 without any change, and you will be able to progressively upgrade your ingresses to the new format.

All in all, this should be pretty straightforward for Traefik users to adopt this new major version and we will carefully listen to your feedback to further improve upon the migration guide.

Routing & Security

One of the fundamental purposes of Traefik is to route network traffic in a secure and intelligent manner. This is the core part of any proxy, but Traefik has always been the cutting edge. And this v3 release doesn’t disappoint with a huge number of new features and improvements.

Let’s start with HTTP, the backbone of the Internet. Since HTTP/2 was approved in 2015, a lot of work has been done on defining the next milestone: HTTP/3. And this one is big as it doesn’t rely on the good old TCP anymore, but instead UDP (being based on QUIC, a new transport protocol developed by Google in 2012). And it comes with many benefits like enhanced performance, faster connection establishment, simpler encryption, etc. HTTP/3 is now out of experimental and fully production ready in Traefik! Thanks to the maintainers of the Quic-Go Project, and especially Marten Seeman, for their efforts in developing this library and enabling this advancement.

Another key technology that is being widely used lately in cloud-native environments is gRPC. gRPC is a modern Remote Procedure Call (RPC) framework that can run in any environment. It uses Protocol Buffers as the interface description language, and provides advanced features such as bidirectional streaming and flow control, cancellation, and timeouts. Traefik v3 now supports gRPC-Web, which adds the capability to call a gRPC service from a web app within a browser. Additionally, Jeremy Jacque added gRPC healthchecks support to Traefik.

Deploying dozens or hundreds of services in production comes with many challenges. One of them is enabling workloads to prove their identity to trust each other. Enter SPIFFE, the Secure Production Identity Framework For Everyone Project defines a framework and set of standards for identifying and securing communications between application services. Thanks to Julien Levesy, Traefik now supports SPIFFE mTLS with its backend servers!

Traefik had a routing syntax that was already very powerful, but we figured out that some parts could be further improved and unified to make the user experience even better.

If you need to protect websites within a Tailscale network, Traefik can now request TLS certificates from the Tailscale API (provisioned with Let’s Encrypt)! The long-awaited Brotli compression algorithm support was added by Greg Linton which in short means quicker website loading. Michael Kuhnt brings SNI routing with Postgres STARTTLS. And finally, you are now able to fully configure your TCP server transports.

As you can see, the work that has been done is just insane and reinforces the pioneering routing & security capabilities of Traefik while sticking to the original virtues of a smooth user experience and top-notch performance.

Augmented Traefik

Back in 2019, with the release of Traefik v2, we had introduced an extension engine, Yaegi, that enabled many to develop middleware (or providers) for their context. To this day, we can count more than a hundred middlewares made available through the catalog, plus many more that are kept private.

Yaegi is an extremely powerful engine whose boundaries go beyond Traefik itself. Given its nature though, Yaegi still requires much work to support the full Go specification and to keep up with every release of the language. This can sometimes result in difficulties in adding very challenging features.

Another extension technology we have been looking at for quite some time is WebAssembly (Wasm). Until today, we were not satisfied with current ABIs (Application Binary Interface) like proxy Wasm because they were closely designed with other technologies in mind (Envoy), making it hard to leverage in our context. With the release of http-wasm providing a standard ABI designed to match the HTTP handler mechanism built-in with Go, the game changed. Thanks to this technology, in less than 3 months, a contributor jumped in and tackled the necessary work.

You read it correctly: you can now leverage Wasm to develop plugins in traefik, removing any existing hurdle and expanding the field of possibilities.

Many thanks go to Jesse Haka for the bigger chunk of the work involved, and José Carlos Chávez for the review effort.

Everything, Everywhere, All-at-Once Observability

With its central place in your architecture, making decisions on your behalf about where to route requests, Traefik made observability one of its core features since v1, supporting many vendors for metrics and tracing analysis.

With the advent of OpenTelemetry, it sounded natural that Traefik would pave the way for adoption and encourage our users to migrate to state-of-the-art toolings. Once again, we can thank Jesse Haka for this contribution.

For Metrics, where Traefik already supports a large panel of vendors with wide adoption (Datadog, Prometheus, and others), we added OpenTelemetry as a cherry on the cake: you can start migrating to a newer stack without compromising your well-established monitoring toolchain.

Tracing was a different story: Traefik had been an early supporter of two standards (OpenSensus and OpenTracing) that have now merged into one. The adoption is already there, and the migration path is well known. As a result, we dropped support for the previous standards in favor of OpenTelemetry.

But OpenTelemetry is not the only trick Traefik 3 has in its hat around observability. For gRPC enthusiasts, we added support for proper health checks. A one-liner that will change the lives of many, at least at work or in the lab. Thank you Jeremy Jacque.

And my own personal favorite, speaking to the geek side of us, we changed the logger system for a zero-allocation (read “better performing”), structure-friendly, rotation-aware logger. This apparent small tweak will make the whole experience of understanding and debugging your system much easier.

Team Player Traefik: Kubernetes & Friends

For those of you Kubernetes heads, chances are you’ve always seen Traefik as your ingress controller of choice, and that you have adopted IngressRoutes as a clean way to get rid of the limitations of the good old (especially old) Ingress specification. This opened the door to many benefits: declarative and more routing options to name two.

Still, Kubernetes is evolving, and Traefik was part of the Gateway API vanguard, supporting the first alphas. It’s no surprise that we’re bringing full support of the Gateway API specification that is now officially out.

The power of Kubernetes also lies in its vendor ecosystem, allowing DevOps to pick the best tools available. In that regard, Traefik has always been a team player, supporting many vendors, third-party orchestrators, observability components, and key-value stores to name a few. Based on popular demand amongst users and customers alike, we are thrilled to share that Traefik v3 comes with better support for Hashicorp’s Consul (for meshing that requires special TLS management) and Nomad (for namespace support). On the latter, we can thank Thomas Harris.

Strengthen Community & Open Source Values

Maybe the best way to end the list is a non-feature that maintainers adopted wholeheartedly: our commitment to make the Traefik v3 experience the best it can be, from using Traefik to actively contributing to the project.

Yes, Traefik has always been open-source and contributions have been at the forefront of each version. Still, despite a strong commitment to sorting each incoming issue and PR, we concurred that maintainers could better document their thoughts around these contributions. Not only for us as a group of maintainers working async but as a testimony to enable contributors for action.

As a consequence, we reinforced our stance to ensure several key points:

Always spend the extra time so each interaction is welcoming, open, and clear
Always prioritize contributions over our own agenda
Always document the ongoing work for easy collaboration
Always create proposals for upcoming features

These small tweaks almost instantly empowered some key contributors who brought two of the biggest features of Traefik v3, namely Wasm support and OpenTelemetry.

We just can’t wait to welcome even more contributors and additional features in the near future.

Next Steps

Like in the previous major release v2, Traefik v3 shows the importance of the community. Every feature that has been discussed today was born from a user asking for it, reporting a bug, or just asking for something “better”. Critical new features have been completed by external contributors, and, to me, this demonstrates a healthy and vibrant open source project.

A release candidate is the beginning of a release process. To make it to GA, we will need your feedback, comments, and ideas to make this landing as smooth as possible. To get involved, you can download Traefik v3 from DockerHub and help us shape it via our OSS Project on GitHub.

Additionally, if you’d like to receive regular updates via email as v3 heads towards GA, sign up here.

Finally, I want to send a huge thank you to all contributors. Your help is invaluable.

See you on GitHub!

Useful Links

Traefik 3.0 RC1 on GitHub & on DockerHub
Traefik Documentation, Website, & GitHub
Our Community Forum

]]>

Enhancing API Observability: Traefik Hub, OpenTelemetry, and the New Era of Data-Driven API Management

Immánuel Fodor — Thu, 07 Dec 2023 20:30:59 GMT

In today's interconnected digital landscape, application programming interfaces (APIs) serve as the invisible architects, shaping the way we interact with technology. They are the unsung heroes behind the scenes, propelling innovation, driving new business models, and seamlessly delivering products and services across diverse channels. As the backbone of the digital era, APIs command a staggering 54% of total requests and are growing twice as fast as web traffic, underscoring their pivotal role in the evolution of technology.

The API market, once valued at $5 billion in 2022, is forecasted to skyrocket to $40 billion by 2030. This meteoric rise is fueled by the fact that two-thirds of companies are either embracing or fully adopting API-first strategies. As the world becomes increasingly connected, APIs are not just a technological necessity but a strategic imperative for businesses looking to thrive in the digital age.

However, with great impact comes the need for visibility and control. Observability has become paramount in the era of modern and distributed systems, as well as for the APIs connecting them. This paradigm shift from monitoring with simple metrics to comprehensive telemetry analysis is vital for today’s issue resolution, performance optimization, and user satisfaction.

Enter OpenTelemetry, a game-changer in the realm of observability.

OpenTelemetry: The Observability Enabler

Designed to streamline telemetry data collection from distributed systems, OpenTelemetry replaces multiple vendor-specific agents, simplifying and unifying dynamic systems telemetry data. It provides a unified standard, ensures consistent instrumentation across various programming languages, and offers enhanced insights into application use, health, and performance. This becomes especially important as organizations deal with the complexities of microservices and cloud-native architectures.

In this blog post, we will delve into the critical role of OpenTelemetry in achieving comprehensive observability and explore how it addresses the challenges posed by modern software architectures and APIs. For context, API gateways play a pivotal role in managing and securing API traffic, together with the need for visibility into APIs’ health, usage, and performance. Whether with issue alerts, event correlation for rapid issue fault-domain isolation, or cross-domains causations and root-cause analysis, OpenTelemetry emerges as a key enabler in ensuring the reliability and performance of API-driven systems.

Understanding OpenTelemetry

OpenTelemetry is an open-source project that emerged in 2019 combining the OpenTracing and OpenCensus projects. It quickly became a game-changer unifying and streamlining the collection of telemetry data from distributed systems across tech stacks. Born out of the necessity for a unified approach governed to date by multiple proprietary approaches, OpenTelemetry provides a standardized, open telemetry framework for comprehensive visibility into health, usage, and performance across infrastructure, services, and applications.

Metric types in OpenTelemetry serve as the bedrock for understanding system performance and behavior:

Counter: Tracks continuously increasing numeric values, ideal for counting occurrences of events.
UpDownCounter: A versatile counter that accommodates both positive and negative changes, offering flexibility in tracking fluctuations.
Gauge: Offers a snapshot of a value at a specific moment, providing insight into the current state of a system.
Histogram: Captures the distribution of observed values over time, offering a nuanced view of data spread and central tendency.

On the other hand, metric labels play a pivotal role in enhancing the granularity and context of metrics. Labels are key-value pairs associated with metric data, allowing for more nuanced and specific insights into system behavior.

Let's consider a practical example in the context of an airline booking system. Imagine you want to monitor the usage of the APIs with a metric called API Request Total (api_requests_total) which can be an instance of the counter metric type.

The following labels could come in handy:

Code: To categorize requests based on the HTTP response code (e.g., "200" for successful requests, "404" for not found, etc.)
Method: To differentiate requests based on the HTTP method used (e.g., "GET," "POST," etc.)
API Name: To specify the name of the API being accessed (e.g., "flight-api")
API Version: To track the version of the API (e.g., "v2")
Token Name: To identify the API key used for authentication (e.g., "my-test-token")

Example metric labels in OpenTelemetry for this use case:

For a specific metric instance, you might have labels like {"code": "200", "method": "GET", "api_name": "flight-api", "api_version_name": "v2", "token_name": "my-test-token"}
Another instance might have labels like {"code": "404", "method": "POST", "api_name": "ticket-api", "api_version_name": "v1.1", "token_name": "prod-app-key"}

Now, when analyzing the example api_requests_total metric, you can gain specific insights thanks to labels:

HTTP Code Analysis: Compare the total count of requests based on HTTP response codes to identify patterns in successful or error-prone API interactions.
HTTP Method Distribution: Understand the distribution of requests based on HTTP methods to optimize API resources accordingly.
API Name and Version: Monitor the usage of different APIs and versions to plan for updates, deprecations, or optimizations.
Token Usage: Track the utilization of different API keys for authentication, identifying potential security concerns or optimizing key management.

At the end of the day, the more metrics and labels you have, the more analysis you can do.

OpenTelemetry Metrics in API Management

OpenTelemetry metrics has great potential to better observe APIs, however, the current adoption seems to lag.

Moreover, even in instances where API management solutions do offer observability features, the availability of regular metrics remains limited. Many solutions rely on user interface (UI)-based interactions, often leading to what can be termed as "ClickOps"—a reliance on manual, point-and-click operations for metric visibility. This approach, while providing some insights, falls short of the automations and cross-domain analysis promised by OpenTelemetry.

In the API management landscape, where the reliability and performance of APIs are mission-critical to the business, the gap in comprehensive observability can hinder proactive issue resolution, optimization, and strategic decision-making.

Do not fly blind!

Use Cases of OpenTelemetry Metrics in API Management

Real-Time Monitoring and Alerting

Leverage OpenTelemetry metrics to establish thresholds for vital parameters such as API response times, error rates, and request counts. This enables real-time monitoring, allowing organizations to promptly identify deviations from expected behavior.

Additionally, integrating OpenTelemetry metrics with GitOps workflows to correlate API performance changes with code deployments facilitates the swift identification of issues stemming from recent changes. This streamlines troubleshooting and ensuring the reliability of API services.

Capacity Planning and Scaling

Utilize OpenTelemetry metrics to gain comprehensive insights into resource utilization, enabling effective capacity planning. Identify bottlenecks, predict resource demands, and optimize infrastructure to meet current and future workload efficiently.

By analyzing historical OpenTelemetry metrics, organizations can forecast trends and plan for future workloads. This proactive approach ensures that API management systems are adequately prepared to handle increased demand, preventing potential performance issues.

Performance Optimization

Dive deep into OpenTelemetry metrics to identify patterns of errors and latency issues. Whether it's pinpointing error-prone API endpoints or analyzing the performance impact of specific client types, OpenTelemetry metrics provide the granularity needed for precise optimization.

OpenTelemetry Metrics in Traefik Hub

Traefik Hub showcases a wealth of OpenTelemetry metrics and labels that redefine how organizations monitor, manage, and optimize their API infrastructure. The expansive range of metrics, totaling over 20, coupled with more than 15 labels, places Traefik Hub at the forefront of empowering users with unparalleled insights.

The four key categories of metrics exposed by Traefik Hub:

Dataflow-related metrics of exposed APIs: These metrics offer insights into the health, performance, and interactions of APIs, allowing organizations to precisely monitor and optimize the flow of data through their infrastructure.
API management object metrics: A holistic view of key objects including but not limited to the number of APIs, users with API access on gateways published by the agent, API keys, and more. This category of metrics allows organizations to track the usage and impact of API-related objects, enabling efficient management and strategic decision-making.
Licensing metrics for usage tracking: This category enables organizations to track their own usage against the object counts included in the license. This functionality empowers users to manage licensing efficiently, ensuring compliance while optimizing the utilization of available resources.
Ingress-level metrics inherited from Traefik Proxy: Building upon the capabilities of Traefik Proxy, Traefik Hub inherits ingress-level metrics that provide a comprehensive understanding of network traffic patterns and behavior.

We recently enhanced our OpenTelemetry metrics by adding support for API versions, so you can stay on top of your API version proliferation. Learn more here.

Your APIs, Your Metrics, Your Tools

We don’t believe in vendor lock-in. In the quest to embrace a flexible, interoperable observability landscape, OpenTelemetry emerges as a beacon of liberation. Leveraging the rich ecosystem around OpenTelemetry not only ensures adaptability but also opens up a realm of possibilities for metric manipulation, integration, and visualization.

The OpenTelemetry Collector stands as a versatile tool in the arsenal of observability enthusiasts. It goes beyond collection, offering the capability to manipulate metrics dynamically. Through transformations, conversions, filtering, and enrichment, organizations can tailor their telemetry data to specific needs. This flexibility allows for seamless adaptation to evolving requirements and diverse use cases.

Image Source: https://opentelemetry.io/docs/collector

The latest release candidate of Prometheus marks a significant stride towards embracing open standards. With native OpenTelemetry metrics ingestion support, Prometheus becomes a part of the larger OpenTelemetry ecosystem. This integration not only simplifies the process of incorporating OpenTelemetry metrics into existing setups but also lays the foundation for a more unified and standardized observability approach usable by anyone since open-sourced.

To truly liberate observability, visualization tools play a crucial role. For example, users can leverage the full spectrum of Grafana's battle-tested features to visualize, analyze, and get insights from their APIs metrics data. Additionally, the embrace of open standards by 3rd party tools ensures that organizations are not tethered to proprietary solutions, fostering a vibrant ecosystem of choice and flexibility to cater to multiple teams and organizations needs.

Grafana Dashboards Example and Other Resources

We’ve made Grafana dashboard examples that you can extend public, delivering insights with a high-level API overview, a detailed per-API, or per user-specific drill-downs.

All API Overview: Get a comprehensive overview of your entire API landscape, including traffic, response times, error rates, and more. Monitor the overall health and performance of all your APIs, enabling swift identification of global trends and anomalies and where to look deeper.

Comprehensive overview of API traffic, response times, error rates, etc.

Per-API View: Dive deep into the specifics of each API, analyzing traffic patterns, latency, and error rates for targeted understanding and optimizations, with full visibility into API versions.

Per-API view of traffic patterns, latency, and error rates.

Per-User/Per-Token Drill-Down: Track the usage patterns of individual users or API tokens, gaining granular visibility into their interactions with your APIs. Analyze the usage and health of specific API consumers on API requests and responses, ensuring optimal security and resource allocation.

Per User/Per Token drill-down into API interactions.

Kick start your observability journey with our step-by-step tutorial, incorporating the Grafana sample dashboards and our static CRD analyzer for early issue detection, embracing a GitOps approach to modernize API operations. And for in-depth information, see our documentation.

Wrapping It Up

Metrics are a key pillar of modern API observability. OpenTelemetry, thanks to its standardized approach, stands as a game-changer, providing invaluable visibility into API health, usage, and performance. Looking ahead, Traefik Hub is committed to extending OpenTelemetry to traces and logs, promising even more complete observability.

]]>

API Versioning with Traefik Hub: Smooth Transitions, Seamless Innovation

Immánuel Fodor — Mon, 20 Nov 2023 16:13:03 GMT

API versioning is a fundamental practice in software development that involves the creation and maintenance of distinct versions for an application programming interface (API). The primary objective is to safeguard the functionality of applications dependent on the API from disruptions caused by changes or updates.

This practice is paramount for preserving backward compatibility, ensuring that applications using older API versions remain functional. Meanwhile, it facilitates the seamless integration with newer clients, allowing them to leverage the enhanced features of the updated API.

However, API versioning is not just a technical concern; it has direct implications for the success and growth of your business. It helps you maintain developer satisfaction, support diverse user needs, innovate, and manage risks effectively. By providing a reliable and consistent experience for API consumers, you can build a loyal user base, foster a thriving ecosystem, and ultimately drive business growth and revenue.

Before we dive into how Traefik Hub improves API versioning, let’s first take a look at why it’s important for both API producers and consumers.

The Importance of API Versioning

For API Producers

In the fast-paced world of API development, the challenge lies in innovating with velocity while avoiding disruptive changes. For API producers, the absence of a common language among different teams discussing changes can result in a lack of context, leading to errors as interpretations vary. Accelerating the development process may intensify these errors, while slowing down compromises the speed of launching new features. In this regard, API versioning is a crucial solution to strike a balance between innovation and resilience.

For API Consumers

On the consumer side, API versioning addresses the challenges of keeping pace with releases and changes that can lead to integration issues and potential disruptions. With a well-defined versioning strategy, API consumers are more informed and can adapt to new releases at their own pace. This not only keeps existing users satisfied but also attracts new users who seek a reliable and future-proof API.

Ultimately, API versioning acts as a form of insurance for both producers and consumers. It mitigates business risks by providing a controlled mechanism for introducing changes, minimizing the chances of costly outages and revenue loss. Additionally, adhering to versioning practices helps ensure compliance with legal and regulatory requirements, safeguarding the business against potential legal challenges and associated penalties.

How Traefik Hub Solves API Versioning Challenges

Traefik Hub transforms the API versioning landscape by providing a unified solution that caters to both API producers and consumers. For API producers, Traefik Hub offers a common actionable language, streamlining the understanding, expression, and management of change through innovative approaches like CRDs—a.k.a., Custom Resource Definitions—and GitOps. This empowers developer teamwork at scale and speed while minimizing risks of breaking changes.

On the other side of the spectrum, Traefik Hub prioritizes the needs of API consumers by emphasizing ease of use and clarity. The platform introduces API Version CRDs coupled with internal semantic versioning and external flexible version matchers, offering consumers a straightforward way to comprehend and interact with APIs. This not only enhances the developer experience but also ensures that changes are clearly communicated and can be adopted without friction. Traefik Hub’s API Developer Portals expose all API versions clearly, promoting efficient information flow between API producers and consumers.

In order to facilitate efficient communication and collaboration, CRDs are used to standardize changes. This common language ensures that everyone involved understands the modifications, reducing the risk of errors and misinterpretations. As a result, teams can move faster without sacrificing the quality and stability of the API. This aligns with the need for speed without compromising accuracy, enabling a smoother development process.

API Versioning Your Way

We believe in letting developers seize control, making their own choices. We don't impose a rigid versioning structure; we empower you to define the version matchers that suit your unique needs—also ensuring that versioned and unversioned APIs can coexist harmoniously.

Traefik Hub offers four API versioning methods, giving you the freedom and flexibility to choose the best approach for your particular situation:

Path-based (/v3): The traditional route for versioning, with a clear, user-friendly path to your API versions.
Query-based (?v=3): A versatile option that embeds the version directly into the API request, simplifying integration.
Media type-based (Accept/Content headers): This approach leverages headers, offering precise control over content negotiation and version selection.
Header-based (any custom header): The ultimate in customization, letting you set up your own headers to manage API versions.

To provide the best possible product, we looked around on the market, and realized most other API management tools offer only a subset of these methods or even stick to just one, limiting you to follow the path they chose.

Mix and Match Versioning Methods

For Traefik Hub, it's not just about having multiple options. You can combine them in ways that make sense for your project. Traefik Hub provides the unique capability to mix and match these versioning methods with AND/OR relations, giving you the power to craft the perfect strategy to meet your specific needs.

Tired of using numbers for publishing versions? Use calendar dates, cheese names, or anything you like. We don’t limit you; it’s your API, it’s your API governance, and your versioning convention.

Structured Flexibility

Though the offered freedom is great, it shouldn’t sacrifice structure and maintainability. Traefik Hub uses semantic versioning internally to keep track of the version changes and to allow you to mark what is the current API version, what are the older ones, and what are future releases.

This flexibility in interacting with your APIs on the request level paired with the structured internal representation allows you, your team, and your clients to always stay on top of the API version proliferation.

And in case you want to expose an always up-to-date version or a fallback version, a catch-all route can be marked for a versioned API, which still allows unversioned access to the underlying service. It can be beneficial to test the latest functionality or to provide a fallback version to clients that access the API without any version matcher (i.e., access a version without explicitly specifying the version anywhere in the request).

Version Control for Headers

Finally, Traefik Hub’s API and API Version CRDs allow you to seamlessly alter regular and security headers on both APIs and API versions. Add, remove, and manipulate request and response headers on a per-version basis to fortify your API infrastructure.

Use Case: Traefik Airlines

Let’s take a look at Traefik Hub’s API versioning capabilities in action. Traefik Airlines, our fictitious company, is a modern airline company that offers a variety of digital services to its customers and partners. To meet the evolving needs while maintaining business stability, Traefik Airlines must implement a robust API versioning strategy. Let's explore how this can be achieved by using Traefik Hub API management.

Traefik Airlines provides a RESTful API for customers to book flights and manage reservations. Customers can retrieve real-time flight information such as schedules, delays, and gate changes. The API offers mobile check-in, boarding pass retrieval, and gate information. Customers can also interact with an API that manages loyalty points and rewards.

Additionally, third-party travel agencies integrate with Traefik Airlines to offer flight booking services to their customers. They also need to maintain integration with airport systems for baggage handling, gate allocation, and passenger tracking. Collaboration with payment providers is with the utmost importance to process transactions securely.

As we can see, there are many clients, many APIs and many developers supporting the company’s digital infrastructure. The demanding environment screams for using API versioning to satisfy all needs, provide business continuity and let the airline expand at the same time, deploying new API releases on a regular basis.

Flexible API Versioning with CRDs

Let’s say, for example, that Traefik Airlines has already deployed Traefik Hub to their Kubernetes clusters running their microservices, and they are now ready to define APIs and API versions.

Let’s pick the Customer API as an example.

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: customer-api
  namespace: apps
  labels:
    area: customers
    module: crm
spec:
  pathPrefix: "/customers"
  currentVersion: customer-api-version-one

---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: customer-api-version-one
  namespace: apps
spec:
  apiName: customer-api
  release: 1.0.0
  title: "v1 with URI path"
  routes:
    - pathPrefix: /v1
  stripPathPrefix: true
  service:
    name: customer-app-v1
    port:
      number: 3000
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000

Here they define the first version of the Customer API.

The API CRD is labeled for later reference in an API developer portal providing documentation, API access policies to allow users API consumption, API gateways to publish it to the defined audience, API collections to create virtual API groups, or API rate limit policies to secure the backends from rouge clients or malicious actors. The CRD also defines a path prefix for the whole API containing all versions, and references the current version.

The API Version CRD defines the first version of the API, internally versioned as 1.0.0 and exposed as /customers/v1 in the URI path, from which the version is stripped when passing the request to the microservice exposing the API. Here, they could have used any text, date, or even a cheese name to publish the API. If they completely omitted the routes block, the version would have become a catch-all route, accepting any unversioned traffic. The service also exposes an OpenAPI specification which will be attached to the API version on the developer portal.

Let’s say they would have opted for another API versioning method. How could they implement it with the provided flexibility but still maintaining a clear structure?

routes:
    - queryParams:
        my-version-param: "1"

In this case the API version is exposed at /customers?my-version-param=1. Need to expose a version for another language, but the two should be applied at the same time?

routes:
    - queryParams:
        my-version-param: "1"
        lang: "hu"

Now the version is available at /customers?my-version-param=1&lang=hu. Add as many parameters with any value as you like.

Need to expose the same version with different methods? We’ve got you:

routes:
    - queryParams:
        v: "1"
        lang: "hu"
    - headers:
        v: "1"
        lang: "hu"
    - headers:
        Content: "application/vnd.example.v1+json"
        lang: "hu"
    - pathPrefix: /v1
      queryParams:
        lang: "hu"

Now the version is available at the following ways:

Using query params at /customers?v=1&lang=hu
Using the custom headers v: 1 and lang: hu at /customers
Using media-type type versioning with extending the standard content negotiating header Content: "application/vnd.example.v1+json" and a custom lang: "hu" at /customers
Using the path prefix and a query parameter at /customers/v1?lang=hu

You can mix and match all these route options in an API version, both the CRD and the web user interface displays it in a structured, easy-to-digest manner.

In case Traefik Airlines need to publish the 2nd version of the API, they deploy the new supporting service with the corresponding new OpenAPI specification, reference it in a new API Version CRD, and update the current version in the API CRD. And that’s it, the new version is published. The main changes to the first version are emphasized below:

---
apiVersion: hub.traefik.io/v1alpha1
kind: API
metadata:
  name: customer-api
  namespace: apps
  labels:
    area: customers
    module: crm
spec:
  pathPrefix: "/customers"
  currentVersion: customer-api-version-two   # points to new version
  
---
apiVersion: hub.traefik.io/v1alpha1
kind: APIVersion
metadata:
  name: customer-api-version-two             # new version CRD
  namespace: apps
spec:
  apiName: customer-api
  release: 2.0.0                             # bumped internal version
  title: "v2 with URI path"                  # new title
  routes:
    - pathPrefix: /v2                        # new version matcher
  stripPathPrefix: true
  service:
    name: customer-app-v2                    # points to the new service
    port:
      number: 3000
    openApiSpec:
      path: /openapi.yaml
      port:
        number: 3000

None of the configuration building on top of the API needs to be altered, the API access policies, rate limits, collections, etc. stay the same. Publishing the new version is propagated to the API gateway and the portal transparently. Now clients have a backward-compatible way to adapt to the new version without any service disruption to the existing consumer configuration.

You can find even more information about API versioning in our documentation.

Wrapping It Up

In today's fast-paced software development landscape, API versioning is not just a technical necessity but a strategic imperative for a thriving and sustainable API ecosystem. As software development velocity accelerates, API versions provide stability, ensuring existing integrations remain intact with backward compatibility. While you evolve and improve your APIs, it gives your consumers a predictable environment to adopt innovations at their own pace.

Traefik Hub API management simplifies transitions, minimizes disruptions, and offers a smooth migration path for legacy APIs by extending rather than replacing non-versioned APIs, allowing coexistence. With this internal semantic versioning system and flexibility in versioning methodology, Traefik Hub accommodates diverse client needs and enhances the overall experience for both developers and clients.

]]>

How to Install Traefik via Azure Marketplace

Nikolas Sachos — Mon, 26 Jun 2023 08:27:59 GMT

If you're a developer or DevOps engineer looking to integrate Traefik Proxy and Traefik Enterprise with Azure Kubernetes Service (AKS), you're in the right place! This guide provides a simple and intuitive walkthrough on how to install Traefik Proxy and Traefik Enterprise through the Microsoft Azure Marketplace, granting you access to the many benefits of the Traefik product suite.
Traefik is one of the preferred choices for running an Ingress Controller and API Gateway on Azure. Recommended by Microsoft in their baseline architecture for Azure Kubernetes Service (AKS), Traefik offers dynamic configuration, middleware support, and seamless compatibility with AKS workloads.

Step-by-Step Instructions for Installing Traefik Proxy via Azure Marketplace

Before you get started, ensure that you have the following prerequisites:

An active Azure subscription.
An existing Azure Kubernetes Service (AKS) cluster.
Helm installed on your local machine.
Azure CLI installed on your machine (for cluster access).

After ensuring you meet the necessary prerequisites, you can move forward with the installation process:

1. First, navigate to Microsoft Azure Portal. Use the top search bar, search for Traefik Proxy, and click on the correspondent link under the Marketplace section.

2. Use the top search bar, search for Traefik Proxy, and click on the correspondent link under the Marketplace section.

3. Next, click the Basics tab, and choose your Subscription, Resource group, and Instance details.

4. Click on Cluster Details, and choose your AKS Cluster Name.

5. In the Cluster Extension Details tab, fill in your Cluster extension resource name by filling in a numeric 6 to a 30-character-long alphanumeric value.

Note: In the Cluster Extension Details tab, fill in your Cluster extension resource name by filling in a numeric 6 to a 30-character-long alphanumeric value.

6. Finally, on the "Review + create tab", click the Create button at the bottom left of the page.

7. After completing the installation process, the following confirmation page will be displayed showing that the deployment is complete and one Traefik Proxy instance is running in your cluster.

To verify that the installation runs smoothly, we can list the pods in the Traefik namespace by running the command:

kubectl get pods -n traefik

The output verifying the Traefik pod is running as expected should be similar to:

NAME                        	            READY   STATUS	RESTARTS      AGE
traefik-5dbc588874-bfbpv    1/1 	   Running            0      	  5m

In order to get the Loadbalancer IP and verify that our services will be accessible from the outside world, we can list the Traefik services by executing the command:

kubectl get services -n traefik

Your output should show an External IP address similar to:

NAME        TYPE               CLUSTER-IP      EXTERNAL-IP    PORT(S)          AGE
traefik     LoadBalancer   10.0.93.113        20.4.131.79	80:32159/TCP,443:32202/TCP   5m

Your Traefik Proxy deployment is now active and operational. It's ready to accept connections on the Loadbalancer External-IP on ports 80 and 443.

8. Once your Traefik installation is running smoothly, the next step is to set up your routing rules and test them. This can be achieved by using the IngressRoute CRD to define your routes, services, and ports. The code below demonstrates how a Service and an IngressRoute can be applied to a cluster with a single command:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: apps
---
apiVersion: v1
kind: Service
metadata:
  name: my-service-name
  namespace: apps
spec:
  selector:
    app: my-app
  ports:
    - name: http
      protocol: TCP
      port: 3000
      targetPort: 3000
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: myIngressName
  namespace: apps
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: my-service-name
          port: 3000
  tls:
certResolver: letsencrypt

Once you run the command above, the output will be

namespace/apps created
service/my-service-name created
ingressroute.traefik.containo.us/my-ingress created

and it will create a new namespace named apps, create a service in that namespace, and set up the routing rules for incoming traffic to this service.

9. Now you can configure access to the Traefik dashboard that will provide a visual insight into the health, performance, and routing configuration of your Traefik instance. To achieve this, we can use an IngressRoute CRD and define the appropriate metadata, entryPoints, and routes. Create a YAML file named dashboard-ingress.yaml, and paste the following content:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefik
spec:
  entryPoints:
	- web
	- websecure
  routes:
  - kind: Rule
	match: Host(`example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
	services:
	- name: api@internal
  	kind: TraefikService
  tls: {}

Do not forget to replace example.com with your preferred domain name.
Now to create the routing rules, apply the file by executing the command:

kubectl apply -f dashboard-ingress.yaml

To confirm that the dashboard has been deployed, your output should be:

ingressroute.traefik.containo.us/traefik-dashboard created

You can access the dashboard at https://example.com/dashboard/. Below is a screenshot highlighting the various HTTP, TCP, and UDP Routers, Services, and Middlewares running across a single cluster.

An interesting feature of the Dashboard is that when an error occurs in a Pod (referred to as Server within the interface), then automatically, a part of the graph will become red, notifying you about the issue while also giving you the exact error message.

10. With our Dashboard operational, let’s now use the Traefik automated service discovery. This is a built-in feature that dynamically discovers and configures routes to services as they are created (or destroyed) in your environment. We can deploy a simple Go-based webserver application and verify the service discovery by watching the changes live on the Dashboard.
Go ahead, and create a file my-app.yaml, and paste the code below:

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: my-namespace
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: my-namespace
spec:
  selector:
    app: whoami
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: my-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        ports:
        - containerPort: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: my-namespace
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: whoami
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls: {}

And apply the manifest by executing the command:

kubectl apply -f my-app.yaml

Your output should look similar with:

namespace/my-namespace created
middleware.traefik.containo.us/prod-rate-limit created
service/whoami created
deployment.apps/whoami created
ingressroute.traefik.containo.us/prod-route created

The namespace, middleware, service, deployment, and ingress routes are now created, meaning your application is now deployed and ready to be used. After a few seconds, you will notice that your Dashboard interface displays two new Routers, one new Service, and one new Middleware in the HTTP section.

By clicking on the Explore link on the Services box, you will see the new Service name my-namespace-prod-route-(randomid)@kubernetescrd, with Type loadbalancer having one Server (the Nodejs pod) using the kubernetescrd Provider. The randomid is a random string that Traefik produces internally to avoid possible conflicts.

At this point, we have completed all necessary steps to enable our Traefik dashboard and deployed an example application to see its effects displayed live on the screen. This will conclude our installation guide, and from this point on, you may apply any other manifests like you normally would.

Now that we've covered the installation process for Traefik Proxy via Azure Marketplace let's explore why you might want to consider Traefik Enterprise for your AKS deployment. If you're running mission-critical applications in production, we strongly recommend considering Traefik Enterprise. Why? Traefik Enterprise offers a robust suite of features designed specifically to meet the demands of such important environments. It provides an array of advanced security measures, sophisticated traffic management capabilities, high availability, scalability, and comprehensive monitoring and troubleshooting tools.

Why Choose Traefik Enterprise for Your AKS Deployment?

Traefik Enterprise is a robust and versatile API Gateway and Ingress solution designed to cater to the needs of enterprises. Its numerous features and benefits make it an ideal choice for Azure Kubernetes Service (AKS) deployments:

Security: Traefik Enterprise offers security features such as IP Allow List, Access Control protocols support, and Distributed Let’s Encrypt. It separates responsibilities between two planes for enhanced security.
Traffic Management: Traefik Enterprise provides dynamic configuration, service discovery, traffic mirroring, deployment strategies, health checks, rate limiting, and service mesh capabilities for advanced traffic management.
High Availability: Traefik Enterprise ensures continuous service availability with its fault-tolerant design. It runs natively in cluster mode, storing all cluster data, and works with AKS for seamless application scaling.
Scalability: Traefik Enterprise scales with your applications and supports various deployment methods for easy replication across different environments.
Monitoring and Troubleshooting: It offers a dashboard, proxy health checks, and support for tracing and real-time monitoring tools. Integrated with AKS, it helps understand traffic flow and service dependencies for efficient troubleshooting

Step-by-Step Instructions for Installing Traefik Enterprise via Azure Marketplace

To get started with the installation of Traefik Enterprise in AKS, make sure you first request a Traefik Enterprise trial.

1. Visit Microsoft Azure Portal and type Traefik Enterprise in the top search bar. Then, click on the correspondent link under the Marketplace section.

2. Click on the Basic tab, then choose your Subscription, Resource group, and Instance details.

3. In the Cluster details tab, choose your AKS Cluster name.

4. Next, in Cluster Extension details, choose your Cluster Extension Resource name, and the number of Proxy instances.

5. By clicking on the Review + Create tab, you will see the summary of your Traefik Enterprise installation on AKS. Next, click the Create button.

6. After completing the installation process, the following confirmation page will be displayed showing that the deployment is complete and one Traefik Enterprise instance is running in your cluster.

Now you can list Traefik Enterprise pods in traefikee namespace to verify that they are running, by executing the command:

kubectl get pods -n traefikee

Your output should list all running pods in traefikee namespace:

NAME                         	          READY   STATUS    RESTARTS    AGE
default-controller-0        	            1/1 	 Running            0	       10m
default-plugin-registry-0   	            1/1 	 Running   	   0      	       10m
default-proxy-d4c5549fc-7sl4f       1/1  	 Running   	   0      	       10m
default-proxy-d4c5549fc-ftgnj        1/1 	 Running   	   0      	       10m

Traefik’s services can be listed by executing the command:

kubectl get services -n traefikee

Your output should be very identical with the one below:

NAME                                         TYPE        CLUSTER-IP   EXTERNAL-IP  PORT(S) AGE
default-ctrl-svc                       ClusterIP        None             <none>     	<none>   20m
default-plugin-registry-svc   ClusterIP  	 10.0.52.229  <none>          443/TCP 20m
default-proxy-svc         	LoadBalancer 10.0.15.250  20.4.133.168 80:30876/TCP,443:31247/TCP   20m

The service default-proxy-svc was assigned an external IP by the provisioned Azure load balancer, and now can accept traffic in ports 80 and 443.

7. Traefik Enterprise is now running, and the next step is to configure it. The configuration can be done using teectl, a comprehensive command-line tool designed to interact with Traefik Enterprise. Follow the installation instructions in order to generate the controller configuration. This configuration is essential for Traefik Enterprise to handle incoming traffic, enable the Dashboard, and define certificate resolvers. The series of commands described below can be used to generate the necessary credentials to interact with your Kubernetes cluster using Traefik's teectl tool, and to configure teectl to use those credentials.
Continue by executing the following commands:

kubectl exec -n traefikee default-controller-0 -- /traefikee generate credentials --kubernetes.kubeconfig=/home/username/.kube/config --cluster=default > tee_config.yaml
teectl cluster import --file=tee_config.yaml
teectl cluster use --name=default

Remember to replace username with your actual user.
The output of the commands above will display your default pod that teectl interacts with, the cluster name, and the context namebe:

Defaulted container "default-controller" out of: default-controller, wait-dns (init)
Cluster "default" imported
Cluster context has been set to "default"

8. Now you must generate the static configuration. This is crucial because it sets up the foundational settings for your Traefik instance. The configuration will include the Providers, entryPoints, Dashboard enablement, and certificatesResolvers and is essential because it provides the foundational settings for your Traefik instance. The static configuration contains settings that define how Traefik will run and interact with your environment, and once set; these settings are not dynamically reloadable.

Create a file named static.yaml, and paste the following content:

providers:
  kubernetesCRD: {}

entryPoints:
  web:
    address: ":80"
  websecure: 
    address: ":443"
  internal:
    address: ":8888"

api:
  dashboard: true

certificatesResolvers:
  letsencrypt:
    acme:
      email: your_email_address
      tlsChallenge: {}

Replace your_email_address with your email address.

Now apply the static configuration by running:

teectl apply –file=./static.yaml

The command above does not produce an output, and at this step, you should have Traefik Enterprise correctly configured. The next step will involve creating the Ingress route for the Dashboard.
Create a file called dashboard-ingress.yaml, and the content:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefikee
spec:
  entryPoints:
	- websecure
  routes:
  - kind: Rule
	match: Host(`example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
	services:
	- name: api@internal
  	kind: TraefikService
  tls:
	certResolver: letsencrypt

Replace example.com with your domain name.
10. Create your routing rules by applying the file dashboard-ingress.yaml.

kubectl apply -f dashboard-ingress.yaml

The output below displays the created ingressroute resource:

ingressroute.traefik.containo.us/traefik-dashboard created

This means that your routing rules to the api@internal service are now applied.
Visit the URL https://example.com/dashboard/. You will be able to access the dashboard.

Your Traefik’s Enterprise dashboard is now fully functioning!

At this point, you can begin deploying your applications as you normally would. For instance, you can apply the following manifest that will create a namespace, service, deployment, and an IngressRoute, as seen below:

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: my-namespace
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: my-namespace
spec:
  selector:
    app: whoami
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: my-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        ports:
        - containerPort: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: my-namespace
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: whoami
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls:
    certResolver: letsencrypt

Replace example.com with the domain name of your choice. In the example code above, we used the image traefik/whoami, which is a tiny Go webserver that prints OS information and HTTP requests to output. So When you visit https://your-domain.com, you will see such information displayed on your screen.
This last step has our deployment guide completed.

Enhancing Your Setup for API Management on Top of Traefik Proxy or Traefik Enterprise

Once you have deployed Traefik Proxy or Traefik Enterprise on AKS, you might consider enhancing your setup with Traefik Hub. Traefik Hub is a Kubernetes-native API management platform that integrates seamlessly with Traefik Proxy and Traefik Enterprise, offering a streamlined solution for publishing, securing, and managing APIs.To get started with Traefik Hub, you can request a free trial here. You'll be able to deploy the Traefik Hub Agent in your Kubernetes cluster, following the guidelines provided in the official documentation. Traefik Hub offers a dashboard and onboarding steps making it easy to get started as demonstrated below.

Wrapping it Up

Traefik Proxy, Traefik Enterprise, and Traefik Hub provide enhanced connectivity and API management for AKS environments. Offering advanced security, traffic management, scalability, and monitoring, they help developers and enterprises streamline their application publishing and security. By following our installation guides and Traefik Hub introduction, you can easily tap into these additional features to optimize your AKS setup.

Interested in getting started with Traefik in AKS? Schedule a personalized demo of Traefik Enterprise with us. We'll guide you through its main features and demonstrate how it can fit into your specific AKS use case.

]]>

Harnessing the Power of Kubernetes-native API Management with Traefik Hub and Rancher Prime

Erwan Paccard — Fri, 16 Jun 2023 13:32:34 GMT

As digital transformation accelerates at an unprecedented pace, several design patterns and architectural choices have emerged to keep up with the rapid change. Microservices, multi-cloud deployments, Kubernetes, and APIs are not just buzzwords; they are critical components of the contemporary digital landscape, driving business agility, application agility, digitization, and B2B value creation.

The surge in digital services has led to a scenario where API requests comprise over 80% of all traffic types and continue to grow, with 90% of digital services being built using public & internal API-delivered services. APIs, therefore, form a critical component of digital transformation, and participating in the API economy has become a top priority for most organizations.

Amid this backdrop, today we’re going to explore the joint value proposition of Traefik Hub and Rancher Prime, two powerful tools that, when combined, can supercharge your digital transformation journey.

Embracing Kubernetes-native API Management

Traefik Hub is the industry’s first Kubernetes-native API management solution. It extends the Kubernetes capabilities with API Management through CRDs, seamlessly integrates with its API, enabling automatic discovery of services, and delivers Kubernetes native ways to publish and manage APIs. This provides a significant operational advantage over proprietary solutions that are not Kubernetes-native. Traefik Hub Kubernetes-native approach based solely on CRDs unlocks operational excellence with scalable and highly available deployments, a full GitOps compliance for powerful automation with tools like Helm, Terraform, and ArgoCD for repeatability and auditability. It also eliminates vendor lock-in by supporting any Kubernetes distribution and ingress controller and offers a lightweight, portable, and efficient solution with separate control and data planes.

Ensuring High Availability & Security

Coupling Traefik Hub with Rancher Prime, the leading Kubernetes management platform ensures high availability with built-in HA with replicas. This redundancy is a fundamental component of high availability, as it ensures that even if one node goes down, the service remains accessible. The Traefik Hub platform, as part of its core offering, allows for easy and automatic configuration of service replicas. Therefore, if one instance faces an issue, another can seamlessly take over, reducing downtime and maintaining service reliability.

Traefik Hub and Rancher Prime also help users extend security with seamless third-party integrations. Rancher Prime provides comprehensive security features including RBAC, SSO, AD/LDAP integrations, and secrets management. Combined with Traefik Hub’s built-in high availability and security features, you get a secure, robust, and reliable application deployment pipeline.

CI/CD Compliance

The combination of Traefik Hub and Rancher Prime provides a powerful toolset for those embracing the GitOps model and continuous integration/continuous deployment (CI/CD) methodologies. They offer a streamlined workflow that seamlessly integrates with your CI/CD pipeline, making it an ideal solution for DevOps teams. Rancher Prime provides built-in CI/CD pipelines that can be configured to trigger deployments based on changes to a Git repository. Rancher Prime’s multi-cluster management capabilities allow CI/CD pipelines to deploy applications across multiple Kubernetes clusters. This feature is particularly beneficial in a multi-cloud or hybrid-cloud environment. With Traefik Hub providing API management across these clusters, you get a unified, efficient, and scalable deployment process.

Custom Resource Definitions (CRDs) in Kubernetes provide a way to define custom resources with your own API types, extending the Kubernetes API. Traefik Hub, being a Kubernetes-native solution, treats CRDs as first-class citizens. This means that it provides native support for CRDs, allowing developers to create and manage custom resources easily.

Deploy Anywhere, with Any Ingress Controller

The Traefik Hub and Rancher Prime solution supports any deployment environment and any ingress controller. This flexibility is crucial for organizations looking to deploy APIs across diverse environments, including on-premises, hybrid cloud, multi-cloud, and edge deployments.

Modern, lightweight, and open architecture

Traefik Hub and Rancher Prime boast a modern, lightweight, and open architecture. They unlock and unleash the full potential of Kubernetes for API Management. They are lightweight, so they won’t weigh down your systems, but they are feature-packed to keep your operations running smoothly. Their open architecture speaks the language of Kubernetes, making integration a breeze. And because they are modern, they are constantly evolving to meet the ever-changing demands of the DevOps world.

Embracing Edge Computing

Edge computing, another significant trend, is fueling new edge applications and creating infinite possibilities. The broad spectrum of edge use cases can be divided into the Near Edge, Far Edge, and Tiny Edge (autonomous vehicles, industrial robotics, drones/UAV, smart grid), with each having specific requirements in terms of devices and proximity to data centers or users.

As applications move to the edge, so do APIs. Specialized Edge APIs are emerging with the same drivers around business value and time. Traefik Hub, Rancher Prime, RKE2, and K3s are well-positioned to cater to these needs due to their flexible deployment capabilities, GitOps compliance, and support for any ingress controller.

Architecture diagram Rancher Prime + Traefik Hub

In conclusion, Traefik Hub and Rancher Prime offer a compelling value proposition for organizations looking to streamline their API management and Kubernetes operations. By embracing these tools, you can not only keep pace with the accelerating digital transformation but also lead, compete, and thrive in the digital economy. The future of API management is Kubernetes-native, flexible, and secure, with full CI/CD compliance. Embrace this future with Traefik Hub and Rancher Prime, and unlock the full potential of your digital transformation journey.

Traefik Labs is excited to announce our sponsorship of SUSECON 2023. We invite everyone to join us at the conference, in person in Munich or online, from June 20th to June 22nd. From Traefik Labs, Sudeep Goswami, Chief Revenue Officer, and Maytham Alfouadi, Solutions Architect will be presenting "Kubernetes-native API management at the K3s edge: The GitOps way" on Thursday, June 22 at 9 am CEST.

Join us to learn about the latest developments in open source technology and API management strategies. We look forward to seeing you there!

]]>

Kubernetes CRDs in Traefik: Overcoming the Limitations of Kubernetes Standardized Objects with Custom Resource Definitions

Nikolas Sachos — Tue, 06 Jun 2023 14:48:46 GMT

While Kubernetes provides built-in API objects and resources, the Kubernetes API can be extended to any object you like using Custom Resource Definitions (CRDs). This allows you to capture the subtleties of your use case and tap into the flexibility Kubernetes offers.

As container orchestration continues to gain traction, Kubernetes has become a (if not the) standard for Developers and DevOps engineers: they’re now used to declaring the desired state of their system instead of manually deploying software all over the world.

But how does one declare this desired state? And what if Kubernetes doesn’t fully capture the subtleties of your use case and you need to be a bit more descriptive than what it is aware of?

The goal of this post is to answer this very question. In the context of Ingress Management (how your cluster accepts and reacts to incoming requests), we’ll walk you through the standardized objects, use them with Traefik, understand their limitations, and show you how to overcome them. (Spoiler, we’ll use Traefik CRDs.)

Understanding Kubernetes Objects

In a world where APIs are more and more prevalent, it’s no wonder that Kubernetes introduced a way to describe how your cluster must handle incoming requests: the Ingress object.

But if you look at the bigger picture, other objects are involved in the process, let’s see which ones:

Ingress: As we just said, Ingresses manage external access to services within a cluster. You’ll describe HTTP/HTTPS request patterns and which services are responsible for handling these.
Pods: The smallest Kubernetes unit, and most of the time a single container (could be more). You can see a Pod as a running instance (replica) of your application.
Services: Because Kubernetes decides where it deploys a Pod, and because it might kill and respawn a Pod based on the health, workload, and lifecycle of the whole system, each Pod has its own ephemeral ID. Services are a construct that provides a stable logical name for accessing the Pods.
Deployments: Deployments are where you tell Kubernetes what containers you want to run in your Pods, and how many replicas you want it to run. There’s of course more to say, but for the sake of our tour, this is enough.
ConfigMaps: ConfigMaps store non-sensitive, configuration-related data as key-value pairs. They allow you to decouple configuration data from container images, making managing and updating your applications easier.
Secrets: Similar to ConfigMaps, Secrets store sensitive data such as passwords, tokens, and keys. They help protect sensitive information by allowing access control and keeping it separate from your container images and application code.

Each object serves a specific purpose and contributes to the overall architecture, but if you have a good understanding of the above concepts, you can already take full advantage of Kubernetes’ powerful features and create robust and scalable applications.

Using Kubernetes Ingress Object with Traefik

So if the Kubernetes Ingress Object describes the external access to your services, what handles it? The answer is the Ingress Controller, and Traefik is one of them (and we like to believe one of the best).

When you run Traefik on your K8S cluster, it reads the Ingress objects to understand the routing you want to achieve, then routes accordingly. It’s as simple as that.

Basic Example: TLS and Two Paths

Basic Example: TLS and Two Paths

Let’s define two different paths (/foo, /bar), routing to two different services (foo-service, bar-service).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: IngressFooBar
  namespace: production
annotations:
   kubernetes.io/ingress.class: traefik
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-tls-secret
  rules:
    - host: example.com
      http:
        paths:
          - path: /foo
            pathType: Exact
            backend:
              service:
                name: foo-service
                port:
                  number: 80
          - path: /bar
            pathType: Exact
            backend:
              service:
                name: bar-service
                port:
                  number: 80

In the above example, you can see the Ingress resource with a specific host (example.com) and two paths. The first path, /foo, routes traffic to the backend service foo-service. The second path /bar routes traffic to the backend service bar-service.

Apart from the obvious, both services are listening on port 80. Additionally, the tls section specifies that TLS should be used for the host example.com and that the TLS certificate and private key are stored in a Secret named example-tls-secret.

To create this secret, you can run the command:

kubectl create secret tls example-tls-secret --cert=path/to/tls.cert --key=path/to/tls.key -n production

When you have simple needs, you need a simple solution, and we saw that Traefik is an excellent fit in that scenario. Traefik is fully compatible with the Ingress specification and will handle any request you can describe using that standard.

Limitations of Kubernetes Ingress

The premise of the Ingress object was to define a standard capable of expressing every routing need the user would face. Then, users would pick their Ingress Controller of choice. Unfortunately (or fortunately), there is more to routing than just paths, and chances are your ingress controller of choice can do (much) more than path-based routing, which is especially true if you’re used to full-featured solutions like Traefik.

These limitations include:

Limited Routing Rules: As we said, the Ingress object, while offering basic routing rules based on the request host and path, falls short in providing advanced routing capabilities such as header-based routing or traffic splitting.
HTTP Only: The primary function of the Ingress object is to route HTTP and HTTPS traffic. Support for other protocols, such as TCP and UDP, simply doesn’t exist.
Limited to routing: the Ingress object does not include features that play along well with your ingress controller, like tracing, logging, rate limiting, circuit breakers, or many other use cases.

Annotations to the rescue

To support these extra features, Ingress Controller vendors started to leverage the annotation system in Kubernetes, where one can add any text information and attach it to the Ingress object. Unfortunately, no standard has emerged for these annotations, and these annotations themselves are limited:

No validation: Because the annotations are “free text”, there is no safety net for users so that they can avoid typos or syntax errors while describing advanced features they want to attach.
No standard: Each vendor has their own set of features and options, defeating the purpose of having a standard in the first place.
Too broad: Because annotations are attached to the Ingress itself, it’s difficult to describe a feature on a finer-grained level (e.g., the path or the service).

Introducing Traefik Kubernetes CRD: A better alternative

So you chose Traefik for a reason, and that reason is you want to leverage every feature it has to offer.

Since annotations already force you to add specific configuration instructions that break the premise of compatibility with other ingress controllers and don’t overcome every limitation, you need a better alternative.

This is where CRDs (Custom Resource Definitions) come into play.

Not only Kubernetes has built-in objects, it also allows you to define your own, extending its description capabilities.

Traefik Kubernetes CRD (Custom Resource Definition) is a powerful tool that overcomes the Ingress specification limitation and allows for more options on top of providing a clear structure, for example:

IngressRoute: This is the extended equivalent of the Ingress object. It adds support for various options, such as load balancing algorithms.
Middleware: Middleware are a concept that you can mix and match for everything that goes beyond routing, whether it’s Access Control, header updates, circuit breakers, path manipulation, error control, redirection, and many others.
TraefikService: Provides an abstraction for HTTP load balancing and mirroring. It allows you to define how traffic should be distributed among multiple services in your cluster.
IngressRouteTCP & MiddlewareTCP: As their name implies, they allow you to define TCP routes and tweak the request in the process.
IngressRouteUDP: Once again, adding support for a different protocol, this time UDP.
TLSOptions: To fine-tune TLS connection parameters such as the minimum TLS version and the cipher suites that should be used.
ServersTransport: To tweak communication between Traefik and the services.

To read more details about these CRDs, visit Traefik CRDs documentation.

Example: Achieving Rate Limiting with Traefik CRDs

Now that we know what Traefik CRDs are, let’s leverage them to add rate limiting to our example, setting up a rate limit of 30 average requests per second with a burst of 50 for a nodejs service, accessible through the example.com host.

In the process, we’ll also ask Traefik to automatically handle certificate generation using Let’s Encrypt, that is is a free, automated, and open Certificate Authority (CA) that provides digital certificates required for HTTPS.

apiVersion: v1
kind: Namespace
metadata:
  name: nodejs
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: prod-rate-limit
  namespace: nodejs
spec:
  rateLimit:
    average: 30
    burst: 50
---
apiVersion: v1
kind: Service
metadata:
  name: nodejs
  namespace: nodejs
spec:
  selector:
    app: nodejs
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-deployment
  namespace: nodejs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodejs
  template:
    metadata:
      labels:
        app: nodejs
    spec:
      containers:
      - name: nodejs
        image: admintuts/expressjs-hello-world:latest
        ports:
        - containerPort: 3000
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prod-route
  namespace: nodejs
spec:
  entryPoints:
    - web
    - websecure
  routes:
    - kind: Rule
      match: Host(`example.com`)
      services:
        - name: nodejs
          port: 80
      middlewares:
        - name: prod-rate-limit
  tls:
    certResolver: letsencrypt

To apply this configuration, save the above content into a YAML in a file, say nodejs-rate-limit.yaml, and apply it to your Kubernetes cluster with the following command:

kubectl apply -f nodejs-rate-limit.yaml

The command output will be:

middleware.traefik.containo.us/prod-rate-limit created
service/nodejs created
deployment.apps/nodejs-deployment created
ingressroute.traefik.containo.us/prod-route created

This will create the resources in your cluster, and a nodejs service will be subject to the rate limit rules defined by the prod-rate-limit middleware. Now you can test your setup, and you should observe that any requests beyond that limit are appropriately throttled.

Bonus: Testing Traefik Rate Limiting

To validate the rate limiting abilities of the configuration we applied above, we can use the hey tool, a popular HTTP benchmarking and load-testing utility. With hey, we can simulate multiple concurrent requests and observe how Traefik enforces the rate limits we have set. Assuming you have hey installed, let's run a test with the following command:

hey -n 100 -c 10 -q 10 http://example.com

Command output:

[...]
Status code distribution:
 [200]    75 responses
 [429]    25 responses

As you can see above, from our 100 requests made to the host, 25 requests were throttled, effectively proving that our rate limiting strategy is working as expected.

Let's break down the parameters used in the hey command:

-n 100: Specifies the total number of requests to be made, in this case, 100.
-c 10: Sets the number of concurrent connections, here we use 10 to simulate concurrent users.
-q 10: Defines the maximum number of requests to be made per second, limiting the request rate to 10 per second.
http://example.com: Specifies the URL endpoint we want to target with our requests.

By leveraging a Traefik Custom Resource Definition with the rateLimit configuration option and tools like hey, you can confidently design, test, and fine-tune your rate limiting strategy to achieve the desired level of control and protection for your applications.

Benefits of Using Traefik CRDs

Traefik CRDs deliver a host of benefits for your Ingress and API management. Here are a few reasons to incorporate them into your workflow:

Flexibility: With Traefik CRDs, you get unparalleled flexibility and fine-grained control over your Ingress traffic. Leverage advanced routing, traffic management, and security features tailored to your bespoke application requirements.
Compatibility: Traefik CRDs work seamlessly with Traefik’s ecosystem of tools and features, making them an ideal fit for your existing Traefik setup.
Advanced Use Cases: Traefik CRDs enable you to implement advanced use cases, such as Canary Deployments, A/B testing, and rate limiting that the standard Kubernetes Ingress object may not support. This allows you to implement complex traffic management strategies to ensure smooth application operation.
Ease of Use: It's easy to define, configure, and manage with standard Kubernetes manifest files, following the same declarative approach as other Kubernetes resources.

Wrapping It Up

In this article, we delved into Kubernetes CRDs in Traefik. We discussed Kubernetes Ingress limitations and how Traefik CRDs overcome these, providing a practical rate limiting example. We highlighted the benefits of Traefik CRDs, including their flexibility and compatibility. Whether you're a software developer or a DevOps engineer, we encourage you to explore the power of Traefik CRDs in your Kubernetes deployments for a more robust and scalable application infrastructure. Remember, the Traefik community is always here to support you.

In addition, CRDs are not only used in Traefik Proxy. Our recent product Traefik Hub, a Kubernetes-native API Management solution that simplifies API publishing, security, and management, also leverages native Kubernetes constructs, like CRDs, labels, and selectors. It is also fully GitOps compliant providing quick time to value and increased productivity for users. In an upcoming article, we will cover how to use CRDs in Traefik Hub to simplify API Management. In the meantime, you can start your Traefik Hub trial here.

]]>

Ushering in a New Generation of API Management With Traefik Hub

Erwan Paccard — Wed, 17 May 2023 15:39:24 GMT

Today we are pleased to announce the general availability of Traefik Hub API management, a modern Kubernetes native solution for publishing, securing, and managing APIs, with industry-first support for major third-party ingress controllers including NGINX.

The ability to design, develop, and launch APIs has become a strategic imperative for organizations focused on cloud native, containerization, and digital transformation initiatives. APIs are the connective tissue that powers nearly every digital product and service today, representing ~83% of total requests on the Internet, and are growing 2X faster than all other Web traffic. Yet, existing API management solutions are struggling to provide a cloud native experience and quick time to value, with their users facing steep learning curves, growing operational complexity, and a lack of deployment flexibility.

API management, the origin story

Before we dive into the details of the announcement, we wanted to share a bit of the back story on how Traefik Hub came to be. Traefik Proxy, our first project, was born in 2015 to tackle the growing complexity of adopting cloud native technologies. With a maniacal focus towards simplification and automation, the Traefik Proxy open source project has achieved over 3 billion downloads and 43k GitHub stars to date. From advanced home lab users to Fortune 100 enterprise architects, many have been using Traefik Proxy as their unified ingress and API gateway, helping it reach and maintain a leadership position in this category for several years in a row, based on rankings published on Ossinsight.

Given our access to such a large and globally distributed install base of Traefik users, we have routinely heard and observed first-hand the key challenges associated with API management.

Large monolithic systems: the API management ecosystem is rife with large and proprietary monolithic systems that imbue an all-or-nothing mindset. This is in direct conflict with modern cloud native principles that promote choice, modularity, and lightweight footprints.
Misalignment with DevOps operating model: existing API management solutions are designed around UI point-and-click workflows rather than Kubernetes-native scripting and automation, thereby causing friction and inefficiencies when addressing the needs of organizations adopting CI/CD and GitOps-centric methodologies.
Limited deployment flexibility: existing API management solutions are tightly coupled to a limited set of deployment environments, creating operational constraints for organizations adopting hybrid, multi-cloud and edge strategies.

Through numerous discussions and user surveys, we validated and concluded the dire need for a modern API management solution that caters to the growing Kubernetes and cloud native user community. Given our company’s track record and DNA in solving similar intractable problems, we felt compelled to rise to this new challenge and deliver a best-in-class Kubernetes-native API management solution.

Introducing Traefik Hub, the first Kubernetes-native API Management solution

Traefik Hub is the industry’s first Kubernetes-native API Management solution for publishing, securing, and managing APIs, with support for both Traefik and major third-party ingress controllers including NGINX, HAProxy, Ambassador, and many others.

Traefik Hub has been built from the ground up to deliver outcomes aligned with the following design principles and goals:

1. Freedom of Choice

Traefik Hub offers choice across multiple dimensions: users can choose any leading ingress controller, including Traefik Proxy or third-party solutions like NGINX. Users can also standardize their Traefik Hub installation across multiple deployment environments, including on-prem, managed Kubernetes in multi-cloud, and edge. Furthermore, the Traefik Hub architecture is both open and modular, providing seamless integration capabilities with third-party API design, testing, mocking, and security solutions, while leveraging industry best practices and standards such as OpenTelemetry and OpenAPI specification (OAS)

2. DevOps-first Mindset

Traefik Hub is a modern solution that embodies a DevOps-first mindset. Initial configurations and ongoing changes can be expressed as code and deployed through GitOps and CI/CD friendly workflows.

3. Security-first Mindset

Any discussion of a modern API management system would be incomplete without addressing the notion of APIs becoming a new attack vector and ways to mitigate them. Traefik Hub offers robust security features such as Role Based Access Control (RBAC), and support for modern industry-standard authentication and authorization mechanisms like OAuth 2.0 and JSON Web Tokens (JWT). Furthermore, Traefik Hub’s open and modular architecture offers integration capabilities with leading third-party API security solutions to provide additional layers of protection.

4. Kubernetes-native Approach

Traefik Hub provides a Kubernetes-native approach through deep integration with the Kubernetes API. Starting with the auto-discovery of existing microservices, Traefik Hub leverages familiar and powerful Kubernetes objects and primitives such as Custom Resource Definitions (CRDs), labels, and selectors to create, manage, and secure APIs at scale.

5. Quick Time to Value

Traefik Hub control plane provides a simple and intuitive point-and-click experience for users who are just getting started with publishing APIs in a single or multi-cluster Kubernetes environment, while also giving advanced Kubernetes users access to its full automation and GitOps capabilities. This dual operating model accelerates Kubernetes-native adoption and
maximizes business agility, giving organizations unprecedented speed and scale in launching new APIs and services.

Getting started with Traefik Hub

Traefik Hub addresses the challenges and limitations of traditional API management, by providing a true Kubernetes-native experience with full GitOps compliance for simplicity, automation, and security. With Traefik Hub, engineers can continue to use their preferred tech stack to build great APIs and applications. We believe Traefik Hub is the most open, modular, and intuitive platform for managing APIs across hybrid, multi-cloud, and edge environments.

If you are unsatisfied with your existing API management solutions, we would love to hear from you and encourage you to take Traefik Hub for a spin.

Sign up for a two-week trial here and experience a better way to manage your APIs.

]]>

ABAX Tracks Over 500,000 Assets with Entirely Cloud Native Strategy

Traefik Labs — Thu, 04 May 2023 23:18:53 GMT

At-a-Glance

Over the course of three years, ABAX entered a new industry and doubled the number of assets it tracks for customers. In that same period, it standardized its compute operations onto Kubernetes and successfully completed its migration to a fully cloud native environment.

Open source-based solutions, such as Rancher Prime, K3s, and Traefik Enterprise, played a central role in these achievements and continue to support the company’s next-generation operations, which run with less than five IT personnel at the helm.

Introducing ABAX

In 2007, ABAX opened its doors to deliver basic tracking solutions through high-quality GPS positioning. Since then, it has expanded services to help companies manage assets in new and better ways, leading it to become one of the largest telematics companies in Europe. Today, its sophisticated fleet and asset tracking solutions help clients reclaim stolen property, extend the lifecycles of expensive machinery, increase worker productivity, meet taxation requirements and even reduce carbon emissions.

In 2020, ABAX embarked on a journey to expand its offerings by developing mobility data solutions and entering new verticals, such as insurance and leasing. As a result, the company has become the preferred mobility data provider for the business-to-business industry on a broader scale.

ABAX's solutions empower customers to operate their businesses more intelligently and sustainably, making it simple to enhance efficiency, adhere to regulations and boost profitability. This approach is known as smart mobility.

The Journey to Cloud Native Operations

Combining telecommunications with information technology, ABAX is committed to using the latest technology advancements to send, receive, store and interpret data for its customers. In 2017, the company started using Kubernetes, Rancher Prime and public cloud providers to scale its operations for accelerated growth in the coming years.

Having proven to itself the viability of cloud-based container operations, it laid a plan in the Fall of 2020 to go fully cloud native by the end of 2022, but those plans went into hyperdrive mid-2021 when its on-premises data center provider announced closure by the end of the year. Thanks to the support of cloud native networking solution, Traefik Enterprise (an integrated API Gateway and ingress controller), ABAX seamlessly migrated its on-premises data center services to the cloud by directing traffic from one environment to the other. Originally planned as a multi-year migration project, ABAX was able to complete its migration in a quarter of the time and successfully went fully cloud native Dec. 30, 2021, one day before its on-premises data center pulled the plug.

Despite the accelerated timeline, SUSE solutions, Traefik Enterprise, and public cloud providers enabled ABAX to not only ensure continuity of services, but also reserve time for innovation. Remarkably, less than five people sit on the IT team that supports more than 70 developers and the environments of nearly 45,000 customers. For the team, maintaining these environments at scale, while having time to implement competitive infrastructure features, is vital for success.

“Providing software as a service for as much as we can is a requirement for us to be able to survive as an operations team due to resource availability,” says Thomas Ornell, principal system engineer (level IV) at ABAX.

A Modern, Open Source Infrastructure

ABAX processes a vast amount of data from over half a million devices every hour, making uninterrupted availability critical to the company's success. To ensure reliability, the company turned to Kubernetes as its primary compute platform, migrating numerous IT processes to the container orchestration system. ABAX runs most of its Kubernetes clusters on Google Public Cloud, with some operations on AWS and Microsoft Azure.

As a proponent of open source, ABAX relies on several open source solutions, including Rancher Prime, K3s and Traefik Enterprise, to optimize its infrastructure operations and deliver services to clients. These tools allow ABAX to manage and deploy its Kubernetes infrastructure efficiently and securely, maintain granular control over user access, and provide next-generation customer services.

“With Traefik Enterprise, everything is automated and easy to use. We maintain functionalities that our developers rarely see because they just work.”

Thomas Ornelll, Principal System Engineer (Level IV), ABAX

“Rancher Prime and Traefik Enterprise have become an integral part of the ABAX cloud native infrastructure,” says Ornell. “With Traefik Enterprise, everything is automated and easy to use. We maintain functionalities that our developers rarely see because they just work.”

Regardless of its dedication to open source, ABAX understands the importance of risk mitigation and utilizes enterprise support to ensure its systems remain stable and secure. By partnering with SUSE and Traefik Labs, ABAX can focus on building its future and delivering exceptional services to its clients with peace of mind.

The Impact

Maximizing Productivity with a Consistent Experience

ABAX, with a lean IT team handling various platform stacks and tools, aims to provide a consistent experience, performance and troubleshooting to its application developers. To achieve this, the IT team needs complete control over the platform stack. Rancher Prime and Traefik Enterprise make ABAX’s vision of a cloud native, infrastructure-agnostic platform a reality, providing developers with a single, consistent experience, regardless of the underlying cloud infrastructure.

The Rancher Prime interface allows developers and operations to work together in production with ease. The experience is the same, no matter which cloud a cluster is hosted on. “We don’t have to train our developers how to use the different user interfaces between the different cloud providers,” says Ornell. “Rancher Prime’s unified control system is key to maximizing productivity.”

Saving Time with Automated RBAC

Rancher Prime’s automated role-based access controls (RBAC) have also made a significant impact in enhancing the IT team’s productivity. Automated RBACs enable the team to grant and limit access to clusters and defined projects within Rancher Prime easily. If a developer works on a particular business domain, they can be granted access to only that business domain inside the production cluster, which is particularly important in highly regulated customer industries like insurance.

“What we're doing is giving developers direct access so they can manage their own workloads to a certain extent, saving us valuable time,” says Ornell. “This is key for us to be able to survive as such a small team managing so many different things.”

Simplifying Kubernetes Deployments with Fleet

Fleet, a feature of Rancher Prime, simplifies managing, deploying and scaling containerized applications across multiple clusters. It simplifies the process of managing distributed application deployments, allowing users to easily start, scale and keep track of the containers that make up their applications. It also saves time and resources by automating how users set up applications in different environments.

Fleet currently runs 51 infrastructure deployments and maintains more than 70 Kubernetes clusters for developers, which would be impossible for the IT team to manually manage otherwise. “We label our clusters with the features we want them to have,” says Ornell. “For example, we’ll label a cluster with Traefik Enterprise. Fleet then automatically deploys Traefik Enterprise to that cluster. Same with Datadog or any other infrastructure piece we might need in a cluster. This is key to being able to do a disaster recovery swap quickly because we wouldn’t have time to manually reconfigure everything when in a disaster scenario.”

Faster Disaster Recovery with a Powerful Trifecta

Speaking of disaster scenarios, ABAX was able to achieve an impressive disaster recovery time for its entire production compute environment (excluding the database environment) in just one hour 15 minutes, thanks to three features of Rancher Prime.

Rancher Prime's multi-cluster management enabled the team to efficiently monitor and control resources across multiple clusters through a single pane of glass, while RBAC and Fleet made deploying fine-grained control over user access to resources across multiple clusters significantly easier. With these three components, the team was able to relaunch its operations efficiently and securely at scale, saving precious time.

According to Ornell, “Getting the environment itself back online wouldn’t take that much longer without Rancher Prime. The real difference lies in the post-recovery work, such as providing proper access for developers, ensuring all infrastructure components are back in the cluster and so on. Rancher’s RBAC and Fleet make these steps significantly faster. For comparison, doing this manually would take days.”

Standardizing Cluster Environments with K3s and Traefik

ABAX uses K3s to standardize cluster environments between development and production. K3s is a lightweight Kubernetes distribution originally developed by Rancher Labs and is now a CNCF project. K3s uses Traefik as the default ingress controller to manage incoming traffic. “The tiny differences between developer machines and production environments leave room for error, especially when trying to figure out why a service is working on a developer machine but not on a production environment,” says Ornell.

“K3s, deployed through k3d, and Traefik enable us to deploy everything in a developer environment as if it were a production environment, making it easier to identify issues when they arise.”

Thomas Ornelll, Principal System Engineer (Level IV), ABAX

The team implemented k3d, a lightweight wrapper used for running K3s in Docker, to run small Kubernetes clusters on every developer machine. “K3s, deployed through k3d, enables us to deploy everything in a developer environment as if it were a production environment, making it easier to identify issues when they arise,” says Ornell.

Consequently, nothing has to change in how the Kubernetes and networking layers operate when applications move from development to production, speeding time to launch.

Resolving Issues with SUSE and Traefik Labs’ Enterprise Support

SUSE’s commitment to open source ethos means that Rancher technology is the same whether it comes from the community or enterprise version. For resource-constrained teams, paid support can be invaluable.

In working with SUSE Support, Ornell has found the team resolves issues before other vendors, working on the same issue, deliver a second reply. “SUSE has the competency to debug Kubernetes comprehensively, so we can figure out what is actually going wrong,” says Ornell. “We have very limited resources available. If we can offload debugging and figuring out what’s wrong to a support team instead of us having to do it, that’s golden on our end”

Support goes to another level when support teams from SUSE and Traefik Labs collaborate to resolve issues. Reflecting on a particular instance, Ornell states, “We couldn’t pinpoint where the issue was and we ended up being on a support call with support representatives on both sides collaborating, which to me is invaluable.”

Accelerating Solutions Via a Partner Ecosystem

SUSE’s partner network also provides additional value for those working with the company. This is how ABAX, for instance, discovered Traefik Labs when it sought to employ a Kubernetes-native ingress controller to manage incoming data from various client machinery, assets and automobiles. After browsing SUSE’s partner application catalog for Rancher Prime, ABAX selected Traefik and immediately integrated it into its infrastructure. Now, Traefik Enterprise handles every incoming request. Traefik Labs also played an essential role in ABAX’s cloud migration acceleration. Moreover, all customer-facing portals run on Traefik via a GKE cluster in Google Cloud, provisioned by Rancher Prime.

What’s Next?

Looking ahead, ABAX, now a major player in the European telematics market, plans to leverage the data it already processes to provide additional customer value. By embracing data sciences and artificial intelligence, ABAX customers will soon gain new insights that will inevitably lead to greater innovations. Backed by open source solutions and expertise, ABAX has a bright future ahead.

Originally posted on suse.com

]]>

Announcing Traefik Proxy 2.10

Nicolas Mengin — Wed, 26 Apr 2023 13:47:21 GMT

With improved native Kubernetes Service load balancing, new Prometheus metrics, new API group, and more.

There is always a lot of excitement around a new major version, but making the best possible Traefik Proxy 3.0 takes time and people. This past year, as we geared up to decide what Traefik proxy 3.0 should look like, we have also been working towards more open governance.

Our goal is to ensure the community can drive the direction of Traefik Proxy by influencing our roadmap and having significantly more input into feature design, implementation, and direct access via activities traditionally done by official maintainers, such as reviews. Check out these requests for design and input, as well as this hot topic on handling the migration to the new Kubernetes CRD definitions.

As part of our efforts to be more inclusive to our community, we have collected a series of small but mighty improvements that bring no breaking changes but enhance the user experience for many of our community members. Rather than holding these features until 3.0 is ready for release, we decided to ship these improvements early so that everyone can benefit from them right away.

Traefik Proxy 2.10 improves your ability to use service mesh with Traefik Proxy, enhances Prometheus metrics, and simplifies your Nomad configuration.

Let's jump into the new release and explore the nitty-gritty of your new abilities and how to use them.

Improved native Kubernetes Service load balancing

Up until now, Traefik Proxy only forwarded incoming traffic to pods. This made it difficult to address specific use cases that require the native Kubernetes load balancing with Traefik because it required the use of workarounds, for example, the creation of external services. These workarounds were unsatisfactory, and this missing feature was a blocker, especially for adopting Traefik for service mesh users.

Now, users have a new option for the providers Kubernetes Ingress and Kubernetes IngressRoute to decide whether the children of any given load balancer are directly in Pod IPs or if a Kubernetes Service is designated as the single child. In this case, the Kubernetes Service itself balances the load to the pods through a list of all endpoints used by the ingress controller in the upstream configuration.

This is especially important to users who employ a third-party service mesh, such as Cilium. Another benefit is that this configuration reinforces Traefik Proxy’s native support for zero-downtime deployments by removing any chance that traffic will be redirected to a non-existent client or pod (should a pod go down).

How it works

Enabling this feature is quite simple; all you need to do is add the `nativeLB` option to the service.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami
namespace: traefik
spec:
entryPoints:
- web
routes:
- kind: Rule
match: PathPrefix(`/who`)
services:
- kind: Service
name: whoami
namespace: app
port: 80
nativeLB: true # Enable the option

Note: In the specific case of a deleted pod, when a request comes in before Traefik has updated its routing configuration, you might generate a 502 bad gateway response because the Traefik configuration does not reflect the actual infrastructure. Using the `maxIdleConnectionPerHost` option mitigates the 502 error by creating a new connection with the backend service (pod), avoiding connection reuse to a pod that has suddenly gone down.

New Prometheus metrics

Prometheus is by far the most used third-party tool for metrics. To enhance the user experience when using Prometheus with Traefik Proxy, you can now split observations of the total request metric based on the value of one or multiple header values. This option allows you to gather more detailed information about your clients based on header information.

Headers are inherently flexible, so we can think of many ways you would use this feature, including creating a custom header to disclose app versions.

How it works

Traefik Proxy will now allow you to define extra labels for the `requests_total` metrics and the request header containing the value assigned to each label.

metrics:
prometheus:
buckets:
- 0.1
- 0.3
- 1.2
- 5.0
headerlabels:
useragent: "User-Agent"

Note: This feature is not enabled by default, therefore, there is no impact on performance by default. During testing, we did not observe any relevant performance hit. However, this does add an additional operation in the critical path of the request’s handling. When you enable the feature, if the header is not present in the request, it will be added automatically with an empty value. The label needs to be a valid label name for Prometheus.

Multiple namespaces in Nomad

Nomad allows you to use multiple namespaces within any given cluster. However, our original integration allowed you to use only a single namespace and required you to spin up an instance of Traefik Proxy per namespace within your cluster. We deprecated the singular namespace in the first Traefik Proxy 3.0 Beta, and this release brings the ability to use a single instance of Traefik to cover all namespaces within a given cluster.

providers:
nomad:
namespaces:
- "ns1"
- "ns2"

Introducing traefik.io API group

We are introducing our new CRD API group, `traefik.io`, with an eye to deprecating the `traefik.containo.us` API group in Traefik Proxy 3.0. Traefik Proxy v2.10 will bring no additional changes to the CRD scheme other than updating the API group. It will also include a warning log about the future depreciation of `traefik.containo.us` resources which will be removed in v3.0. You can read more about the community conversation in pull request 9600 and the Kubernetes CRD improvements in issue 9125. We would love to hear your feedback!

To use the new API group, install/update the CRDs and RBAC in your cluster, and you'll be able to define middlewares, IngressRoutes, etc., as either `traefik.containo.us` or `traefik.io`.

Wrapping up

We love that these features add flexibility to Traefik Proxy and hope you find them useful as well. We want to thank all the community members who have contributed to 2.10 through conversations on GitHub in issues and pull requests, donating code, testing, and sharing feedback on the 2.10 release candidate. This time, the special shout-out goes to Philipp and Sebastien, the top contributors of v2.10.

If you are interested in contributing to Traefik Proxy, join us on GitHub, where you can request features/enhancements, help us design the next version, or even get your hands dirty by creating or reviewing pull requests. For more information, check out the Release Notes and updated Docs. If you have questions, pop over to our Community Forum, where you can work with other community members to find your answers. And as always, we look forward to your feedback.

]]>

Announcing Traefik Enterprise 2.10

Nicolas Mengin — Thu, 20 Apr 2023 10:47:09 GMT

With support for claim data in OIDC access tokens, headers for rate-limited responses, and Kubernetes secrets for storing Authentication Sources credentials.

Delivering on our promise to relieve organizations from connectivity chaos, today, we are announcing the release of the latest Traefik Enterprise version.

The 2.10 release is short and sweet but has three new features to help current and future customers better secure and manage their APIs and services.

Let’s take a closer look.

OIDC middleware now inspects claims in access tokens

OIDC is being widely used for authentication and authorization purposes. To use OIDC as an authorization method, users need to implement claims. Claims contain user information and meta information on the OIDC service in the form of a name/value pair.

Until now, the OIDC middleware inspected claims in ID tokens, validated if the claim group, grp, has the value admin, and restricted the workload, allowing access only to the ID token that contains the claim grp and value admin.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-oidcAuth
spec:
  plugin:
    oidcAuth:
      source: oidcSource
      redirectUrl: example.com/callback
      session:
        secret: mysupersecret123
      claims: Equals(`grp`, `admin`) # Check the claim in the ID Token

While ID tokens are obtained only with the explicit consent of a human, access tokens are obtained through an automated process between systems and are considered a more secure method of adding an authorization layer to OIDC.

With the 2.10 release, we allow Traefik Enterprise users to add claims in Access tokens as well, enabling them to secure their APIs and services better.

Here is an example of how to use claims in Access tokens with the OIDC middleware.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-oidcAuth
spec:
  plugin:
    oidcAuth:
      source: oidcSource
      redirectUrl: example.com/callback
      session:
        secret: mysupersecret123
      claims: Equals(`access_token.grp`, `admin`) # Check the claim in the Access Token

Note: This feature works with JWT access tokens only.

Load sensitive data from Kubernetes Secrets

With this release, we are further improving how customers secure their APIs in Traefik Enterprise, by allowing for sensitive options values to be referenced in Traefik Enterprise using the Secrets object in Kubernetes.

With this improvement, credentials and other sensitive information are hidden in the static configuration.

To reference a Kubernetes Secret in Traefik Enterprise, use the form of a URN:

urn:k8s:secret:[namespace]:[name]:[valueKey]

Here’s an example of this configuration with JWT:

#...

authSources:
  jwtSource:
    jwt:
      signingSecret: urn:k8s:secret:traefikee:jwt-secret:secretValue

Note: The Traefik Enterprise Controllers need to be running within a Kubernetes cluster and users need to load the Kubernetes Secrets within the Traefik Enterprise deployment namespace.

Headers for rate-limited responses

Rate limiting is essential to API security. Limiting how often APIs can be called and throttling connections helps protect against traffic spikes and DDoS attacks.

This feature is an excellent addition for users who want to limit API consumers to a predefined number of requests per minute and inform consumers on the fly about their remaining amount of requests.

Using the ResponseHeaders option, users can configure Traefik Enterprise to inject the X-Rate-Limit-Remaining header in the response. This indicates how many tokens an API consumer has left in the token bucket before they get a 429 response.

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-ratelimit
spec:
  plugin:
    rateLimit:
      responseHeaders: true

Summing up

Traefik Enterprise 2.10 is by no means a major release, but we are confident that it offers substantial benefits for customers looking to streamline their API security. If you are currently looking to level up your API game and solve the challenges that come with APIs, join our upcoming webinar and learn:

How an API gateway unlocks APIs in a cloud native world
How API gateways deliver great time-to-value and reduce infrastructure complexity

And in case you haven’t heard, we are at KubeCon Europe 2023! Feel free to drop by Booth S24 for a chat and a live demo of our new industry’s-first Kubernetes-native API management solution, Traefik Hub.

]]>

Production-Ready Kubernetes Deployments with the Traefik API Gateway and Supabase

Colin Wilson — Wed, 05 Apr 2023 16:11:40 GMT

API gateways have rapidly evolved to become a crucial component of most modern cloud infrastructure — particularly within microservices architectures. More so than other cloud components, API gateways need to be secure, robust, resilient, and easy to manage, and the Traefik API gateway is definitely up to this challenge.

In this guide, I want to give you a tour of the Traefik API gateway and explore how to configure it to enable a production-ready, self-hosted deployment of the Supabase Platform on Kubernetes.

Supabase describes itself as “an open source Firebase alternative”. The platform provides a collection of application backend features such as Authentication, Database (Postgres) and Storage (S3) and is quickly growing in popularity amongst many application development communities.

Why choose Traefik Proxy for this guide? Well, because the current Supabase Helm Chart utilizes a rather bare-bones deployment of Kong and Nginx. And although functional, both these configurations fall short of a production-ready environment, namely, a lack of authentication for the GUI and rate limiting of the API.

Substituting Kong and Nginx for Traefik Proxy makes up for the above shortcomings and provides a host of additional benefits:

Built-in Let's Encrypt integration
Simpler to configure
Several Middlewares available
Management of a single API Gateway instead of two
An upgrade path to Traefik Enterprise

Prerequisites

A Kubernetes Cluster (with cert-manager installed)
Knowledge of Kubernetes and Helm

Note: Many of the Traefik configurations and concepts used here also apply to other application platforms with similar API gateway requirements.

Architecture

The above architecture represents a high-level overview of the end goal. A production-ready deployment of Supabase on Kubernetes with Traefik API gateway securing the Supabase API and GUI (Studio).

Deploy Supabase on Kubernetes

Note: If you already have a Supabase deployment on Kubernetes, you can skip to the next section to install and configure Traefik Proxy. Since Traefik will be configured to expose the Supabase endpoints it’s important to disable any related Supabase endpoint ingress configurations in your existing deployment.

Use the Supabase Helm chart provided by the supabase-community repo to deploy Supabase on a Kubernetes cluster. The chart’s values.yaml options should be configured to meet the requirements of a production environment (and with Traefik Proxy in mind). So, configure:
- An externally managed or cluster replicated PostgreSQL database
- An external transactional mail service, e.g. SendGrid, Mailgun or Sparkpost
- An external (or internal HA) S3 compatible storage provider, e.g. AWS or Cloudflare
The chart’s default values.yaml enables an nginx ingress for both the Supabase Studio and API services (studio.ingress.enabled and kong.ingress.enabled). Since we’ll be configuring Traefik ingress routes for these services later on, both should be set to false.
Generate the JWT keys and secret values required to create the JWT secret inside the cluster.
Create a database secret containing the username and password of your PostgreSQL database and another secret with your SMTP provider credentials.
Deploy the Supabase helm chart inside your cluster.
helm -n default install supabase -f my-custom-values.yaml.

Use the kubectl port-forward command to check the Supabase API endpoints and GUI have successfully deployed in your cluster.

To check Supabase Studio, run the following command:

> kubectl port-forward service/supabase-supabase-studio -n default 3000:3000

Navigate to http://localhost:3000/projects and you should be presented with the Supabase Studio frontend:

Once you’ve confirmed Supabase Studio is working, go ahead and test the API endpoint next.

Navigate to http://localhost:3000/project/default/api?page=auth. This API Docs page features API commands with your previously configured access credentials substituted in. Click to copy the generated curl command used to authenticate with the /rest/v1/ endpoint.

Use the kubectl port-forward command once again, this time to proxy the Supabase API:

> kubectl port-forward service/supabase-supabase-kong -n default 8000:8000

Next, execute the curl command you copied from the API Docs in Studio:

curl 'https://api.supabase.local/rest/v1/' \
-H "apikey: SUPABASE_SERVICE_KEY" \
-H "Authorization: Bearer SUPABASE_SERVICE_KEY" | jq

The result should start similar to the partial snippet below:

  {
  "swagger": "2.0",
  "info": {
    "description": "",
    "title": "standard public schema",
    "version": "10.2.0.20230209 (pre-release)"
  },
……

Supabase is now successfully deployed and primed to be made production-ready with the help of the Traefik API gateway 🎉.

Deploy and configure the Traefik API gateway

As mentioned earlier, the default deployment of the supabase-kubernetes helm chart requires some important additional steps before it’s production-ready.

Install Traefik (Helm)

Before installing Traefik, make a minor addition to the values.yaml file so Traefik redirects all HTTP requests to HTTPS:

ports:
  web:
    redirectTo: websecure

Next, install Traefik via the traefik-helm-chart:

helm install -f myvalues.yaml traefik traefik/traefik

Configure the Traefik API gateway for Supabase’s APIs & GUI

Now it’s time to create the necessary Traefik configurations.

If you followed the Supabase chart deployment instructions above, you’ll recall the ingress configurations for both the Studio and API services were disabled (set to false).

Both ingress routes require a valid SSL certificate for their respective domains. e.g.

api.supabase.example.com for the Supabase API ingress route
studio.example.com for the Supabase Studio ingress route

Ensure these certificate secrets exist before creating the ingress routes that reference them.

Now, create the ingress routes that will publicly expose both the API endpoints and Studio services, this will also enable SSL/TLS termination for both routes:

api.supabase.example.com.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: api.supabase.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`api.supabase.example.com`)
      services:        
        - name: supabase-supabase-kong
          port: 8000
  tls:
    secretName: api.supabase.example.com

studio.example.com.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: studio.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`studio.example.com`)
      services:        
        - name: supabase-supabase-studio
          port: 3000
  tls:
    secretName: studio.example.com

Create two corresponding DNS records for your domain with your DNS provider, api.supabase.example.com and studio.example.com, both pointing to the IP address of the load balancer associated with your Traefik instance.

Protect the Supabase Studio dashboard with Traefik middleware

Navigate to https://studio.example.com, and you should (again) be presented with the Supabase Studio web interface. You’ll notice the Studio dashboard is currently unprotected — i.e., there’s no authentication step before access.

The Supabase dashboard can be secured using a Traefik *Auth middleware such as BasicAuth. Traefik's ForwardAuth middleware with an external authentication server like Authelia is another option.

However, I highly recommend taking access control to another level with Traefik Enterprise. Its built-in middleware OpenID Connect Authentication integrates with existing authentication deployments. See 'Going further with Traefik Enterprise' below for more details.

Rate limiting the Supabase API

At this point, Traefik is configured so both the Supabase Studio and API endpoints are secured against unauthorized or unauthenticated access.

So, can this deployment be considered production-ready? Not quite yet. The Supabase API has no protection against bursts of incoming traffic, be it from unexpected high traffic or malicious abuse, such as Denial-of-Service (DoS) attacks, it's a concern that cannot be left unaddressed.

Traefik has this covered with yet again another middleware option, the RateLimit middleware.

Define a rate limit to suit your needs using a middleware object:

# Here, an average of 200 requests per second is allowed.
# In addition, a burst of 100 requests is allowed.
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: ratelimit-supabase-api
  namespace: default
spec:
  rateLimit:
    average: 200
    burst: 100

This will set an overall rate limit across all Supabase API endpoints. This is a good starting point, plus the RateLimit middleware is capable of more granular rate limiting via options such as sourceCriterion.ipStrategy and sourceCriterion.requestHeaderName which sets rate limits based on incoming IP and request header groups respectively.

Apply the new middleware to the api.supabase.example.com ingress route:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: api.supabase.example.com
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`api.supabase.example.com`)
      services:        
        - name: supabase-supabase-kong
          port: 8000
      middlewares:
            - name: ratelimit-supabase-api
  tls:
    secretName: api.supabase.example.com

Once applied, your Supabase API route now has a safeguard against high volumes of traffic. You can also set a rate limit on the Studio service if required.

Production ready?

The Traefik API gateway is now handling:

TLS/Termination for the Supabase Studio and API routes
Secure authentication and authorization to Supabase Studio
Rate limiting of the Supabase API route

Your Supabase deployment is now production-ready!

Going further with Traefik Enterprise

Although this guide demonstrates a production-ready deployment of Supabase using the open source version of Traefik, it’s not without its limitations. Upgrading to Traefik Enterprise offers a path to overcoming these limitations. Let’s take a look at some ways in which Traefik Enterprise achieves this.

Authentication and authorization

As mentioned, Traefik Enterprise offers much more robust authentication solutions over middleware like BasicAuth and ForwardAuth. For example, the built-in OpenID Connect Authentication middleware means Traefik Enterprise can easily accommodate your existing authentication and authorization infrastructure to provide access control to Supabase.

The above is a simplified illustration of how the OIDC Authentication middleware functions. For a more detailed look at how to configure the OIDC middleware, check out Matt Elgin's article, 3 OIDC Configurations with Traefik Enterprise, from Basic to Advanced.

High availability

The very start of this guide mentions the importance of API gateways in modern microservices architectures. In contrast to Traefik’s open source edition, Traefik Enterprise is deployed by default as a cluster of proxy nodes. Substituting the default Supabase API gateway (a potential single point of failure) for Traefik Enterprise enables the Supabase endpoints to be served in a highly available and scalable fashion by a cluster of proxies. Not to mention the reduced complexity from now only managing a single API gateway installation, Traefik Enterprise.

Migrating to Traefik Enterprise also introduces a wealth of authentication protocols (beyond JWT) for authenticating with the Supabase API (including LDAP, OAuth2, OpenID, and HMAC). This opens the door for many other applications to interact with your API.

Distributed rate limiting

The rate limit middleware configurations applied to Supabase endpoints in this guide limit requests per an individual Traefik proxy. Traefik Enterprise’s distributed rate limiting ensures that requests are limited over time throughout your cluster and not just on an individual proxy.

Configuring this in Traefik Enterprise is as simple as adding a single line to your existing open source RateLimit middleware config:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: ratelimit-supabase-api
  namespace: default
spec:
  plugin: # <- just add this line to enable distributed rate limiting
	  rateLimit:
		average: 200
		burst: 100

Support

Last but not least — support. Traefik Enterprise comes with built-in support. A team of engineers is at your disposal should you encounter any issues or need any assistance with the installation, setup, or management of Traefik Enterprise. Quality support (in my opinion) is an important part of what makes a deployment “production-ready.”

Conclusion

This Supabase deployment has demonstrated how the Traefik API gateway can help build a production-ready application through the implementation of some of its key features.

Bear in mind authentication, rate limiting, TLS termination, and high availability are just the tip of the iceberg when it comes to the capabilities of the Traefik API gateway. If your application architecture requires a more complicated feature set, such as service mesh, security compliance, or disaster recovery, Traefik Enterprise can accommodate these needs.

To learn more about how the Traefik and Traefik Enterprise API gateways can enable production-ready applications and platforms, check out Traefik’s excellent glossary and the documentation for both Traefik Proxy and Traefik Enterprise. If you’d like to try out Traefik Enterprise for yourself, you can sign up for a 30-day free trial.

]]>