Token as a Service (TaaS) is the operating model that emerges when an organization decides to expose AI inference as a metered, multi-tenant, governable service. NeoClouds expose it to external customers. Sovereign operators expose it to national constituents. Enterprises expose it to internal lines of business. Telcos expose it at the edge. The shape of the service is the same in all four cases, even when the substrate underneath varies from a colocated GPU rack to a hyperscaler region to an air-gapped sovereign cloud.

The hard part of TaaS is not inference. Modern inference runtimes already solve throughput, batching, and KV cache locality. The hard part is everything that surrounds inference once it becomes a product: tenant identity, admission control, token-level metering, model routing, guardrail composition, agent and tool authorization, audit, and a trust boundary the buyer believes. And because a growing share of inference consumers are AI agents rather than humans, the trust boundary now extends beyond the model call into the tools those agents invoke, most commonly through the Model Context Protocol (MCP). A TaaS architecture that governs only the inference call governs only half the workload.

That control plane is a gateway problem. This article describes a reference architecture for building it, organized around four propositions:

TaaS requires a unified runtime layer that spans the API surface, the AI inference path, and the agent tool surface. The Triple Gate Pattern is that runtime.
The control plane has two distinct authorization domains that must be coordinated but not conflated: token-level cost and quota control at the AI Gateway, and Tools/Tasks/Transactions-Based Access Control (TBAC) for AI agents at the MCP Gateway. Tool calls are not inference and may not even run on GPU infrastructure, but they are initiated by inference and generate further inference, so a TaaS meter that ignores them systematically underreports agentic consumption.
Before a provider selects a substrate, it must select a deployment topology. The choice between a shared multi-tenant service, dedicated isolated stacks, and dedicated stacks with a federated control plane is the decision that determines blast radius, operability, and whether the offering is a service at all. Substrate is a second, orthogonal choice.
The meter and the policy plane must be independent of the model provider and the compute substrate. Architectural neutrality is the buyer's only defense against lock-in and the only credible posture for sovereign and regulated workloads.

The Token Economy Has Arrived

Three shifts converged through Q1 and Q2 2026 that turned TaaS from a slide into a category:

Inference has overtaken training in compute share. Industry analysts project that inference workloads will account for roughly two-thirds of all AI compute in 2026, up from about one-third in 2023. The center of gravity for AI infrastructure investment is now the inference path, not the training cluster.

Tokens have become the unit of value. The industry now prices, sells, and benchmarks AI in tokens, with tiered consumer offers ranging from free through low- and mid-priced plans up to premium research tiers in the tens to hundreds of dollars per million tokens. This vocabulary has been adopted by hyperscalers, model providers, NeoClouds, and AI factory operators alike. Several NeoCloud and AI infrastructure providers have already launched explicit "token factory" or inference-as-a-service offerings under this banner.

Sovereignty has shifted from training to inference. Sovereign AI strategies designed eighteen months ago around training clusters are now confronting a workload mix in which the politically and economically interesting computation is inference, and inference is portable, distributed, and continuous. The sovereignty question has moved from "can the model be trained inside the border" to "can every inference be governed, metered, and audited inside the border."

The result is a market that wants to buy inference the way it buys electricity: by the unit, on a predictable contract, with a meter both parties trust, and with the option to switch suppliers without rewriting applications.

Why TaaS is an Architecture Problem, Not an Inference Problem

A naive TaaS stack looks like a model server, a load balancer, and a billing script. That stack survives a proof of concept and fails the first contract negotiation. The failure modes are predictable.

The meter cannot be trusted. If the seller runs the meter, the buyer cannot audit it. If the model provider runs the meter, the seller cannot resell across providers without losing visibility into margins. The meter needs to live in a layer that neither party owns exclusively.

The pricing model breaks under real usage. Token consumption is elastic. A short prompt can produce a long output. An agent loop can consume thousands of tokens per task. A retrieval-augmented call can amplify context size by an order of magnitude. Finance teams trained on per-seat SaaS contracts cannot model their costs against this profile. Predictable, transparent metering with pre-request estimation has become a procurement requirement, not a feature request.

Multi-tenancy fails open under load. Without admission control at the API boundary, a single batch job can saturate an inference cluster and degrade latency across every tenant on the system. Effective capacity management requires policy enforcement at the request boundary, not at the container or GPU level.

Agent traffic breaks single-layer governance. Agentic workloads do not stop at the LLM call. They proceed into tool invocations, database queries, file system actions, and downstream API calls, each of which is itself an authorization and metering event. An LLM proxy sees the model's interaction but not the agent's subsequent actions. An MCP proxy sees the tool calls but not the LLM conversation that triggered them. Neither covers the full workflow.

Compliance is a per-tenant question. Different tenants will require different guardrails, data residency, audit retention, and model whitelists. A static safety pipeline cannot serve all of them. Guardrails must be composable per route and per tenant.

Lock-in is the buyer's first objection. A TaaS offering that ships with a fixed model provider, a fixed substrate, and a fixed observability stack is a hyperscaler bundle by another name. Sovereign and regulated buyers will reject it on first read.

The capabilities below are the requirements that follow from these failure modes.

The Six Capabilities Every TaaS Provider Must Deliver

Capability	What It Does	Where It Lives in the Triple Gate
Tenant identity and entitlement	Authenticates the calling tenant, resolves the active plan, and applies the base quota and rate policy	API Gateway
Token-level cost and quota control	Counts input, output, cached, and tool-call tokens by tenant, model, and route; enforces burst-tolerant rate limits and hard token quotas with pre-request estimation	AI Gateway
Model and route policy	Steers requests across models and providers based on cost, latency, locality, and tenant policy; supports multi-provider failover	AI Gateway
Composable safety pipeline	Applies prompt injection, content safety, PII, topic, and custom guardrails per tenant and per route, with parallel guard execution and structured refusals	AI Gateway
Agent and tool authorization (TBAC)	Authorizes AI agents at the tool boundary using Tools/Tasks/Transactions controls expressed as policy against JWT claims	MCP Gateway
Unified audit and observability	Produces tamper-evident records of who consumed what, when, under which policy, including LLM calls and tool calls; exports to vendor-neutral OpenTelemetry backends	Across all three gates

These six capabilities form the TaaS control plane. They sit between the consumer and the inference substrate, and they are the surface where the buyer's trust is established or lost.

The Triple Gate Pattern as the TaaS Control Plane

The Triple Gate Pattern organizes the control plane into three coordinated gates, each owning a distinct surface but sharing one identity, policy, and observability fabric.

The API Gate

The API Gate terminates the external contract. It owns tenant authentication, plan resolution, base quota, request shaping, schema enforcement, and the public surface that a customer signs against. For TaaS, the API Gate is where the legal meter begins and where structured API contracts protect backend services.

The AI Gate

The AI Gate terminates the inference path. It owns model routing, token-level counting, semantic caching, the composable safety pipeline, multi-provider failover, and the runtime policy that turns a tenant's plan into a per-request decision.

The Hub 3.20 release operationalizes this through the AI Token Rate Limit and Quota Middleware, which provides burst-tolerant token rate limits and hard token quotas with pre-request estimation and shared state across gateway replicas, allowing oversized or over-budget prompts to be rejected before they reach the model. The Parallel LLM Guard Middleware runs multiple guardrails concurrently on the same prompt, so the total enforcement time equals the slowest guard’s, rather than the sum. The Content Guard Regex Engine lets teams define deterministic, sub-millisecond patterns for product codenames, customer identifiers, regulated IDs, or any business-specific format. Graceful Error Handling allows refused requests to return structured HTTP 200 refusals that agent middleware chains can process without crashing.

The MCP Gate

The MCP Gate terminates the agent tool path. It implements Tools/Tasks/Transactions-Based Access Control (TBAC), the authorization model for AI agents. TBAC operates across three dimensions: Tasks (which business objectives the agent is allowed to pursue), Tools (which specific MCP servers and tools the agent can invoke), and Transactions (parameter-level constraints on what the agent can do with each tool, such as monetary limits, allowed repositories, or allowed channels).

Because the MCP Gate terminates the connection before the tool server receives it, TBAC cannot be bypassed by a compromised agent. This is the control that closes the agentic governance gap left open by pure LLM and pure MCP proxies.

A unified runtime layer behind all three gates carries identity, policy, telemetry, and configuration. The same JWT identity that enters the API Gate flows into model routing and token accounting decisions in the AI Gate and into TBAC enforcement in the MCP Gate. The same observability backbone produces a single timeline of consumption for billing and audit. The same declarative configuration model deploys identically across public cloud, on-premises, and air-gapped environments through GitOps workflows.

The reason this architecture wins for TaaS is that it collapses three otherwise-fragmented control planes into one. Fragmented control planes do not produce trustworthy meters.

Token-Level Controls and TBAC: Two Distinct Authorization Domains

A common source of confusion in early TaaS designs is the conflation of two authorization domains that operate on different objects and at different layers. The reference architecture treats them as separate but coordinated.

Token-level controls at the AI Gateway govern cost and capacity. The object of authorization is the token. The question being answered is whether a tenant's plan can fund a given inference call under current pool conditions. The mechanisms are token counting, quota enforcement, rate limiting with burst allowance, and pre-request estimation so that oversized prompts are rejected at admission rather than discovered after the fact.

The shared state across gateway replicas matters here: distributed enforcement is what prevents tenants from circumventing limits by spraying traffic across replicas. This domain is what makes the meter trustworthy.

TBAC at the MCP Gateway governs agent behavior. The object of authorization is the agent's intended action. The question being answered is whether a given agent identity, acting on behalf of a given user, is permitted to invoke a given tool with given parameters as part of a given task. The mechanisms are policy expressions that combine JWT claims with the MCP method, the tool name, and parameter values. This domain is what makes the audit defensible and the agent containable.

The two domains are coordinated by shared identity. The same JWT that authenticates the tenant at the API Gate can carry the claims that drive token accounting at the AI Gate and TBAC at the MCP Gate. The same observability fabric produces a unified record of token consumption and tool invocation against that identity.

Designs that try to answer both questions with a single mechanism produce a meter that underreports on agentic traffic (because tool calls are not counted) or an authorization model whose policy decisions are distorted by token economics (because budget state leaks into permission logic). The failure is one of conflated semantics, not of shared infrastructure. The two domains can and should run in the same gateway runtime, sharing identity, state, and observability. What the Triple Gate Pattern separates is the decision logic at each gate boundary, while unifying identity and observability underneath.

The AI Sovereignty Maturity Model and the TaaS Buyer's Journey

Buyers do not arrive at TaaS in the same state. The AI Sovereignty Maturity Model places them on a curve that maps to procurement reality. The model is organized around five levels of sovereignty and seven dimensions of architectural control, and it is intended to make the tradeoffs between levels explicit rather than collapse them under marketing labels like "data residency" or "hybrid cloud."

At the lower levels, organizations operate inside a vendor's control plane, sometimes with data in a chosen region but with governance configured in a console that cannot be exported. The litmus test of sovereignty at these levels is straightforward and usually fails: can the organization leave the vendor without rewriting the workload? At the middle levels, the data plane runs in the customer environment, but external dependencies remain, including licensing validation, telemetry, or cloud-hosted safety models.

At the highest levels, organizations fully host both their data and control planes on their own infrastructure. This provides complete operational independence, including support for air-gapped environments. Because governance is defined through code, teams gain the flexibility to move their entire stack between platforms without needing to rewrite any code.

The TaaS buyer's journey runs up this curve. A lower-level organization is a TaaS consumer dependent on a hyperscaler meter. A mid-level organization can begin operating a hybrid TaaS, but still depends on external services for parts of its governance. A higher-level organization can offer sovereign TaaS to internal lines of business under a defensible compliance posture, and ultimately to external customers, partners, or constituents under contract.

The Migrate, Modernize, Transform framework maps to this journey. Migrate replaces the ingress and runtime layer without rewriting workloads, taking the organization off legacy controllers and onto a unified runtime. Modernize introduces composable safety, identity, and observability across the Triple Gate, lifting the operating model from controlled consumer to hybrid operator. Transform brings TBAC, token-level entitlements, multi-tenant audit, and air-gap deployment online, producing a sovereign provider posture.

A TaaS reference architecture that does not articulate this journey leaves buyers stranded between pilot and production.

Deployment Topology

Substrate is the question most TaaS conversations start with. It is the wrong place to start. Before a provider chooses an integrated private cloud AI appliance, an open-source Kubernetes estate, or a sovereign hyperscaler region, it must answer a more fundamental question: where does the tenant boundary fall?

That question has three answers, and the answer determines blast radius, provider operability, regulatory posture, and whether the offering is a service or merely a set of installations. Topology and substrate are orthogonal. Any of the three topologies below can run on any of the substrate categories detailed in the next section.

Diagram of Three Architectural Patterns for Tenancy and Metering — *Diagram of the three deployment topologies. The Triple Gate is the constant across all three. Only the placement of the tenant boundary and the meter changes.*

Pattern A: Shared Tenancy

In the shared topology, one Triple Gate fronts one cluster and serves many tenants. The tenant boundary is logical. It lives inside the gateway and is enforced by identity, token-level quota, route policy, and TBAC. There is one meter, one policy fabric, and one audit store, all partitioned by tenant identity.

This is the topology with the lowest operating cost and the fastest path to revenue. The provider runs one stack. It is the natural starting point for NeoClouds monetizing GPU capacity and for enterprises turning a single AI factory into an internal chargeback for their lines of business.

It is also the topology with the highest blast radius. A noisy tenant, a misrouted policy, or a control plane fault affects every tenant on the system. The shared topology depends entirely on the gateway correctly enforcing isolation, which is why the token-level controls, distributed quota enforcement, and per-tenant cache isolation described earlier are not optional features in this topology. They are the tenant boundary.

The shared topology typically corresponds to Levels 3 and 4 of the AI Sovereignty Maturity Model.

Pattern B: Dedicated, Isolated

In the dedicated isolated topology, each tenant receives its own cluster, Triple Gate, and meter. The tenant boundary is physical. No plane is shared between tenants.

This is the topology that regulated and sovereign buyers usually force. Defense, intelligence, healthcare, and financial services tenants frequently cannot accept a logical boundary inside a shared gateway, regardless of how well that boundary is enforced. Physical isolation, up to and including air gap, is a procurement precondition rather than a preference. The blast radius is minimized because a fault in one tenant's stack cannot reach another.

The cost of the dedicated isolated topology is operational. The provider now runs N stacks, and N independent meters produce N independent views. Each installation is defensible on its own, but the provider lacks a consolidated view for billing, capacity planning, or cross-tenant compliance reporting. At small N, this is acceptable. At a meaningful N, it stops being a service and becomes a set of installations that happen to share a vendor.

The dedicated isolated topology typically corresponds to Level 4 of the AI Sovereignty Maturity Model.

Pattern C: Dedicated, Federated

In the dedicated federated topology, each tenant retains a physically isolated data plane, as in Pattern B, but the meter, policy fabric, and audit plane are federated into a single control plane operated by the provider. The data plane is dedicated. The control plane is shared.

This is the topology that resolves the tension Pattern B leaves open. The provider preserves physical tenant isolation, which satisfies the regulated buyer, while regaining a consolidated meter, a consolidated policy discipline, and a consolidated audit trail, which together make the offering a service the provider can bill, forecast, and govern at scale. It is the natural topology for sovereign and regulated TaaS offered to many tenants, for telco TaaS aggregated across metro edge sites, and for any provider whose tenant count is large enough that N disconnected installations is not a viable operating model.

Pattern C is also the topology that is genuinely hard to build, because federating a meter and a policy fabric across physically isolated, sometimes air-gapped, data planes is an architecture problem rather than a configuration option. A control plane that can federate without coupling the data planes, without creating a shared failure domain, and without compromising the air-gap posture of any single tenant is a differentiated capability. A substrate-neutral, model-neutral, GitOps-defined control plane is the prerequisite for it. This is the topology worth owning.

The dedicated federated topology typically spans Levels 4 and 5 of the AI Sovereignty Maturity Model, and is the topology a Level 5 sovereign provider operates.

Choosing a Topology

Dimension	Pattern A: Shared	Pattern B: Dedicated, isolated	Pattern C: Dedicated, federated
Tenant boundary	Logical, inside the gateway	Physical, no shared plane	Physical data plane, shared control plane
Meter	One meter, partitioned by identity	N independent meters	N data-plane meters, one federated meter
Blast radius	Highest	Lowest	Low
Provider operability	One stack to operate	N stacks, no consolidated view	N stacks, one consolidated view
Typical buyer	Neoclouds, internal LOB chargeback	Defense, isolated, regulated tenants	Sovereign and regulated TaaS at scale
Maturity Model	Level 3 to 4	Level 4	Level 4 to 5

The progression is not a maturity ladder in the sense that every provider should aim for Pattern C. A NeoCloud serving thousands of small tenants may operate Pattern A indefinitely. The point is that the topology is a deliberate decision with known trade-offs, and that a provider often operates more than one topology at once: Pattern A for its general tenant base and Pattern C for its regulated tenants. The Triple Gate is the constant across all three. Only the placement of the tenant boundary changes.

A note on consumers. The diagrams for these patterns are easy to draw with human users as the consumers, but a growing share of TaaS consumers are AI agents. An agent consumes tokens recursively through tool calls, and those tool calls are invisible to an API-only front door. In every topology, the MCP Gateway is the component that makes agent consumption visible and governable. A TaaS topology drawn with only an API Gateway is incomplete in the shared case and misleading in the dedicated cases.

Substrate Categories

The reference architecture is substrate-agnostic by design. The capability set is constant, and the topology is chosen independently. The deployment characteristics below vary with the class of infrastructure underneath, and each category can host any of the three topologies from the previous "Deployment Topology" section. The categories are described here in the abstract because the architectural argument does not depend on any specific vendor stack. A solution architect maps a given category to the specific platform a customer has standardized on.

Integrated Private Cloud AI Appliances

A pre-validated stack of accelerated compute, AI software, and management delivered as a single consumption-priced system. The Triple Gate runs in front of the appliance and turns it into a multi-tenant TaaS endpoint with zero external dependencies, which is the architectural prerequisite for intelligence, healthcare, and financial services workloads. This category is the natural home for the dedicated topologies of the previous section and for buyers who want to skip the hybrid phase entirely.

Sovereignty-Oriented Open-Source Kubernetes Estates

An open-source Linux and Kubernetes foundation, often paired with a VM platform and a central management layer, adopted specifically as a sovereignty posture. The Triple Gate runs as the ingress and control plane on the cluster, which means the same gateway that already handles application ingress handles the AI and agent surfaces, avoiding a second control plane. This category fits European sovereign operators and regulated enterprises particularly well.

Sovereign and Dedicated Hyperscaler Regions

A hyperscaler region operated under sovereign or dedicated terms with jurisdictional controls and a separate operational boundary. The Triple Gate runs inside the region as a tenant-facing control plane, turning hyperscaler-managed inference into a private TaaS for internal lines of business or regulated downstream customers. This category gives banks, telcos, and government buyers hyperscaler economics without a hyperscaler-controlled meter.

Dedicated AI Factory and NeoCloud Substrates

Purpose-built GPU infrastructure at thousands of accelerators, often spanning data center and edge sites, operated by a NeoCloud or by an enterprise running its own AI factory. The Triple Gate is the front door of the factory, the layer that turns raw GPU capacity into a sellable, metered, governed service. A multi-site factory grid is the dedicated federated topology of Pattern C expressed in hardware: a Triple Gate at each site, isolated data planes, and one federated control plane and meter spanning all of them.

Edge and Telco Fleets

A distributed fleet of small clusters at metro edge sites, often colocated with radio access infrastructure or with local data residency requirements. Tokens are produced close to the user, and metering must aggregate across many sites without losing per-tenant fidelity. The Triple Gate runs at each site with a federated control plane that consolidates metering, policy, and audit centrally while keeping the inference path local. This is the substrate for telco TaaS, sovereign distributed AI, and latency-sensitive industrial inference.

FIPS-Hardened Federal Deployments

Any of the categories above can be hardened for federal and high-regulation use by enabling FIPS 140-3 mode and sourcing every component from FIPS-validated, signed, and continuously rebuilt container images. This positions the architecture for federal procurements after September 2026, when FIPS 140-2 validated modules move to the CMVP Historical List and are no longer recommended for new federal systems. The architecture preserves its sovereignty and audit posture down to the supply chain.

The Meter Buyers Trust

The architectural argument for TaaS reduces to a single procurement question. Whose meter is this?

If the meter belongs to the model provider, the buyer cannot resell. If the meter belongs to the substrate operator, the buyer cannot switch substrates. If the meter belongs to the application, every application reinvents billing, audit, and compliance, and none of them agree.

The meter must live in a layer that:

The buyer can operate inside their own jurisdiction or air gap.
The seller can resell across multiple model providers and substrates.
The auditor can inspect without depending on either the buyer or the seller for the data.

That layer is the gateway plane. The Triple Gate Pattern, with token-level cost and quota control at the AI Gate and TBAC at the MCP Gate, is the architectural realization of that requirement.

There is a fourth property the meter must have, and it is the one most TaaS designs miss. The meter must be able to federate. A provider serving regulated tenants in the dedicated isolated topology (see "Deployment Topology" section) has a defensible meter per tenant and no defensible meter for the business. A meter that can consolidate data planes that are physically isolated, sometimes air-gapped, without coupling them is what turns a set of installations into a service. That capability is the dedicated federated topology, and it is the hardest of the three to build. It is therefore the one that separates a TaaS provider from a TaaS vendor.

The buyers who will dominate the TaaS economy over the next three years are those who build on a substrate-neutral, model-neutral, vendor-neutral control plane that can be deployed in whichever topology each tenant requires and federated across all of them. Every other path leads to a meter the buyer cannot trust, a topology the provider cannot operate at scale, or a contract the buyer will eventually want to renegotiate.

Implementation Checklist

The following checklist captures the minimum viable TaaS architecture. It is intended as a starting point for solution architects evaluating a build.

Identity and Tenancy

Per-tenant identity with federation to enterprise and external IdPs (OAuth 2.1, OIDC, JWT)
Per-tenant plan, quota, and entitlement resolution at the API Gate
On-Behalf-Of authentication for agent workflows, with JWT identity propagation across all three gates
Per-tenant API key and credential lifecycle

Token-Level Cost and Quota Control at the AI Gate

Token-level accounting for input, output, cached, and tool-call tokens
Burst-tolerant token rate limits and hard token quotas with pre-request estimation
Shared state across gateway replicas for distributed enforcement
Per-tenant, per-model, per-route attribution
Export to billing, finance, and SIEM systems

Model and Route Policy at the AI Gate

Multi-model and multi-provider routing with policy-driven steering
Multi-provider failover routing with tenant-aware fallback
Semantic caching with per-tenant cache isolation
Identity-based routing by user, role, or business unit

Composable Safety Pipeline at the AI Gate

Composable, multi-vendor safety pipeline with parallel guard execution
Topic control, jailbreak detection, content safety, PII protection
Support for guard models from multiple safety vendors within a single pipeline
Content Guard Regex Engine for custom deterministic patterns
Graceful error handling with structured refusals that do not break agent control flow

Agent and Tool Governance at the MCP Gate (TBAC)

Tasks: which workflows or sequences of tool calls are allowed
Tools: which specific MCP servers and tools each agent identity can invoke
Transactions: parameter-level constraints (amounts, channels, repositories, schemas)
Per-tool rate limiting and human-in-the-loop approval flows for high-risk actions
List filtering for tool discovery responses
Full per-invocation logging (agent identity, tool, parameters, outcome, timestamp)

Deployment Topology

Explicit selection of shared, dedicated isolated, or dedicated federated topology per tenant or tenant class
Logical tenant isolation inside the gateway for the shared topology, validated under contention
Physical tenant isolation, up to an air gap, for the dedicated topologies
Federated control plane and meter across isolated data planes for the dedicated federated topology
Ability to operate more than one topology concurrently from one architecture

Sovereignty and Deployment Posture

Deployment patterns across Private Cloud AI, sovereignty-oriented open-source Kubernetes substrates, sovereign hyperscaler regions, dedicated AI factory substrates, NeoCloud operators, and edge or telco fleets
Air-gap and FIPS 140-3 hardened image sources
Per-jurisdiction data residency and audit retention
GitOps-compatible declarative configuration with portable governance

Operability

One identity, policy, and telemetry fabric across all three gates
OpenTelemetry-compatible observability with no external phone-home dependency
Multi-cluster API federation for distributed deployments
Disaster recovery and multi-region failover

Conclusion

The hyperscalers will sell TaaS. Model providers will sell TaaS. Substrate operators will sell TaaS. Each of these offerings will arrive with a meter, and each meter will be operated by the seller. Buyers will accept this for the duration of the first contract and then push hard against it at renewal, because the lessons of every prior consumption-billed category, from electricity to bandwidth to cloud, are that buyers eventually demand a meter they trust.

TaaS is not a product category. It is an operating model. The organizations that will own the next decade of inference economics are the ones whose architecture treats the gateway plane as the meter, the agent authorization layer, and the trust boundary in a single coordinated runtime: deployable in whichever topology each tenant demands, federated across all of them, neutral to any model provider, and operable on any substrate inside any jurisdiction.

That is what the Triple Gate Pattern is for.

FAQs

What is Token as a Service (TaaS)?

Token as a Service is the operating model where an organization exposes AI inference as a metered, multi-tenant, governable service. NeoClouds, sovereign operators, enterprises, and telcos all expose TaaS in the same basic shape, even though the compute substrate underneath differs.

Is TaaS an inference problem or an architecture problem?

TaaS is an architecture problem, not an inference problem. Modern inference runtimes already handle throughput, batching, and KV cache locality. The hard part is tenant identity, admission control, token-level metering, model routing, guardrail composition, agent authorization, and audit.

What is the Triple Gate Pattern?

The Triple Gate Pattern is a control plane architecture with three coordinated gates: the API Gate for tenant authentication and plan resolution, the AI Gate for token-level cost control and model routing, and the MCP Gate for agent and tool authorization. All three share one identity, policy, and observability fabric.

What is TBAC (Tools/Tasks/Transactions-Based Access Control)?

TBAC is an authorization model for AI agents that governs access across three dimensions: which business objectives an agent can pursue (Tasks), which specific tools it can invoke (Tools), and what parameter-level constraints apply to each action (Transactions), such as monetary limits or allowed repositories.

How is TBAC different from token-level quota control?

Token-level controls at the AI Gateway govern cost and capacity by counting tokens and enforcing quotas. TBAC at the MCP Gateway governs agent behavior by authorizing specific tool invocations. They run on different objects (tokens versus agent actions), operate at different gates, and must stay coordinated through shared identity without being conflated into a single mechanism.

What are the three TaaS deployment topologies?

The three topologies are Shared Tenancy, where one Triple Gate serves many tenants behind a logical boundary; Dedicated, Isolated, where each tenant gets its own physically separate stack and meter; and Dedicated, Federated, where tenants keep isolated data planes but share one federated control plane and meter.

Which TaaS topology should a sovereign or regulated buyer choose?

Regulated and sovereign buyers, including defense, healthcare, and financial services tenants, typically require the Dedicated, Isolated or Dedicated, Federated topology because they need a physical tenant boundary rather than a logical one enforced only inside a shared gateway.

Why does a TaaS meter need to be independent of the model provider?

If the model provider or substrate operator owns the meter, the buyer cannot resell across providers, switch substrates, or fully trust the billing data. An independent meter at the gateway layer lets the buyer operate it inside their own jurisdiction and lets an auditor inspect it without depending on either party.

What is the AI Sovereignty Maturity Model?

The AI Sovereignty Maturity Model is a five-level framework describing an organization's progression from operating inside a vendor-controlled console to fully self-hosting both data and control planes with portable, code-defined governance across any substrate.

Why does agentic AI traffic require an MCP Gateway in addition to an AI Gateway?

AI agents consume tokens recursively through tool calls, and those tool calls are invisible to an API-only or LLM-only front door. A TaaS meter that only observes inference calls underreports agentic consumption. The MCP Gateway makes tool invocations visible and governable.

Latest from Traefik Labs

We Built Distro Zero, the Most Hardened Gateway Image Ever Shipped.

The Control Plane for Token as a Service: A Reference Architecture for Sovereign Inference at Scale

The EU Cyber Resilience Act and Your Infrastructure Layer