AI Agent Control Plane: Why a Proxy Alone Cannot Govern Autonomous Agents

Earlier this year, Brex open-sourced a tool called CrabTrap. It is an LLM-as-a-judge HTTP proxy that intercepts every request an AI agent makes, evaluates it against a policy, and allows or blocks it in real time. [1] It picked up 602 GitHub stars in its first weeks, and enterprise teams took notice: if this is what governance looks like at the network layer, what does it look like at the system level?

Many teams already run an AI gateway or proxy in front of their models. It handles routing, keys, and cost tracking, and it feels like the natural place to add control. Then the agents start acting on their own: calling tools, pulling data, handing off to other agents. Read the AI Agents Need a Control Plane, Not Just a Proxy blog if you want a high level explanation.  

The question follows quickly: is the gateway enough? The short answer is that a proxy is necessary but not sufficient. It secures the pipe, but it does not govern what the agent decides to do with what flows through it.

What an AI Agent Control Plane Actually Governs

The clearest way to define an AI agent control plane is to borrow the distinction networking has used for decades. The control plane decides what is allowed. The data plane is where traffic and execution actually happen.

Routers move packets in the data plane; the control plane sets the routing rules those packets obey.

Apply that split to agents and it maps cleanly. Individual agents run in the data plane. They call models, invoke tools through MCP servers, and hand off tasks to other agents.

An AI control plane sits above that execution and sets the rules, permissions, and oversight for every agent at once. It is not watching one agent; it governs the fleet.

Why the Distinction Matters

The distinction is not academic. When you conflate the two planes, policy gets hard-coded into each agent, so every new agent reopens the same questions about permissions and oversight. Separating the control plane makes those rules a shared, enforceable layer rather than scattered configuration.

That reframes the core question. The relevant question is no longer how a single agent behaves. It is how every agent behaves as part of a larger system, where one agent's output becomes another's input and a single misstep can propagate.

The terms agent control plane, AI control plane, and agentic control plane all describe this same idea.

The Scale That Makes It Urgent

The scale makes the shift urgent. Agents now run as fleets across teams, embedded in day-to-day operations rather than sitting in isolated pilots. Governing that many autonomous actors one proxy rule at a time does not scale.

This control-plane model is where enterprise AI is headed. It is the vision behind Fiddler's AI Control Plane: see every action, understand every decision, control every outcome.

Why a Proxy or Gateway Cannot Govern Agent Behavior

Start with what a gateway does well, because it handles essential functions. A gateway or proxy handles

• Routing across models,
• Failover when a provider degrades
• API key management
• Rate limiting and cost tracking
• Structural access control: whether a given user or agent may call a given tool at all.

These functions are necessary, and you should not run production agents without them.

What the Gateway Cannot See

The boundary appears once you ask what the gateway can actually see. It processes prompts and completions as text and logs that a tool was invoked. It does not inspect the content that tool returns, and it has no representation of the decision the agent made with that content.

The gateway knows a search endpoint was called. It does not know the endpoint returned a customer record, or that the agent then acted on it.

The Inbound Path

That limitation shows up most clearly on the inbound path. Agents do not just send data out; they pull data in through MCP servers, WebFetch calls, and tool endpoints that can return PII or PHI directly into agent context. MCP, the open standard introduced to connect agents to external tools and data, has made this inbound path both common and standardized [2].

Existing security tooling was largely built for the outbound direction, so this inbound vector is the less-examined risk. It is structurally different from deciding whether an agent may call a tool, and it introduces threats that live in the tool layer rather than the access list [3], [4].

Binary Controls vs. Semantic Threats

This is the split between binary identity controls and semantic, content-level threats. A gateway answers a yes-or-no question: can this agent call this tool?

It cannot answer the harder ones. What did the tool actually return, did that response carry a prompt injection payload, and did the agent leak a record into its next call? Those threats live in content and behavior, not in access lists [5].

The Multi-Agent Multiplier

The problem multiplies in multi-agent systems. When one agent's tool response becomes another agent's instruction, a malicious payload does not stay contained to the call that received it. It travels.

A gateway logging each hop as an authorized request reports a clean audit trail even as an injection propagates, because every individual call was permitted. Permission was never the question.

Observability Must Precede Autonomy

This is the core argument. Observability must precede autonomy. You cannot give agents more autonomy than your ability to oversee them, and a proxy does not provide that oversight [6].

Closing this vulnerability takes inline enforcement on the agent's request and response path, applying allow, block, and redact verdicts before data leaves the network. That is a governance function, not a routing one.

The gateway can point traffic at the right model. It cannot judge whether the model should have acted on what came back.

The Capabilities That Separate Control From Connectivity

If a proxy governs connectivity, a control plane governs behavior. In practice that resolves into five capability areas: Standardized Telemetry, Reliable Evaluation, Continuous Monitoring, Enforceable Policy, and Auditable Governance. Each addresses a distinct part of the problem.

1. Standardized Telemetry

Standardized Telemetry is the foundation. It captures full execution context and decision lineage across multi-step tool and model calls. A team can then reconstruct what an agent did, why it did it, and what happened next; without a consistent trace format, every downstream capability is guessing.

2. Reliable Evaluation

Reliable Evaluation is the capability teams most often underinvest in. You evaluate an agent before it ships, then keep evaluating the same way in production, so what you measured pre-launch is what you monitor after. Fiddler Centor Models (formerly Fiddler Trust Models) make this practical: batteries-included, in-environment evaluation with no external API calls, no per-evaluation cost, and under 100ms response time.

The cost dimension matters here. The Evaluation Trust Tax is the per-call cost enterprises incur when external LLMs are used for evaluation. It shows up on the customer's own LLM provider bill, and it scales with every evaluated trace.

Rather than quote figures that shift with model pricing, teams can model their own deployment with the Evaluation TCO Calculator; results vary by model, deployment size, and traffic volume.

3. Continuous Monitoring

Continuous Monitoring watches every decision in production instead of sampling a slice of traffic. Full coverage matters because the rare trace is often the one that fails. Down-sampling to save money is how drift and regressions slip through unnoticed.

Agent behavior also changes as upstream models, prompts, and tool schemas shift, so a snapshot taken at launch stops describing reality within weeks.

4. Enforceable Policy

Enforceable Policy is where oversight becomes action. It runs inline and pre-execution, blocking or redacting before an action fires, rather than raising an alert after the damage is done. An after-the-fact log tells you a record leaked; a pre-execution verdict stops the leak.

5. Auditable Governance

Auditable Governance ties it together as enterprise-wide visibility and control over all AI. It is more than logs. It includes an AI registry: a single view of live, testing, and retired agents and applications across the organization.

That registry maps to frameworks like SOC 2, ISO 42001, and the NIST AI Risk Management Framework. Governance is the primary lens here; security compliance is one part of it.

Spanning Both Layers

These capabilities have to span the full range of production agents: first-party agents your teams build, third-party agents you deploy, and coding agents running across your developer organization.

At the production layer, that means first-party and third-party agents, LLM applications, and predictive AI deployments.

At the creation layer, coding agents introduce a distinct threat surface. Tools like Cursor, GitHub Copilot, and Claude Code operate with broad file system access, call external APIs through MCP servers, and execute actions without human review. The threat vectors are concrete:

• Prompt injection delivered through MCP tool responses
• Unauthorized exfiltration of credentials or source code in generated output
• Behavioral misalignment where an agent reasons around policy constraints rather than complying with them

Governing this layer requires coding agent observability, policy enforcement at the IDE/CLI/MCP boundary, and agent posture management: continuous assessment of agent configurations, permissions, and policy compliance across endpoints. The Fiddler AI Observability and Security platform is the only platform that spans both layers, from code generation through production.

Deciding Between a Gateway, a Control Plane, or Both

Most teams reach for a gateway first, and that is the right instinct. You adopt it where you already see problems: unpredictable model cost, or tool calls no one is auditing. A gateway addresses that directly.

The mistake is treating the next step as a replacement decision. A gateway and a control plane are not competing choices. The control plane governs behavior across every gateway, harness, and agent you run, while the gateway keeps the traffic flowing underneath it.

One operates on packets and permissions. The other operates on decisions and outcomes.

Here is the practical guidance:

• If your needs stop at routing, cost control, and access control, a gateway may be enough for now.
• Once agents take autonomous actions with real permissions, you need observation, evaluation, and enforceable policy that a proxy cannot provide.
• That threshold, not team size, is what drives the decision.

A useful test: ask whether you could reconstruct, after an incident, exactly what an agent did and why. If your gateway logs can only tell you which endpoints were called and in what order, you have connectivity records, not behavioral oversight. That is the line where a control plane stops being optional.

One more criterion belongs in the decision: A well-designed control plane works across providers such as Azure OpenAI, Amazon Bedrock, and Google Gemini, rather than locking you into one stack. Fiddler is framework, model, and cloud agnostic by design, so the governance layer does not become the thing that dictates your architecture.

Agent Governance Failure Modes in Production

Deploying agent governance introduces its own failure modes. These are the ones we encounter most often.

• Policy drift across environments: Teams define policies in staging and forget to propagate changes to production. When governance runs through disconnected tools, there is no single source of truth for policy state.
• Evaluation sampling that creates blind spots: When evaluation costs are high, teams down-sample. They evaluate 10% of traces and extrapolate. Hallucinations in financial calculations and prompt injection attempts occur at low frequencies. A 10% sample can miss them entirely.
• Audit trails that satisfy engineers but not regulators: Logging requests and responses is not sufficient. Regulators in financial services expect tamper-evident records that include the policy version in effect and the authorization context.
• Governance that only covers first-party agents: Third-party agents and coding agents operate in your environment but outside your development lifecycle. Governance that only instruments first-party agents leaves the fastest-growing categories unmonitored.
• Framework lock-in disguised as governance: Some tools require adopting a specific agent framework to get governance features. A control plane must be framework-independent to be useful across an enterprise.

From Traffic Control to Behavioral Governance

A proxy keeps agents connected and secure at the traffic layer, and that is a necessary function. Governing autonomous behavior is a different job. It requires a control plane that observes every action, evaluates every decision, and enforces policy before damage is done. Observability must precede autonomy, because you cannot grant an agent more freedom than you can oversee.

The enterprises moving fastest on agents are the ones that treat governance as infrastructure, not an afterthought. The governance patterns being set now will define how AI operates at scale for the next decade. If your team is deploying agents and running into governance blind spots, request a demo to see how Fiddler governs agents in production.

References

[1] Brex, "Building CrabTrap," Brex Journal, 2026. [Online]. Available: https://www.brex.com/journal/building-crabtrap-open-source

[2] Anthropic, "Introducing the Model Context Protocol," Anthropic, Nov. 25, 2024. [Online]. Available: https://www.anthropic.com/news/model-context-protocol

[3] X. Hou, Y. Zhao, S. Wang, and H. Wang, "Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions," arXiv preprint arXiv:2503.23278, Mar. 2025. [Online]. Available: https://arxiv.org/abs/2503.23278

[4] M. M. Hasan, H. Li, E. Fallahzadeh, G. K. Rajbahadur, B. Adams, and A. E. Hassan, "Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers," arXiv preprint arXiv:2506.13538, Jun. 2025. [Online]. Available: https://arxiv.org/abs/2506.13538

[5] S. Willison, "Prompt injection," Simon Willison's Weblog, 2022-2025. [Online]. Available: https://simonwillison.net/series/prompt-injection/

[6] G. Huckins, "Are we ready to hand AI agents the keys?," MIT Technology Review, Jun. 12, 2025. [Online]. Available: https://www.technologyreview.com/2025/06/12/1118189/ai-agents-manus-control-autonomy-operator-openai/