Governance Is What Lets Enterprises Move Faster on AI

A bank puts a customer care agent in front of its end users. A bad actor comes in, runs a jailbreak, and something goes wrong. The incident itself is recoverable. The harder question: who is accountable for it?

There are at least five candidates: the bad actor, the business unit that decided to expose the agent to customers, the provider of the agent (if it came from a third party), the model provider that trained the model powering it, or the tools the agent called along the way. Ask that question inside most enterprises and the answers don't match, which is why accountability keeps ranking as the number one blocker to agent adoption, above cost and above performance.

The reason this feels newer than it is has to do with how fast the last three years moved, especially as the rush after ChatGPT landed on a narrow set of use cases like information extraction, summarization, question answering over internal documents, and code generation. While teams with a scenario that fit were in good shape, everyone else spent a year trying to prove ROI on scenarios that didn't, until models started taking actions. Once agents looked like a way to apply those same productivity gains to every part of the organization through API calls, boards started asking for agents in production by year end.

The promise held up, and the bottleneck moved. What stalls adoption now is autonomy, and whether the instrumentation, guardrails, and processes exist to handle the ways agents can go wrong.

Governance is the answer, though not in the sense that usually gets people's attention. A useful analogy is a race car, where the brake is not there to slow the car down, but rather to let the driver corner harder, stay on the track, and finish ahead of the competition.

The gap between what enterprises have deployed and what they can prove is already wide. Roughly 78% of business executives lack strong confidence they could pass an independent AI governance audit within 90 days [1], and many already have agents running in production. By one estimate, a $20 billion enterprise with weak governance loses about $70 million a year, not to random errors or unavoidable glitches, but to preventable oversight [2].

Governance, risk, and compliance (GRC) programs have been running for 20+ years, but the nature of the risk changes with each wave of technology and the foundation underneath it does not, which is why the durable version of AI governance rests on three things in order: visibility, control, and accountability.

An AI Control Plane Is What Makes Governance Operational

An AI control plane is the system that delivers standardized telemetry, reliable tests and experiments, continuous monitoring, enforceable policy, and auditable governance across the AI lifecycle, from code generation through production. It sits above applications, models, agents, tools, and enterprise data, and below business outcomes, acting as the system of trust for agents.

The relationship to governance is straightforward. Governance defines what has to be true. The control plane is the machinery that makes it true and proves it stayed true. That machinery runs as a loop with four parts:

1. Define the controls, drawing from corporate policy, enterprise AI risk, regulatory requirements, and operational concerns like token cost.
2. Implement them, either out of the box or with AI-assisted coding, so a requirement like masking sensitive employee data in an HR workflow becomes a working guardrail.
3. Enforce them wherever the workload runs, not only where it was built.
4. Track enforcement, measuring whether the control worked, what the business impact was, and whether the threshold needs adjusting.

Most programs execute the first three and stop. The fourth is what closes the loop, and skipping it leaves teams unable to say whether a control helped, hurt, or did nothing at all.

The loop only works when the three layers underneath it are in place.

Nobody Can Govern an Agent They Cannot See

Visibility is what makes exposure assessable. Without it, every downstream control is guesswork.

Shadow AI Is the Starting Problem

Shadow AI is common enough now to have a name. Builders ship faster than the organization can catalog, let alone govern, and the honest answer to whether a given workload can be trusted is often no, because nobody can say what the workload is. Discovery comes before any control worth writing.

Operational Risk Categories Carry Over

Most AI governance approaches fall short here, treating AI risk in isolation from enterprise operations. Two categories transfer directly:

1. Business continuity: The discipline used to cover earthquakes, fires, and backup data centers. The AI version asks what happens to the business when a model provider's API goes down for two minutes, and whether a fallback exists.
2. IT risk: Same categories, new assets, and the same requirement to connect them to the AI portfolio.

Connecting those categories to AI assets is what turns a basic inventory into an actual picture of exposure.

Controls Only Exist Where a Metric Is Watching

Enforcement requires observability. Without knowing what is running and what it is doing, there is no way to know what is being enforced.

Every risk is mitigated against a control, and every control is measured by a metric. That chain is what turns a policy document into something operational. A control with no metric behind it, and no tooling watching that metric, exists on paper.

Where Controls Come From

Controls come from four places:

• Corporate policy
• The enterprise AI risk the organization is exposed to
• Regulatory requirements the market defines
• Operational concerns like infrastructure and token cost

Token cost often gets overlooked, because operational concerns don't read as risk in the compliance sense. An agent burning through more tokens than its budget allows is still a control problem, and stopping it before the bill arrives is still enforcement.

Where Enforcement Lives Depends on the Layer

A risk and compliance officer or a CISO cares about the whole stack (application layer through AI layer through infrastructure) but the implementation lands in different places. Security risk categories get enforced inside security tooling. A requirement that sensitive information never surfaces in an output is an AI-layer concern, enforced and verified through tracing and monitoring. The practical target is full coverage across the stack, likely across more than one platform, feeding a single view of risk and control status.

Governance Starts at the Idea, Not the Review

The old model was a submitted use case, a review, an approval, then a build, and it assumed roadmaps measured in quarters. Now everyone is a builder, and governance needs to be present from the first sentence someone types into an AI-assisted development environment. At that moment the idea can be checked against use cases already approved. If one covers the same ground, the builder gets a short list of guardrails and a green light. If the use case is high risk, risk and compliance gets flagged before a line of code is written. Builders tend to prefer this, because it takes the risk analysis off their plate.

Agent Behavior Needs Evaluation in Three Places

Evaluating only the final output misses most of what an agent does. The three places that matter:

1. Offline at build time, where published benchmarks help when selecting a third-party agent.
2. Continuously in production, watching for behavioral drift the way predictive AI models get watched for decay.
3. At every step inside a run, because an agent breaks a request into steps and decides what to call at each one.

Tracing alone won't cover it, since a trace describes what already happened. If the failure was leaked information, the information is gone by the time the trace confirms it, so the work has to include hunting for patterns and enforcing policy before the action executes. The goal behind each evaluation matters too, because confirming a guardrail is working and optimizing for cost, latency, or prompt quality need different metrics. The most useful question for an AI leader is not which metric to track, but whether the organization has all the metrics its controls actually require.

Someone Has to Own the Agent When It Goes Wrong

Closing the loop means connecting controls to outcomes. AI assets connect to use cases, use cases expose risk, risk is mitigated by controls, controls are measured against metrics, and metrics prove a business objective was met.

That last link is the one almost everyone drops, and it does two jobs. It gives leadership a running view of whether AI assets are delivering what they were funded to deliver. It also means a breach traces in both directions, from the metric that failed on a monitoring dashboard back to the specific KPI and financial exposure it touched.

Who Is Accountable Today

The market has not settled this question. Lawyers, regulators, and business owners give different answers, and enforcement varies by country. An industry is forming around the gap, including startups that document an enterprise's model selection decisions as legal evidence that options were weighed at a point in time.

The working answer for now is narrower. Whoever deploys an agent owns what it does, because the agent inherits that individual's access controls. What the enterprise can do is build the tracing instrumentation to reconstruct where a failure originated, which makes internal accountability provable even while external accountability stays unresolved. Enforcing anything against a third-party model or agent provider waits on regulation to catch up.

“Huma-in-the-loop” describes a person watching from the side, stepping in when something looks wrong. As routine execution shifts to agents, the human role moves toward orchestration: directing what runs, reviewing what comes back, and owning the result. “Human-in-the-lead” is the more accurate description, and it matches where accountability actually sits.

Three Lines of Defense Still Apply

The structure already exists in most regulated enterprises:

• The business owner: accountable for the workload and its controls.
• Risk and compliance (GRC): defining the controls under which the business operates.
• Audit: verifying the controls worked.

The second and third lines need access to the entire AI portfolio regardless of which department a workload sits in, while each business unit needs control over its own realm. That split is what lets governance scale across departments adopting agents independently, without a central team becoming the bottleneck.

Agents Changed the Stack, but Predictive AI Still Runs Underneath It

The attention has moved to agentic systems, but the enterprise AI portfolio didn't get smaller. Forecasting demand, ranking what a customer sees, scoring a transaction for fraud, and classifying a claim are predictive AI problems, and no amount of agentic architecture changes that. Those workloads are still in production, and they still carry business-critical decisions.

That matters for governance in one specific way. A risk and compliance officer asking what AI is running in the business needs one answer covering agentic, generative, and predictive AI, not three answers from three tools. Predictive AI models drift, generative applications hallucinate, and agents take actions. Different failure modes that share the same requirement: seeing them, controlling them, and proving the controls worked.

This post draws on a conversation with Maryam Ashoori, VP of Product and Engineering at IBM watsonx.governance, from Fiddler's AI Explained AMA series. Watch AI Explained webinars on demand.

References

[1] Grant Thornton. "2026 AI Impact Survey Report." April 2026. https://www.grantthornton.com/services/advisory-services/artificial-intelligence/2026-ai-impact-survey

[2] Fiddler AI. "Governing AI That Keeps Evolving With Maryam Ashoori." AI Explained Podcast, Episode 23. https://www.fiddler.ai/podcasts/governing-ai-keeps-evolving-with-maryam-ashoori