AI Governance in the Age of Generative AI

On a Thursday afternoon, internal audit asks the governance officer for the approval record, risk tier, owner, and enforcement evidence of a customer-facing generative AI assistant that went live in March. The product team shipped it under a pilot exception and never registered it.

EY’s 2026 survey of 202 senior AI decision-makers at large US public companies found that 47% admitted their organization had skipped its own AI governance process for an urgent deployment [1]. Closing that distance takes an AI governance framework with a registry, named owners, risk tiers, enforced policy, evaluation and monitoring, human oversight, and an audit trail generated while systems run.

What an AI Governance Framework Is

An AI governance framework is the set of policies, roles, processes, and controls an organization applies across the AI lifecycle so that every system in production can be accounted for, reviewed, and evidenced. ISO/IEC 42001 describes the equivalent structure as an AI management system, “a set of interrelated or interacting elements of an organization intended to establish policies and objectives, as well as processes to achieve those objectives, in relation to the responsible development, provision or use of AI systems” [2].

A policy is one artifact inside that structure; the framework adds owners and controls that verify it was applied. AI governance tools and platforms are narrower still: software that implements some of those controls and stores the evidence. A regulated enterprise can have a well-written responsible AI policy and still fail an examination, because the examiner wants the registry entry plus approval and enforcement records.

The Core Components of an AI Governance Framework

Each component below has to produce an artifact an examiner can review, and generative and agentic systems put more pressure on every one of them than a credit-scoring model does. Outputs are open-ended, the system acts rather than predicts, and the evidence is spread across prompts, retrieved context, tool calls, and responses.

AI Registry: Track Every System Through Retirement

The registry is a single view of systems throughout testing, live use, and retirement. It covers models and AI applications, including LLM applications and agents, with the owner, purpose, lifecycle status, and the regulatory frameworks each one must satisfy. NIST’s AI RMF places inventory inside its GOVERN function: “Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities” [3]. For a generative AI assistant, the record has to carry more than a model name. The foundation model, the system prompt version, the retrieval sources, and the tools the system can call each change behavior without any retraining event. AI lifecycle governance, the practice of applying controls at every stage from intake to retirement, extends the record past decommissioning. A retired assistant still has logs and a decision history that may carry a retention obligation, so its registry entry stays open and its owner stays named until those lapse.

Ownership: Name Four Accountable Owners per System

Every registered system needs four named people: a business owner accountable for the use case, a technical owner for the model configuration, including prompts and tools, a control owner for the policies and thresholds applied at inference time, and a compliance owner for the regulatory mapping and evidence package.

NIST’s AI RMF assigns the top of that chain explicitly: executive leadership “takes responsibility for decisions about risks associated with AI system development and deployment” [3]. A Chief AI Officer often sits between that leadership and the owners and coordinates use cases and risk processes across business units. A cross-functional governance committee drawn from risk, legal, IT, security, privacy, and the business lines reviews intake decisions and exceptions and records them as part of the governance process.

Risk Tiering: Scale Review Depth to Impact

Risk tiering means inventorying every system and scoring its impact to set review depth. The EU AI Act fixes its tiers in law, from prohibited practices through high-risk obligations to minimal-risk systems [4]. An internal summarization tool for meeting notes may need a registry entry and owner, with periodic review. A generative assistant that quotes coverage terms to insurance customers needs more before release: an impact assessment, pre-deployment evaluation, inline enforcement, a human approval path for consequential responses, and a retention plan for its logs. The tier assigned at intake is what justifies that difference to an examiner.

Policy Enforcement: Check Requests and Responses Inline

A policy prohibiting customer PII in third-party model calls is only an intention until a control checks each request. A governance team turns that policy into inference-time enforcement, which checks the request before the model is invoked and the response before it reaches the user or a downstream system. Each check can allow, block, or redact, depending on the policy and use case. Fiddler Guardrails, part of the Fiddler AI Observability and Security Platform, performs this work inline on the agent request and response path with under 80ms response time. It runs through the LLM or Model Context Protocol (MCP) gateway the organization already operates. Coverage includes prompt injection and jailbreaks alongside PII/PHI detection and redaction, and scoring runs in-environment without external LLM API calls. That inline enforcement is how the platform turns written policy into policy enforcement. The enforcement log it produces shows the policy was applied, and an examiner can review it alongside the policy document.

Evaluation and Monitoring: Carry Test Thresholds Into Production

Evaluation measures a generative system against defined cases before release: hallucination rate, faithfulness to retrieved context, bias across customer segments, and refusal behavior on prohibited requests. Governance teams use monitoring to measure the same behaviors on live traffic after release and receive alerts when they drift from the pre-deployment baseline. Neither substitutes for the other; NIST AI 800-4 states that pre-deployment evaluations “are predominantly done in controlled testing environments that cannot account for real-world dynamics” [5]. The same document groups production monitoring into six categories: functionality, operational, human factors, security, compliance, and large-scale impacts. In Fiddler, evals run custom evaluators through Tests & Experiments before release. Monitoring then scores production traffic against the thresholds those tests set, so governance teams can use a faithfulness threshold set before release as the alert condition after it. That continuity is what AI observability contributes to governance: a test result and a production record that reference the same metric.

Human Oversight: Set Escalation Triggers Before Actions

A governance team should attach oversight triggers to what the system is about to do. The EU AI Act requires oversight of high-risk systems “commensurate with the risks, level of autonomy and context of use,” including the ability to intervene or stop the system [4]. IMDA instructs organizations to deny by default when the approval infrastructure fails or no approval policy exists [6]. OWASP’s Top 10 for Agentic Applications names further triggers, including confirmation for destructive actions and a pause on unexpected goal shifts [7]. Typical escalation triggers a governance team can set, with each approval or override recorded against the named approver:

• Destructive action: The agent attempts an action such as delete, transfer, or publish.
• Scope breach: The agent attempts a tool call outside its authorized scope.
• Liability: The system is about to take an action that creates financial or legal liability.
• Repeated failure: The agent logs multiple failed tool calls in one session.

Audit Evidence: Record Identity, Policy, Action, and Time

The audit trail has to record the identity, applicable policy, action, and time for every consequential event. For high-risk systems, the EU AI Act requires automatic recording of events over the lifetime of the system [4]. In Grant Thornton’s 2026 survey of 950 business leaders, 78% lacked strong confidence that they could pass an independent AI governance audit within 90 days [8], a timeline that depends on evidence already being recorded rather than reconstructed. Continuous audit records support compliance evidence; they do not make a system compliant, and qualified legal and compliance counsel must validate the regulatory mapping.

Governing Generative and Agentic AI

Gen AI governance has to start with outputs. A generative assistant produces open-ended text shaped by the prompt, retrieved context, foundation model, and the model’s tendency to confabulate. By contrast, a predictive model commonly returns a bounded output that can be validated against labeled data.

An agent goes further by planning and deciding how to act across multiple steps and systems. OWASP’s agentic taxonomy names privilege escalation through delegation chains and the misuse of legitimate tools within authorized permissions. It also describes a single fault that “propagates across autonomous agents” as a cascading failure [7].

EY’s survey found that 49% of organizations using agentic AI had not updated their governance framework for agentic risks, and 26% cannot detect unauthorized agents operating internally [1].

IMDA’s authorization guidance extends the framework by requiring least-privilege permissions that are scoped and non-transferable. Those permissions must also be time- or session-bound. It also calls for explicit escalation paths for elevated permissions and a preference for deterministic access controls over prompt-based rules [6]. Two additional controls apply to agentic systems:

• Prompt provenance: OWASP’s guidance treats user text, documents, retrieved content, and tool outputs as untrusted, locks system prompts, and version-controls prompts with peer review, so the instruction history of a system can be reconstructed [7].
• Tool-call audit trails: The same guidance calls for “immutable logs of all tool invocations and parameter changes,” which is the record an examiner needs to establish what an agent did between the prompt and the response [7].

Agentic Observability connects those records by organizing telemetry across the agentic hierarchy of application, session, agent, trace, and span. A governance reviewer can move from an alert to the specific tool call or model response that caused it.

US banking regulators have, for now, placed these systems outside formal model risk guidance. SR 26-2, issued April 17, 2026, supersedes SR 11-7 and states that generative and agentic AI models “are not within the scope of this guidance,” while expecting a bank’s risk management and governance practices to determine appropriate controls [9]. Its principles still apply to traditional statistical and quantitative models plus non-generative, non-agentic AI models. With no supervisory checklist, each bank’s own program has to show its controls exist, a theme the Generative AI GRC webinar takes up.

How the Main Frameworks Fit Together

The frameworks a regulated enterprise will be asked about serve different purposes, and many programs combine them. The table below compares their legal status, certification, scope, and best fit.

Where to Start

1. Inventory every system: Record systems throughout testing, live use, and retirement, including models, LLM applications, agents, pilots, and vendor-embedded assistants, with purpose and lifecycle status. Discovery across cloud accounts and gateways can surface unregistered systems.
2. Assign owners: Name the business, technical, control, and compliance owner for each entry, and route exceptions through the governance committee on the record.
3. Tier by risk: Score each system by autonomy and combined customer or regulatory impact, and document the tier and its rationale at intake.
4. Enforce on the highest-risk use case first: Take the one system whose failure would be hardest to explain and write its policy as request and response checks. Verify the allow, block, and redact paths before extending the pattern to the next tier.
5. Review the evidence: Issue an internal audit request for that system and answer it from the registry, enforcement log, evaluation results, and approval records alone. Whatever cannot be produced is the next item on the backlog.

What an AI Governance Platform Adds

An AI governance platform is software that holds the registry, records policy enforcement decisions and their metadata, retains evaluation and monitoring results, and produces evidence packages from them. The category sits between a GRC system of record, which tracks obligations and controls at the organizational level, and after-the-fact dashboards, which record that something happened without connecting it to a policy or an owner. Governance teams use the platform to connect each system to its responsible owner and retain the applied policy with the recorded result.

The Fiddler platform keeps that registry with owners and lifecycle status, and links each entry to its evaluation, monitoring, and enforcement records.

The platform’s governance capabilities record each enforcement decision with the triggering identity and applicable policy. A leading healthcare payer reports 75% less time preparing for audits, now handled by automated trails. Governance teams can align that evidence to GDPR, HIPAA, NAIC, the EU AI Act, and Office of Management and Budget (OMB) memo M-26-04 by mapping each requirement to an owner, a control, and retained records. Alignment and evidence generation are not a guarantee of legal compliance.

Request a demo to review how the required controls would fit the governance program’s deployment environment.

References

[1] EY, EY Survey Finds That Autonomous AI Implementation Outpaces Oversight, Yielding an AI Governance Gap (September 15, 2026). https://www.ey.com/en_us/newsroom/2026/09/ey-survey-finds-that-autonomous-ai-implementation-outpaces-oversight-yielding-an-ai-governance-gap

[2] ISO, ISO/IEC 42001:2023 Information Technology, Artificial Intelligence, Management System. https://www.iso.org/standard/42001

[3] NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (January 2023). https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf

[4] EUR-Lex, Regulation (EU) 2024/1689 (Artificial Intelligence Act), Consolidated Text of 27 July 2026. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX%3A02024R1689-20260727

[5] NIST AI 800-4, Challenges to the Monitoring of Deployed AI Systems (March 2026). https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.800-4.pdf

[6] IMDA, Model AI Governance Framework for Agentic AI, Version 1.5 (May 2026). https://www.imda.gov.sg/-/media/imda/files/about/emerging-tech-and-research/artificial-intelligence/mgf-for-agentic-ai.pdf

[7] OWASP, OWASP Top 10 for Agentic Applications 2026 (December 2025). https://genai.owasp.org/download/52117

[8] Grant Thornton, 2026 AI Impact Survey Report. https://www.grantthornton.com/services/advisory-services/artificial-intelligence/2026-ai-impact-survey

[9] Federal Reserve, SR 26-2: Revised Guidance on Model Risk Management (April 17, 2026). https://www.federalreserve.gov/supervisionreg/srletters/sr2602.htm

[10] European Commission, Navigating the AI Act (Updated August 7, 2026). https://digital-strategy.ec.europa.eu/en/faqs/navigating-ai-act