Why AI Defense Has to Become Autonomous

Every year at the National Collegiate Cyber Defense Competition, students across the United States and Canada defend a network against professional red teamers in a compressed simulation of a live cyber attack. When those student defenders turned to AI models for help, asking questions like "is this a piece of malware?", the models refused roughly 70% of the time. Stranger still, when the students explained they were authorized blue teamers who were allowed to ask, the models became 2.7 times more likely to refuse [1].

That finding, published at ICLR as the Defensive Refusal Bias study, quantifies something security practitioners have long suspected. The people saying "I'm the good guy, help me" get turned away, while attackers have unlimited shots at the goal.

In a recent AI Explained, David Campbell, founding member of OWASP and author of that study, walked through what this asymmetry means for enterprises deploying agents, why defense has to become autonomous to keep up, and how the Behavior, Identity, and Control framework gives teams a way to govern agents in production.

Attackers Now Work at Machine Speed

Sophisticated attacks used to require nation-state resources. Stuxnet took an estimated 18-plus months and over a hundred people to pull off. Today, a determined actor doesn't need to know Python to execute an idea. They describe the goal, and AI formulates and executes it. Phishing shows the same shift. The telltale typos are gone, and even unsophisticated operations can now customize millions of emails per day using open source intelligence, at a fraction of a penny each.

The math gets worse on the defensive side. An AI-powered audit against a decade-old code base can return a hundred thousand findings, and triaging that volume is a problem CISOs never faced when an audit surfaced four criticals and a hundred mediums. Attackers can tolerate false positives. Defenders drown in them.

His conclusion after four years of adversarial AI red teaming is that the model was never the whole problem. "AI security is cybersecurity," he argued. The distribution inside the model is largely in the hands of frontier vendors. What enterprises control is the system around it, and that's where enterprises fall short: missing agent telemetry, missing boundaries, missing visibility into what agents are doing with the privilege they've been granted.

What Autonomous Defense Looks Like

Offense has always held the advantages of time and stealth, and AI has widened both. The only way to level the playing field, in his view, is defense that operates at the same speed as the attacks: systems that mitigate low-hanging risks on their own, recognize anomalous behavior, and escalate to the right team or person to accelerate mitigation. He expects autonomous defense to arrive within three to five years.

Guardrails still matter in that world, as one necessary layer of the stack. The conversation landed on an analogy worth keeping: guardrails are a seat belt, not the whole car. A seat belt doesn't stop the driver from speeding, but it protects them when something goes wrong. The rest of the car, the identity and control layers around the model, is what keeps an agent on the road. He proved the limits of relying on any single layer in the field, breaking into an agentic deployment not by defeating the model but by defeating the guardrails around it.

Autonomous defense without governance carries its own risks, though. An agent granted broad privilege at machine speed can do damage at machine speed, which is why the governance layer has to be designed alongside the defense itself.

Governing Agents With Behavior, Identity, and Control

The BIC framework, developed with his co-author on a forthcoming O'Reilly book on AI security engineering, breaks agent governance into three layers:

• Behavior is everything within the model: what it can generate, the skills files, the harness it runs in. This is where alignment and red teaming live, and it's largely shaped by frontier model vendors rather than enterprise teams.
• Identity is the authority and context granted to the agent: authentication, authorization, secrets, and how long access lasts. Agents look like humans but move at machine speed, so identity has to be ephemeral. Grant a human access to an S3 bucket and the work might happen in three days. Grant an agent access and it happens instantly, after which the access is no longer needed.
• Control is everything else: the agentic observability stack, the deployment harness, the runtime environment. This is the layer that makes it possible to replay what an agent did and why.

The framework's value is diagnostic. A model explaining how to build a weapon is a behavior failure. A model writing to disk where it shouldn't is an identity failure. A deployment that can't be reconstructed after the fact is a control failure. Each layer points to a different fix and, often, a different owner.

For smaller teams that can't invest in all three at once, his recommendation was unambiguous. Start with control. Without visibility into what happened, there is no protecting anything and no mitigating risk. Visibility is also how shadow AI gets found. Employees are already wiring agents to internal tools without a security review, and he named it one of the biggest agent security risks nobody is preparing for. And it's already common: a recent DigiCert survey found that nearly half of organizations, 47%, don't trace their agents at all today [2].

The through line of the conversation is that the unglamorous fundamentals, least privilege, short-lived access, audit trails, observability, are what carry over from three decades of cybersecurity into the agentic era. The systems are new. The discipline isn't.

This post draws on a conversation with David Campbell, founding member of OWASP, from Fiddler's AI Explained AMA series. Watch AI Explained webinars on demand.

References

[1] D. Campbell, N. Kale, U. M. Sehwag, B. Herring, C. Q. Knight, D. Borges, N. Price, and A. Levinson, "Defensive refusal bias: How safety alignment fails cyber defenders," arXiv preprint arXiv:2603.01246, Mar. 2026. [Online]. Available: https://arxiv.org/abs/2603.01246

[2] DigiCert, "Latest DigiCert research shows AI security risks already hitting enterprises, with 78% reporting incidents," DigiCert, Inc., Lehi, UT, USA, Press Release, Jun. 25, 2026. [Online]. Available: https://www.digicert.com/news/latest-digicert-research-shows-ai-security-risks-already-hitting-enterprises-with-78-Reporting-Incidents