Building Reliable Agents Takes More Than Better Models

AI agents are taking on work that would have been out of reach a year ago. Tasks that once meant a single prompt and a single response now stretch across long, multi-step workflows, the kind of thing that used to take a human analyst most of a week. Research from METR puts numbers to the shift: the length of task a frontier model can complete on its own has been roughly doubling every seven months [1], and that pace has picked up more recently. The models underneath have gotten good enough that the ceiling on what an agent can attempt keeps rising.

And yet agents still break in production, often in ways their builders didn't see coming. Part of the reason is an assumption that quietly runs through a lot of AI work: that a more capable model gets you a more reliable agent. It doesn't. A model can be genuinely strong, able to reason through hard problems and produce work that looks finished, and still fall apart the moment it has to act on its own in a real environment, repeatedly, without someone checking each step. Reliability is a different property than capability, and it comes from somewhere else.

A recent Fiddler AI Explained session dug into that difference, drawing on a guest who works at the seam between frontier model development and real-world deployment: building the hard datasets and reinforcement learning environments that expose where models fail, then watching what happens when those same models get connected to real tools, data, permissions, and business processes. The conversation walked through why smarter models haven't made agents reliable, and where the reliability actually comes from.

A Capable Model Is Not a Reliable Agent

The heart of it is that a model predicts, but an agent has to act. Model training optimizes across a broad distribution of tasks. Agent reliability is a different target: the system has to work in one specific environment, repeatedly, and hold up under the messy conditions of real use.

That reframes the question teams should be asking before they ship. A benchmark asks whether the model can solve a given problem. A production environment asks whether the system can solve it repeatedly and fail safely when it can't. That second half is the part most teams underweight. Failing safely is not a model property. It's something you have to design for.

The divide shouldn't be overstated. A stronger model does make the job easier, because it starts to behave like a competent actor with less scaffolding around it. But for complex agent deployments, the engineering doesn't disappear. Building an agent that works reliably is a system design problem and an architecture problem, not only a research problem, and the two require different skills. Assuming that solving one solves the other is where a lot of AI budgets quietly go sideways.

The Harness Does the Work the Model Can't

If the model is the reasoning engine, the harness is everything that turns it into something operational. That includes the pieces that surround the model: system instructions, context assembly, memory, execution loops, guardrails, sandboxes, and the points where the agent should stop and ask a human. The model is the brain. The harness is what the model can do, what it can see, how anyone knows what the model actually did, and what happens when it fails.

Each component is a place where reliability is either engineered in or left to chance. Context assembly is a good example of how much this matters, and how easy it is to get wrong. One lesson that came out of building agent tool interfaces at Fiddler: you can't take an existing web API and pass it straight through to the agent as a thin wrapper. Web APIs are built for other purposes and tend to return far more than an agent needs. Hand all of it back and the excess floods the context window, skewing the agent's reasoning and pushing it off the task it was actually given. The interface an agent uses has to be designed for the agent, not inherited from whatever the API already returned.

The harness is also where the "fail safely" requirement gets satisfied. Guardrails, sandboxes, and escalation points are what let a system stop cleanly when the model reaches the edge of what it can handle, instead of confidently doing the wrong thing. Knowing when to hand back to a human is the part that genuinely needs human judgment, and it's the hardest part to automate away.

Where Agents Actually Break

When a well-built agent meets the real world, failures cluster in a few predictable places:

• Tool calls: Sometimes the agent picks the wrong tool. More often it picks the right tool for the wrong reason, calls it with an argument it should never have used, or invokes something it had no business touching. This is where guardrails earn their place, and it maps directly to the context assembly problem above: a lot of tool-call failure traces back to how the tools were exposed to the agent in the first place.
• Consistency across the long tail: An agent can be accurate 80 percent of the time and wrong the other 20, and that 20 rarely spreads evenly. It clusters around specific users, edge cases, and long-tail scenarios that never got tested. Teams optimize for the majority path, the way product builders always have, and the failures hide in the cases the Pareto principle told them to deprioritize. Running an agent repeatedly against representative long-tail and adversarial cases, with real permissions and integrations in play, is how those clusters surface before customers find them.
• Permissions and output format: User authorization turns into a blocker in production in ways it never did in testing. And output format, whether the agent returns something usable and well-structured or something that needs a human to clean it up, is a frequent and underrated failure point.

Two deployment habits make all of this worse.

• The first is automating an agent before instrumenting it. Teams move fast, wire the agent up, and skip building the observability and controls around it first.
• The second is testing only the happy path. Teams validate the scenario they expect, skip the harder cases, and then get surprised by exactly the edge cases they never checked.

Ultimately reliability comes from the system, not the model. The organizations that scale agents with confidence are the ones who can expand agentic deployments safely as their oversight improves, catching failures in testing rather than in front of customers, and widening what an agent is trusted to do as the evidence supports it.

This post draws on a conversation with Juhi Parekh, GM of Key Frontier AGI Accounts at Turing, from Fiddler's AI Explained AMA series. Watch AI Explained webinars on demand.

References

[1] Kwa, T., West, B., et al. "Measuring AI Ability to Complete Long Tasks." METR, March 2025. https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/