Key Takeaways
- Continuous evaluation for AI agents runs scorers automatically on live production traces, unlike offline test sets that only cover pre-release scenarios.
- Trace classification is the routing layer that decides which scorer runs on which trace, so teams cover every interaction without wasting compute.
- Only reference-free scorers can run continuously in production, and their cost and latency determine whether full trace coverage is affordable.
Production Traffic Breaks What Offline Test Sets Never See
A financial-services support agent passes every case in its offline test suite. The team ships it with confidence.
Within a week, live traffic surfaces a class of refund questions the suite never covered. The agent quotes the wrong policy, and users notice before the team does.
This is the failure mode that continuous evaluation for AI agents is built to catch. Offline evaluation covers the paths you thought to test. Production surfaces the inputs, tool calls, and edge cases you did not.
In practice, teams often learn about these failures from user reports, after multiple users are already affected. Continuous evaluation for AI agents runs automated scorers against live production interactions in near real time [1]. It is the online counterpart to offline testing.
You do not wait for a release cycle. You score every interaction as it happens.
The distance between the two stacks is where risk lives. A test suite holds a few hundred curated cases. A production agent can handle 12K unscripted requests a day.
Each request is a new combination of intent, context, and tool state. No fixed suite anticipates that distribution. Our agent evaluation guide covers the fundamentals; this piece focuses on the routing layer that makes coverage affordable.
Sampling Misses Incidents and Generic Scorers Add Noise
Two distinctions decide what continuous evaluation for AI agents can run in production.
First is offline versus online evaluation. Offline evaluation runs against a fixed test set before release. Online evaluation runs against live traffic after release.
Second is supervised versus unsupervised evaluation. Supervised evals need a known correct answer. They cannot run on every live interaction.
Unsupervised evals, also called reference-free, score a trace without ground truth. Reference-free metrics suit open-ended tasks where no single correct answer exists, which is exactly the case in production where ground truth is unavailable [2]. Only reference-free scorers can run continuously.
The reason is structural. A supervised eval compares an output against a reference answer. For an arbitrary user question in production, no reference answer exists yet.
You would have to label it after the fact, which defeats near real-time scoring. Even so, two common approaches still leave you exposed.
- Down-sampling: Scoring a fraction of traffic to control cost means incidents on unsampled traces go undetected.
- Generic scorers: A single vague quality prompt produces noisy, inconsistent judgments; we recommend specific, binary scores.
Down-sampling looks like prudent cost control. In practice it is a false economy. The traces you skip are exactly the ones a user reports later.
Without a routing layer, you face a bad choice. Run every scorer on every trace, which is expensive. Or run too few, which leaves blind spots.
Production agents need evaluation systems built for their tasks, not one-size-fits-all scoring [3].
Trace Classification Routes the Right Scorer to the Right Trace
Trace classification is the routing layer at the heart of continuous evaluation for AI agents. It decides which scorer runs on which trace.
What a Trace Classification Actually Is
Three terms are easy to confuse, so define them first.
- Trace: The full record of one agent execution, from input to final output.
- Span: One step inside a trace, such as a single tool call or model completion.
- Trace classification: A label assigned to a trace that determines which scorers apply to it.
A trace classification is not a score. It is routing metadata.
Classifications can come from existing signals, such as the tool used, the route taken, or the customer tier. They can also come from a lightweight classifier scorer that reads the trace.
Classification happens at ingestion, before any scorer runs. The cleanest source is metadata your agent already emits. When metadata is ambiguous, a small classifier scorer can label the trace, at the cost of one inference call per trace.
Return to the refund agent for a concrete example. A policy-lookup trace retrieves context, so it belongs to the RAG class.
A refund-status trace calls an order API, so it belongs to the tool-call class. Each class earns a different scorer set.
Mapping Trace Classes to Scorers
The naive pattern runs every scorer on every trace. It works until volume and cost climb. The routed pattern classifies each trace first, then runs only the scorers that class needs.
# Naive approach: run every scorer on every trace.
def score_all(trace):
return {
"faithfulness": faithfulness(trace),
"groundedness": groundedness(trace),
"goal_accuracy": goal_accuracy(trace),
"topic_adherence": topic_adherence(trace),
}
# Routed approach: classify the trace, then run only the scorers that apply.
SCORERS_BY_CLASS = {
"rag_answer": [faithfulness, groundedness],
"tool_call": [goal_accuracy],
"refusal": [topic_adherence],
}
def classify(trace):
if trace.retrieved_context:
return "rag_answer"
if trace.tool_calls:
return "tool_call"
if trace.refused:
return "refusal"
return "default"
def score_routed(trace):
trace_class = classify(trace)
scorers = SCORERS_BY_CLASS.get(trace_class, [])
return {s.__name__: s(trace) for s in scorers}
Routing means a RAG answer is scored for faithfulness and groundedness. A tool call is scored for goal accuracy. A refusal is scored for topic adherence.
Every trace is covered, and no compute is spent on irrelevant checks. The economic difference compounds above 100K traces a day. The naive pattern runs four scorers per trace whether they apply or not; the routed pattern runs one or two.
That reduction is what makes 100% coverage affordable rather than aspirational. It builds on the span-level tracing in our Agentic Observability foundation.
Choosing Scorers That Run Without Ground Truth
Reference-free scorers are the ones that survive in production. They need no labeled answer.
- Faithfulness and groundedness: Check whether a generated answer is supported by the retrieved context [2], [4].
- Hallucination and answer completeness: Check whether the output invents facts or omits required information.
- Topic adherence and goal accuracy: Check whether the agent stayed on task and reached its objective.
Faithfulness has a precise origin. In the RAGAS framework it measures whether the claims in a generated answer can be inferred from the retrieved context [5].
Not every check needs a judge. A schema validator or a math verification is deterministic and more consistent than an LLM-as-a-Judge scorer.
A useful rule is to reserve the LLM judge for open-ended semantic checks. Faithfulness against retrieved context is a judgment call, so a judge fits. A JSON schema or a numeric total is verifiable, so a validator fits.
Running these scorers on 100% of traces in-environment is what Fiddler Centor Models (formerly Fiddler Trust Models) are built for. They are batteries-included and run inside your own environment. There is no external API call and no per-evaluation cost, at under 100ms response time.
What to Watch For When You Score Every Trace
Watch Out For This
- Cost: An external LLM judge on every trace triggers the Evaluation Trust Tax, the per-call cost providers add to your LLM bill per trace.
- Judge inconsistency: LLM judges show documented position, verbosity, and self-enhancement biases [6], so we recommend binary pass or fail scoring and a short explanation.
- Misconfiguration: Classifying on the wrong metadata routes traces to the wrong scorer, so a class goes silently under-covered.
- Sensitive data: Agent traces can carry PII; redact sensitive content in trace payloads before evaluation rather than blocking the trace.
That cost scales with trace volume, so it grows as you approach full coverage. Two related costs travel with it: incident risk exposure from down-sampling, and operational overhead from building custom eval infrastructure. Model all three for your own deployment with the Evaluation TCO Calculator; figures vary by model, deployment size, and traffic volume.
What Continuous Evaluation for AI Agents Unlocks Next
Running the same evaluators before and after launch removes a subtle source of error. What you measure in pre-production is exactly what you monitor in production, with 100% trace coverage and no sampling.
A failing classified trace becomes a starting point, not an endpoint. It links to span-level decision lineage, so you move from a bad score to the failing step quickly.
Span-level telemetry rolls up to aggregate insights across the agent's timeline. This is how our observability lifecycle surfaces visibility across the full agentic hierarchy, and how multi-agent trace handoffs become debuggable.
Each failing trace you diagnose and annotate also becomes signal for the next iteration of the agent. Continuous evaluation stops being a passive safety net. It becomes a source of prompt-refinement and fine-tuning data.
One problem remains open. Classification schemes are still hand-built by engineers. Auto-discovering trace classes from production behavior, before an incident defines them for you, is unsolved.
Conclusion
The financial-services team from the opening now runs continuous evaluation for AI agents on every production trace. RAG answers route to faithfulness and groundedness scorers. Tool calls route to goal accuracy.
The refund questions that once slipped through are now scored the moment they happen. To see how the same evaluators run from pre-production through production, explore our lifecycle evaluations.
The harder work ahead is not running scorers. It is discovering which trace classes matter before a production incident names them for you.
References
[1] Microsoft Learn, "Continuously evaluate your AI agents," Microsoft, 2026. [Online]. Available: https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/continuous-evaluation-agents
[2] Databricks, "Best Practices and Methods for LLM Evaluation," Databricks, 2026. [Online]. Available: https://www.databricks.com/blog/best-practices-and-methods-llm-evaluation
[3] Databricks, "The Key to Production AI Agents: Evaluations," Databricks, 2025. [Online]. Available: https://www.databricks.com/blog/key-production-ai-agents-evaluations
[4] NVIDIA, "Mastering LLM Techniques: Evaluation," NVIDIA Developer, 2025. [Online]. Available: https://developer.nvidia.com/blog/mastering-llm-techniques-evaluation/
[5] S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, "RAGAS: Automated Evaluation of Retrieval Augmented Generation," arXiv, 2023 (rev. 2025). [Online]. Available: https://arxiv.org/abs/2309.15217
[6] L. Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," in Proc. NeurIPS, 2023. [Online]. Available: https://arxiv.org/abs/2306.05685