Key Takeaways
- OpenTelemetry standardizes how telemetry is collected and transported, but AI telemetry conventions and frameworks may still use different attribute names for the same operational concept.
- Fiddler maps AI telemetry attributes to shared semantic concepts, allowing dashboards, alerts, metrics, and evaluators to work consistently across conventions.
- Users can manage mappings from the Fiddler UI at runtime, helping maintain application portability and avoid vendor lock-in.
The Challenge of Interpreting AI Telemetry Across Conventions
Modern AI application development gives engineering teams the flexibility to choose from a growing number of models, frameworks, and software development kits (SDKs). This flexibility also creates an observability challenge. Several AI telemetry conventions have emerged over the past few years, but they have evolved independently and often represent the same underlying concepts differently. As a result, service owners and platform operators struggle to monitor and govern AI applications consistently across frameworks.
OpenTelemetry provides a standard framework for generating, collecting, and exporting traces, metrics, and logs. The OpenTelemetry Protocol (OTLP) provides a common method for moving that telemetry between applications, collectors, and observability platforms. However, collection does not ensure that a backend can interpret every attribute consistently. Let’s look at five attributes commonly used for AI observability. Although the underlying concepts are similar, OpenTelemetry GenAI, OpenInference, and the Vercel AI SDK represent them differently:
This comparison shows why transporting telemetry through a common protocol does not guarantee consistent analytics. The observability platform must recognize that these different attribute names represent similar concepts.
Production environments commonly include several conventions because standards change, SDK adoption varies between teams, and organizations add custom instrumentation. An observability platform therefore needs to understand what an attribute means without depending on a single telemetry convention or forcing teams to adopt a vendor-specific convention.
Mapping Attributes to a Common Concept
Fiddler provides a server-side abstraction layer that maps incoming attribute keys with a common set of framework-independent AI telemetry semantic concepts, rather than introducing another client-side convention. For input-token usage, the mapping looks like this:
The destination, input_tokens, is a concept in Fiddler's approach. It describes what the value means to the platform, independently of the convention used by the producer.
A dashboard can query the input_tokens concept once. Since Fiddler maps each raw attribute to that concept, it allows for consistently calculating usage across conventions.
This approach also allows different telemetry conventions to coexist. OpenTelemetry GenAI, OpenInference, Vercel AI SDK, Claude Code, Google ADK, and other formats remain inputs to the mapping layer. Teams can continue to emit telemetry using the conventions supported by their frameworks.
Mapping Span Types Across Agent Frameworks
Semantic differences also affect how frameworks classify agent operations. An agent trace may contain model calls, tool executions, retrieval steps, guardrails, evaluators, and nested agents. Each framework based on its convention may use a different attribute and value to identify these operations, making it difficult for observability platforms and users to group and analyze them consistently.
Fiddler maps these representations to ten higher-level span types: llm, tool, agent, chain, embedding, retriever, reranker, guardrail, evaluator, and the generic span fallback.
Let's consider a trace containing model calls and tool executions. The raw span-classification attributes may come from different conventions, but the mapping layer identifies the operations as llm and tool. As agent workflows become more complex, consistent span types help teams follow each model, tool, and agent operation across the complete trace regardless of its semantic convention.
Preserving the Original Telemetry
Mapping these representations creates consistency across frameworks, but the platform must still preserve the source telemetry for debugging and verification.
Fiddler stores the original attribute name as it arrived and records the resolved semantic concept alongside it. The resulting data model provides both the attribute key emitted by the framework and the semantic concept assigned by Fiddler.
The original name provides provenance and helps teams debug framework-specific behavior. The semantic concept gives dashboards, alerts, metrics, and evaluators a consistent way to use the value across applications.
Attributes without a mapping remain available under their raw names. Operators can inspect and query them, then add a mapping when the attribute represents a concept that should be available to platform or application owners.
Supporting Framework Changes Without Rewriting Observability Logic
Fiddler currently documents more than 30 semantic concepts across token usage, cost, model and provider identity, agents and tools, sessions, content, performance, span identity, and metadata. It also provides more than 140 predefined mappings across roughly a dozen frameworks and SDKs.
This allows platform teams to build shared observability views while application teams retain the instrumentation that fits their frameworks.
Users can add or delete mappings from the user interface (UI) at runtime without changing application code or redeploying instrumentation. This helps organizations operationalize observability and control for AI applications at scale while maintaining application portability and avoiding vendor lock-in.
Semantic Mapping and the Fiddler Control Plane
Semantic mapping is a foundational component of the Fiddler AI Control Plane. Mapping framework-specific telemetry into semantic concepts makes the same operational context available to Evals, Monitoring, Enforcement, and Governance.
Refer to the Fiddler semantic mappings documentation for more information.