We’re thrilled to announce that the Fiddler AI Observability Platform has been recognized as a key partner in the newly launched Amazon SageMaker partner AI apps. This native integration enables enterprises to validate, monitor, analyze, and improve their ML models in production, all within their existing private and secure Amazon SageMaker AI environment.
Amazon SageMaker AI makes it easy to build, train, and deploy machine learning and foundation models at scale. SageMaker AI offers infrastructure and purpose-built tools for each step of the ML lifecycle including IDEs and notebooks backed by high-performance accelerated computing, purpose-built infrastructure for distributed training at scale, governance and MLOps tools, inference options and recommendations, and model monitoring and evaluation.
This collaboration reinforces Fiddler’s position as the go-to AI Observability platform for enterprises seeking transparency, trust, and actionable insights for machine learning (ML) and large language models (LLMs) applications. The unified Fiddler platform supports a wide range of ML models, including tabular, deep learning, computer vision, and natural language processing, while providing accurate monitoring for LLM applications on hallucination, toxicity, PII leakage, prompt injection attacks, and other critical metrics.
Launch More Models Into Production with Fiddler AI Observability
As a select Amazon SageMaker AI partner at launch, Fiddler uniquely addresses one of the challenging aspects of the MLOps lifecycle: monitoring ML models in real-world environments. With its unified AI observability platform, Fiddler enables data science and engineering teams to move beyond the experimentation and testing phases, confidently deploying and maintaining more high-performing models in production. By bridging this production gap, enterprises can establish a continuous feedback loop in their MLOps workflows, ensuring models are transparent, and trustworthy at scale.
Fiddler continuously monitors production models for issues like performance degradation, data drift, and data integrity problems, providing real-time alerts to enable model diagnostics and root cause analysis for quick issue resolution. Its intuitive and customizable dashboards and reports connect model metrics to business KPIs, allowing teams to make data-driven decisions to improve model performance. With support for custom metrics and segments, Fiddler empowers enterprises to create and track their model metrics unique to their use case.
AI Observability Within Amazon SageMaker AI is Now Frictionless
Amazon SageMaker AI customers benefit from a fully managed solution that includes purchase, licensing, provisioning, support, and billing all within the secure confines of their SageMaker development environment.
- Trusted AI Observability for the Enterprise: Fiddler provides a unified platform for monitoring ML models, including tabular, deep learning, computer vision, and natural language processing. It enables data science and engineering teams to track a wide range of model metrics, detecting performance issues, data drift, and anomalies in real-time
- Rapid Deployment and Scaling: Available within Amazon SageMaker Studio, ML platform owners can avoid additional security hurdles and InfoSec reviews, accelerating the path to scale production ML deployments
- Seamless Workflow Experience: Fiddler’s native integration with SageMaker provides a smooth user experience, covering the entire ML lifecycle — from model building and deployment to ongoing production monitoring — within the AWS ecosystem
- Completely private and secure: SageMaker AI ensures sensitive data stays completely within customers’ SageMaker environment and will never be shared with a third party
How SageMaker Partners Like Fiddler AI Improve ML Model Performance
Consider a leading online travel booking site aiming to enhance its hotel ranking model. The travel booking site trains the ranking model to predict the likelihood of a hotel being clicked on or booked by a specific user. The improved model would consider factors like pricing, location, user reviews, and contextual details such as travel dates and user devices. This allows them to refine their search result rankings, making them more relevant to individual preferences. The net effect is that the travel booking site can increase booking rates, enhance customer satisfaction, and drive revenue growth through a more personalized and efficient search experience.
The travel booking site uses Amazon SageMaker AI for model serving and training and Fiddler for AI Observability. Once the model is in production, Fiddler is used to monitor ongoing performance and ensure alignment with business objectives:
- Monitor ML Metrics: Continuously track key ML metrics like performance, data drift, data integrity, mean average precision (MAP), normalized discounted cumulative gain (NDCG), and other custom metrics specific to the use case like booking rates and click-through rates, ensuring the model remains relevant and performant in production
- Detect Issues Early with Alerts: Proactively identify model issues caused by evolving user preferences or incomplete input data, enabling timely retraining to maintain effectiveness
- Resolve Issues Quickly with Model Diagnostics: Address performance degradation with root cause analysis and model diagnostics to surgically pinpoint exactly where the issue is, and use insights to quickly resolve issues and improve the model’s performance
- Drive Data-led Responsible AI: Start with MTTI and MTTR monitoring followed by user-defined custom metrics tailored to unique use cases and tracking evidence for Governance, Risk, and Compliance (GRC) audits. By aligning ML model metrics to business KPIs, enterprises can drive revenue growth and deliver responsible AI and measurable business value
Join Us in Amazon SageMaker AI
Fiddler is excited to be one of the select AI partners featured in Amazon SageMaker AI. We’re excited to empower enterprises to productionize high-performing, transparent, and trustworthy ML applications at scale.
Ready to make your models trustworthy? Browse, discover, and try Fiddler in the Amazon SageMaker Partner AI Apps, via your Amazon SageMaker Studio, and keep your ML models performing at their best.
Frequently Asked Questions
1. What is AI Observability, and why does it matter in Amazon SageMaker?
AI observability refers to the ability to continuously monitor, diagnose, and improve machine learning models in production. In Amazon SageMaker, it helps enterprises ensure models are performant, compliant, and aligned with business goals.
2. Is my data secure when using Fiddler with SageMaker?
Absolutely. All data remains within your secure Amazon SageMaker AI environment. The Fiddler AI Observability and Security platform integration adheres to SageMaker’s data privacy principles, ensuring no data leaves your infrastructure or is shared externally.
3. What types of ML models does Fiddler support?
Fiddler supports a broad range of model types, including tabular, deep learning, computer vision, natural language processing (NLP), and large language models (LLMs). You can monitor models trained using SageMaker or imported from other environments.
4. What are the typical stages of ML model development and where does Fiddler have a role?
Machine learning (ML) model development typically follows a structured lifecycle with several key stages, each critical for building performant, trustworthy, and production-ready AI systems.
Stages:
Problem Definition and Data Collection
Every ML project starts by identifying a clear business problem and determining the type of prediction or automation needed. Teams then gather and prepare data from relevant sources, ensuring it’s complete, representative, and compliant with governance standards.
Data Preparation and Feature Engineering
The collected data is cleaned, transformed, and enriched to make it suitable for model training. This often involves handling missing values, encoding categorical features, and engineering new features that improve model performance.
Model Training and Experimentation
Data scientists train models using algorithms appropriate to the problem type (e.g., regression, classification, deep learning, or large language models). During this phase, teams conduct multiple experiments to optimize hyperparameters and identify the most effective model.
Validation and Evaluation
Trained models are rigorously tested using validation datasets to assess accuracy, fairness, and robustness. Key metrics, such as precision, recall, F1 score, or AUC, help ensure the model generalizes well to unseen data. Fiddler helps validate ML models before they go into production.
Deployment to Production
Once validated, the model is deployed into a live environment. Platforms like Amazon SageMaker AI streamline this process by offering scalable infrastructure for hosting, inference, and integration with business applications.
Monitoring and Continuous Improvement
After deployment, models must be continuously monitored for performance degradation, data drift, bias, and compliance issues. Fiddler AI Observability, now natively available in SageMaker, enables enterprises to track model behavior in real-time, detect anomalies, and trigger retraining when needed.