Artificial intelligence (AI) is an extremely valuable technology, but it comes with risks that every business faces when using models in production. Recently, O’Reilly partnered with our team at Fiddler to bring you a free eBook on building high performance Responsible AI. This book contains the key strategies and frameworks we’ve learned for building AI safely and responsibly, so you can prevent issues like model bias and data drift from negatively affecting your customers and your business.

Over the next few weeks, we’ll be going chapter by chapter to share some of the most valuable insights from this book. Let’s start with Chapter 1: Introduction to Model Performance Management and Explainability.

What is Model Performance Management (MPM)? 

Machine learning (ML) models are highly complex, data-driven systems that present new challenges to software teams. Model Performance Management tracks and monitors the performance of ML models through all stages of the model lifecycle. This lifecycle includes:

1. Defining a business problem that should be modeled
2. Gathering training data (generally historical, real-world data)
3. Developing a model using a variety of techniques, from simple logistic regression and decision trees to more advanced neural networks for deep learning 
4. Validating the model to ensure it works as intended and complies with company standards and government regulations
5. Deploying the model to production
6. Continuously monitoring the model in production for performance issues or outages
7. Improving the model iteratively over time

What challenges does MPM help solve?

Machine learning models are quite different from traditional software systems, causing teams to struggle with maintaining high-quality models in production. 

• Inconsistent performance: Fluctuations can happen because ML models are trained on historical data that might look different from the real data they see in production. 
• Lack of control and debuggability: Because complex ML systems behave like a black box, practitioners may not understand the model well enough to know how to fix it when there are problems.  
• Amplification of bias: Models can amplify hidden bias in the data they’re trained on, putting the business in legal and reputational risk. 
• Hard to improve performance: Since it’s difficult to understand and track what improvements need to be made, often models won’t get more investment after the initial launch. 

Model performance management has several key benefits that help teams address these challenges.

How does MPM benefit businesses that are using AI? 

1. MPM validates models prior to launch

Before deploying a model, it’s important to validate that it solves the intended business problem and doesn’t have any adverse effects. MPM can help you explain the model in human-understandable terms, so you can answer questions like “How is the model making a prediction?” and “Are there any biases?”

2. MPM continuously monitors model performance in production 

Machine learning models are trained on historical data, and when the live data shifts (such as the COVID-19 pandemic causing shifts in consumer behavior), the model’s performance can degrade. MPM can reassess the model’s business value and performance on an ongoing basis through model monitoring. 

3. MPM proactively addresses model bias 

It’s impossible to eliminate bias from the world. But we can work to eliminate bias from ML models, to make sure they’re not amplifying or propagating the bias reflected in the real-world data (like the famous case of Amazon’s ML models for recruiting favoring resumes with traditionally male first names). MPM monitors for bias and helps businesses address it immediately to avoid costly penalties, such as regulatory fines or reputational loss. 

4. MPM explains past predictions

Because MPM tracks a model’s behavior from training to serving, it can explain what factors led to a certain prediction to be made at a given time in the past. Explainable AI is vital for validation and compliance, allowing stakeholders to reproduce a model’s predictions along with explanations (this is known as “prediction time travel”).

As you may have noticed, one of the core features of MPM is being able to explain a model. Explainability is the subject of Chapter 2 of the O’Reilly eBook — stay tuned for a post on our blog that covers the key takeaways from that next chapter. You can download the full eBook for free by clicking here: