4
Min Read
Who is responsible for model monitoring in your organization? As machine learning (ML) takes center stage for many businesses, it is essential to have set roles and responsibilities. All machine learning models experience drift, affecting the accuracy of their decisions. Without continuous model monitoring, you won’t discover what’s wrong until serious errors start to occur.
So who has the responsibility to keep track? Is DevOps used in machine learning? Are DevOps professionals equipped to monitor and correct machine learning performance?
Increasingly, teams are choosing to break out MLOps as its own practice. In this blog, we’ll explore what makes MLOps different from DevOps. We’ll also share how to empower your team with model monitoring tools that take the guesswork out of MLOps best practices–even when they fall to DevOps.
Let’s start by establishing shared definitions of these two approaches.
DevOps is an approach which unites two departments: software development and IT operations. This mashup represents a collaborative approach to releasing updates and continuously improving a system. Coding and testing the improvements is the work of development, while the releases, allocation of resources, and guidance for future updates fall to operations.
MLOps relates to the continuous improvement and quality assurance of machine learning systems. MLOps helps data scientists, machine learning engineers, and other stakeholders collaborate to ensure high-performing ML models, as well as set processes to retrain and deploy new models..
What makes MLOps different from DevOps? DevOps is concerned primarily with the creation and release of code. MLOps leverages data inputs as well as code. Large amounts of data that originate in the outside world must be used to train machine learning models. This training enables accurate predictions, forecasts, and even automated decisions.
Consider a familiar example - a banking app. On the DevOps side, the bank’s IT team is concerned with functional features, security, and the user experience. But if one of the desired features is the ability for customers to apply and be pre-approved for a loan within the app, this introduces a machine learning element. To issue the lending decisions, the banking algorithm will leverage data like the user’s income, credit history, desired loan, employment status, and more to decide whether or not to approve them. A need to include data means machine learning development and monitoring goes beyond the commands in the code.
This leads to a second differentiator between DevOps and MLOps: quality assurance testing. In DevOps, the outcomes of testing are relatively binary - either something works or it doesn’t. In MLOps, the algorithm will always “work” to make some type of an output. But what is an acceptable vs unacceptable decision? The bank whose app we described above might be fine with rejecting potential borrowers with a specific credit score. But what about unanimously rejecting all applicants from a specific zip code? This is the kind of model bias and data drift that can emerge within ML models before or after release.
Since MLOps teams can’t detect these issues through pre-release testing, they need a tool to correct the algorithm in real-time, without having to recreate the model from scratch. That’s where Fiddler enters the picture.
Fiddler is a platform created to uniquely address the challenges of the MLOps lifecycle. Data Science (DS) and IT teams both benefit from the accelerated ability to analyze and understand changes in model performance using explainable AI.
What is explainable AI? The ability to know and explain why models are performing in certain ways–or changing their performance over time. Explaining these outcomes is key to working efficiently, achieving improvements, and ensuring stakeholder buy-in from across and outside the organization. Explainable AI helps ensure fairness in ML models and that the team can trust the decisions the model makes.
We have paired this deep insight with real-time alerts when the model makes an unexpected decision. This empowers your MLOps team to take a proactive approach to ML maintenance and improvements. Plus, we have a deep commitment to compliance and information security. Data is encrypted to the highest possible standard, security is monitored by a third party, and data is completely unrecoverable once deleted. We want to empower you to trust your AI–which means you have to be able to trust Fiddler first.
Is your AI fair? Is your data drifting? With Fiddler on your side, you won’t have to wonder or wait to find out. Try Fiddler for free to see just how quickly you can explain and improve your machine learning models.
