MLOps: 2021’s Must-Have Skill
For most practitioners, the machine learning workflow is straightforward. Training, validation, and deploying models can all be done manually. As machine learning pipelines grow more complex and technical risks increase, however, this is no longer scalable either technically or culturally. Manual processes lack robustness and discipline and culturally fail to fit into the larger IT organization that employs automated DevOps practices. Moreover, static systems are giving way to adaptable machine learning systems that dynamically adjust to changes in data and other environmental factors, such as with MLOps.
MLOps helps address the key challenge of utilizing machine learning models in a production environment: how to continuously train, integrate, deploy, and monitor models. In a production environment, the code that actually runs the models is only a small part of the overall machine learning workflow.
Not surprisingly, machine learning practitioners have borrowed DevOps methodology from software engineering and have customized it for machine learning to give rise to MLOPs, helping orchestrate larger and more complex machine learning pipelines.
Core Concepts: DevOps to MLOps
MLOps is a hot topic right now and its main focus is applying some of the core of DevOps to machine learning, as well as expanding upon those practices. For a machine learning engineer or data scientist who does not code first and does not have a deep understanding of DevOps, there are some core competencies that they need to learn.
1. Pipelines
Pipelines are the implementation and automation of machine learning workflows. The meaning of the term can vary depending on the context or platform used. For example, scikit-learn pipelines are automated processes to execute a typical set of tasks, such as data normalization, dimensionality reduction, etc. A TensorFlow TFX pipeline is a sequence of tasks that includes added steps, such as deployment to production. Thus pipelines can incorporate any task in a typical machine learning workflow, whether they are few or many.
2. Continuous Integration
Continuous integration refers to the practice of developers merging their code changes into a central repository that in turn triggers automatic builds and validation tests. This improves software quality and identifies issues earlier. Machine learning continuous integration expands on this to include both data validation and model validation.
3. Continuous Delivery
Continuous Delivery expands upon continuous integration. In addition to automatic builds and successful validation tests, the data, code, and models are all related to a QA or test environment, or even a production environment. Extract steps such as validating models that meet prediction performance benchmarks prior to being deployed are an example.
4. Continuous Training
Continuous training is central to MLOPs. Retraining triggers can include the input of new data. In well-orchestrated systems, this newly trained existing model is served at the end of the pipeline with no other updates necessary. Retraining triggers that result in new model implementation, such as feature engineering, feature selection, or hyperparameters, can require a new pipeline deployment
5. Continuous Testing
Continuous integration and continuous delivery systems for machine learning require continuous testing. This can be accomplished in several ways, such as running the model against a testing service that provides various sets of inputs and tests for each of the expected outputs. Other test metrics can be performed, such as load tests, to ensure your model meets acceptable latency and throughput benchmarks. Depending on how they are scaled, models can quickly hit performance bottlenecks due to CPU/GPU, memory, concurrency, and bandwidth constraints.
6. Continuous Monitoring:
If the model fails in production, the fault-tolerant system may fall back to a previous version of the model.
Testing and monitoring are part of DevOps best practices. In a machine learning context, the meaning of performance is not only focused on technical performance (such as latency) but, more importantly, on predictive performance. MLOps best practices encourage making expected behavior visible and setting standards that models should adhere to, rather than relying on a gut feeling.
Monitoring models in production ensure your models are performing in the wild. Model degradation (decreasing performance), model staleness (inputs were stale), and model attacks (data skews caused by live vs training data mismatch), changing data distributions, and data dependencies can all affect the performance of your model.
7. Reusable Infrastructure
MLOps can help a company standardize a reusable infrastructure that can maximize code and pipeline reuse when starting a new machine learning project. Rolling out a standard cloud architecture on AWS or Azure is a common practice and handles orchestration, storage, training, tuning, and monitoring. Standard configurations also provide a base for measuring cost, throughput, performance, and other metrics.
8. Reproducible Environments
Another central tenet that MLOps borrows from DevOps is reproducibility. Quickly copying and spinning up new machine learning pipelines can be quick and effortless. Reproducible environments also serve to preserve the state in cases where governance is important. Version control is also well established in software engineering. Data versioning and model versioning (parameters, hyperparameters, the algorithm used, and Machine learning. Reproducible pipelines are also essential to create fault tolerance for machine learning pipelines.
Learn More about MLOps at ODSC West 2021
If you would like to learn more about how MLOps can help your organization get the most out of their machine learning models, check out ODSC West 2021, the leading applied data science training conference, this November. ODSC West will feature dedicated tracks with expert-led, hands-on training sessions and workshops on MLOps and other cutting-edge data science and AI topics.
Register now to take advantage of our limited-time offer of 30% off your pass.
Highlighted MLOps sessions include:
- Deep Dive into Flyte: Ketan Umare | Creator and Chair | Flyte
- Unifying Development and Production Environments for Machine Learning Projects: Chip Huyen Adjunct Lecturer | Founder Stanford University | Startup on real-time ML
- Using Reproducible Experiments To Create Better Machine Learning Model: Milecia McGregor | Senior Software Engineer | Iterative
- MLOps… From Model to Production: Filipa Peleja, PhD | Lead Data Scientist | Levi Strauss & Co
- Operationalization of Models Developed and Deployed in Heterogeneous Platforms: Sourav Mazumder Data Scientist |Thought Leader, AI & ML Operationalization Leader | IBM
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform.
