Data scientists work tirelessly to build and train a model then face the daunting challenge of deploying it into production. The model itself is only a fraction of the overall ML system. Moving a model from development into operations involves provisioning infrastructure, installing and configuring software, preparing data, and testing both code and data. With manual processes, these steps can require several months of effort. Integration with existing operations is also challenging. Gartner recently reported that the complexity of solution integration with existing infrastructure was named as a top barrier to AI implementations.
Software applications are typically code-driven. Developers deploy software and then they may not touch it again until there’s a requirement to fix a bug or add a new feature. Not true for ML systems. Model performance is affected by many factors, including changes in data, users, ML systems, performance requirements or the external environment. Model updates are code, data and environment-driven. Once a model is in production, it needs to be actively monitored for any degradation in performance. Without automation, it can take weeks to detect that a model’s accuracy has deteriorated and several additional weeks to retrain the model with the newest data or remediate issues affecting model performance.
Figure 1: Model performance decays over time due to changing data, code, users, system environment and other external factors. Machine learning models must be monitored in production and retrained or redeveloped periodically. (Source: Forrester)
Imagine creating, deploying and managing ten models. Or hundreds. Businesses may wish to turn to data science models to gain competitive advantage, but the onerous workflow processes related to model development, deployment and maintenance often get in the way. In the coming years, ML systems will serve as a major differentiator for companies. Markets will be led by the most agile organizations, who can train, deploy, monitor and update machine-learning systems in the shortest amount of time. Agile companies will be adept at putting new ideas into action and quickly altering course when business conditions change.
ML Agility
ModelOps, also known as MLOps, is an application of DataOps principles and automation to machine learning systems. ModelOps turns data science workflows into robust, repeatable processes executed in minimal time and with virtually zero errors. When you use the DataKitchen DataOps Platform to implement ModelOps, your end-to-end machine-learning lifecycle is optimized for agility and quality. A DataKitchen ModelOps workflow includes:
- Self-Service Sandboxes – An environment instantiated (and deleted) on-demand that includes everything a data scientist needs to create, train, test and deploy machine-learning models: complete toolchain, security vault, prepackaged data sets, role-based access control for teams, integration with workflow management, and a Kitchen workspace that ties it all together as a unified environment.
- Continuous Deployment – Building, integrating, testing and deploying ML models and ML systems as an automated software-controlled process that can be executed on-demand and complete in minutes or hours.
- Meta-Orchestration – a hierarchy of orchestrations. Automated execution of complex, multi-step data pipelines that span data centers and varied toolchains.
- Automated Testing – Testing in all phases of orchestration from data ingestion to transformation to model execution to delivery of polished visualizations. The DataKitchen Platform offers a single test environment encompassing heterogeneous toolchains.
- Continuous Monitoring – Repeated evaluation of model accuracy and efficacy in production with automated orchestrations or alerts when results meet statistical criteria.
- Continuous Governance – Governance workflows as repeatable, verifiable orchestrations implemented using governance-as-code automation.
ModelOps as a Competitive Advantage
Many data science professionals believe that models by themselves can provide competitive differentiation. Machine learning and AI will offer ways for companies to gain competitive advantage. What many fail to understand is that the organizational workflows related to ML model creation, deployment, monitoring and management will serve as an even greater point of competitive differentiation. Companies interested in machine learning and AI will hire smart data scientists, but not every company will employ them productively. The enterprises that implement ModelOps will be capable of iterating and updating models with far greater agility. Model agility translates into business agility. Data teams that embrace ModelOps with automated platforms like the DataKitchen DataOps Platform will be better able to develop agile business processes and emerge as market leaders.
