On-Demand Webinar · 56 min

Your Model Is Not an Island: Operationalize ML at Scale with MLOps

Operationalizing a model is a collaboration problem, not a modeling problem. Chris Bergh on how MLOps, a derivative of DataOps, orchestrates the whole machine-learning pipeline: data access, training, deployment, monitoring, and governance. Recorded 2020; updated August 2026.

Presented by Chris Bergh

What you'll learn 7 points
  • Gartner's finding frames the session: a majority of AI models never make it into production, and enterprises take an average of nine months to get a model from pilot to production.
  • A model is one step inside a larger chain. Data engineering, data science, data visualization and self-service BI, data governance, and production operations each have their own tools and their own development process, and the model is built in only one of them.
  • Eight success factors for ModelOps: treat models as part of a bigger process rather than something unique, test in development and automate deployment to production, monitor everything in production including the models, put your models in a DAG and everything in git, treat experiments as variations of production, treat real-time models almost the same as batch models, do not build your own ModelOps solution, and see ModelOps as one part of DataOps.
  • Cloud providers position their parts as a ModelOps solution, but the session lists what is missing: end-to-end pipelines rather than CI/CD alone, a sandbox and environment pipeline, support for a complex team organization spanning data centers, an allowance that data people are not software people, and the DataOps process metrics that drive change.
  • Real-time serving is a deployment option, not a separate discipline. TensorFlow Serving, Seldon Core, KFServing, or a Flask REST API will all serve a model, and the operational metrics to monitor are request counts, latency, throughput, memory and CPU consumption, and 4XX and 5XX error rates, each with defined upper and lower bounds that trigger alerts.
  • At Celgene the DataKitchen platform integrates hundreds of data sets into a unified star schema, runs more than 10,000 automated tests, absorbs more than 100 schema and data changes per week, and manages models in docker containers inside recipes, with very few errors or missed SLAs.
  • Without DataOps a team spends roughly 97 percent of its effort on tools, technology, data, process, and people and 3 percent on data operations. The target ratio in this session is 80/20, a level software teams already reach.

Slides

71 slides

Questions from this session

What is ModelOps and how does it relate to DataOps?

ModelOps, also called MLOps, is the operational practice around machine learning models: deploying them, monitoring them in production, and retraining them. This session's argument is that it is one of several Ops areas inside DataOps, alongside ETLOps or DataEngOps for data engineering, AnalyticOps or self-service ops for BI, and DataGovOps and DataSecOps for governance and security. Each maps to a functional team, and all of them need one enterprise platform rather than four.

Why do so few machine learning models reach production?

Gartner reports that a majority of AI models never make it into production and that enterprises take an average of nine months to get one from pilot to production. The session attributes this to four causes: models are part of a system rather than standalone artifacts, the data itself is hard, teams are scared of production errors, and there is no accepted way to prove a model is still working once deployed.

How do you monitor a machine learning model in production?

Monitor the whole chain, not just the model: the data going in, the transformations acting on it, the model, and the visualization coming out. For a model served in real time over a REST API, track request counts, latency, throughput, memory and CPU consumption, and error rates, and set upper and lower bounds for each so alerts fire automatically. Feed live data and outcomes back so you know when to retrain.

Should you build your own ModelOps platform on cloud provider parts?

This session says no. Google, Microsoft, and AWS all supply components and expect you to assemble them, which leaves the hardest parts unbuilt: end-to-end pipelines across the full toolchain, the sandbox and environment pipeline developers need, coordination across a complex team organization and multiple data centers, and the process metrics that show whether anything improved. ModelOps is also not the same thing as DevOps CI/CD.

What does it mean to treat experiments as variations of production?

It means running the model pipeline as a parameterized function so a training run, a feature optimization run, and the production run are the same process with different named parameter sets. Parameters vary the data inputs, the model outputs, and the feature creation steps in the workflow, with credentials in secure storage. The practical payoff is a time machine: you can rerun any past configuration, and an experiment already has a path to production.

What is a self-service data science sandbox?

It is a governed analytic environment that a central IT or data group gives to a self-service team, monitors while it is in use, and takes back or changes when the term ends. Each request differs: one team needs 24 months of business data and Tableau, another needs customer data with Python and SQL access, another needs anonymized deposit account data with Power BI and a data dictionary. Some sandboxes stay one-off experiments, and the ideas that earn it get reimplemented centrally with recipes, tests, and orders.

Where to go next