MLOps
In plain wordsThe working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself.
The engineering discipline that treats a machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats. model as a production system rather than a one-off experiment, covering the practices, roles and tools that let models be built, released, watched and retrained repeatably. It adapts DevOps — automation, version control, continuous integration and delivery (CI/CD), monitoring — to systems whose behavior depends on data as well as code, so that data, features, models and configuration are all versioned and tested. Typical components are reproducible pipelinesmachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way., experiment trackingexperiment tracking, n.: Keeping an automatic logbook of every model-training attempt — settings, data, code and results — so a team can compare attempts, repeat the good ones and see what changed., a feature storefeature store, n.: A shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live., a model registrymodel registry, n.: A central catalog where a team stores every version of its trained models, with records of how each was made and which version is approved for use., automated validation gates, deploymentmodel deployment, n.: Putting a trained AI model to work in a real product so that people or other software can use it — and doing so carefully enough that the change can be undone if something goes wrong. with staged rollouts, model monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm. for data driftdata drift, n.: When the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable. and falling accuracy, and continuous training, in which pipelines retrain and redeploy models on fresh data. Maturity frameworks, such as Google Cloud's levels 0 to 2, run from manual notebook-to-production handoffs to fully automated retraining. The underlying premise is that the model is a small part of a larger system whose data dependencies and feedback loops create distinctive technical debtmachine learning technical debt, n.: The hidden future cost of quick shortcuts in machine-learning systems — tangled data dependencies, messy glue code, unchecked feedback loops — that later make them hard and risky to change..
Our MLOps setup retrains the fraud model nightly and blocks deployment if holdout recall drops.
Origin
A blend of 'machine learning' and DevOps. No coiner is known: the word appeared in practitioner talks, blog posts and startup marketing around 2017–2018, building on concerns made famous by D. Sculley and colleagues' NeurIPS 2015 paper 'Hidden Technical Debt in Machine Learning Systems'. Google Cloud's guide to MLOps maturity levels (c. 2020), vendor platforms, and a wave of books and courses around 2020–2021 established it.
Usage note
Usage ranges from a narrow sense (deployment and monitoring tooling) to a broad one (the whole lifecycle, including data engineering). LLMOpsLLMOps, n.: The everyday work of running chatbots and other products built on large language models: tracking prompts, testing answers, watching cost and quality, and updating safely when models change. adapts the practices to systems built on large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., where prompts, retrieval settings and model vendors change more often than weights.
Related
- LLMOpsLLMOps, n.: The everyday work of running chatbots and other products built on large language models: tracking prompts, testing answers, watching cost and quality, and updating safely when models change.
- machine learning pipelinemachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way.
- model registrymodel registry, n.: A central catalog where a team stores every version of its trained models, with records of how each was made and which version is approved for use.
- experiment trackingexperiment tracking, n.: Keeping an automatic logbook of every model-training attempt — settings, data, code and results — so a team can compare attempts, repeat the good ones and see what changed.
- model monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm.
- feature storefeature store, n.: A shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live.
- AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale.
- machine learning technical debtmachine learning technical debt, n.: The hidden future cost of quick shortcuts in machine-learning systems — tangled data dependencies, messy glue code, unchecked feedback loops — that later make them hard and risky to change.
Sources
- Sculley, D. et al. (2015). Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28 (NeurIPS 2015).
- Kreuzberger, D., Kühl, N. & Hirschl, S. (2023). Machine Learning Operations (MLOps): Overview, Definition, and Architecture. IEEE Access 11.
Mentioned in
AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., batch inferencebatch inference, n.: Running an AI model over a big pile of inputs as one job rather than answering each request live, like processing a stack of forms overnight. Results come back later but cost less., concept driftconcept drift, n.: When the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold., experiment trackingexperiment tracking, n.: Keeping an automatic logbook of every model-training attempt — settings, data, code and results — so a team can compare attempts, repeat the good ones and see what changed., machine learning engineermachine learning engineer, n.: An engineer who turns machine-learning ideas into working systems: preparing data, training and testing models, and running them reliably inside real products., machine learning pipelinemachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way., model deploymentmodel deployment, n.: Putting a trained AI model to work in a real product so that people or other software can use it — and doing so carefully enough that the change can be undone if something goes wrong., model monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm., model registrymodel registry, n.: A central catalog where a team stores every version of its trained models, with records of how each was made and which version is approved for use., canary deploymentcanary deployment, n.: Releasing a new model or software version to a small slice of users first and watching closely, so that any problem hits only a few people before the change reaches everyone., data versioningdata versioning, n.: Keeping a record of every version of the data used to train and test AI models, so that any past result can be traced back to exactly the data that produced it., feature storefeature store, n.: A shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live., LLMOpsLLMOps, n.: The everyday work of running chatbots and other products built on large language models: tracking prompts, testing answers, watching cost and quality, and updating safely when models change., machine learning technical debtmachine learning technical debt, n.: The hidden future cost of quick shortcuts in machine-learning systems — tangled data dependencies, messy glue code, unchecked feedback loops — that later make them hard and risky to change., MLflowMLflow, prop. n.: A free software platform that records machine-learning experiments, packages the resulting models and keeps track of which versions are ready to be used., shadow deploymentshadow deployment, n.: Quietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over., training–serving skewtraining–serving skew, n.: A mismatch between how data is prepared when a model is trained and when it is used for real, which makes the live model quietly worse than its tests promised.