MLOps Fundamentals
MLOps applies DevOps principles to machine learning systems. It provides practices and tools for reliably building, deploying, monitoring, and retraining ML models in production so they remain accurate and maintainable over time.
itArtificial intelligence and machine learning | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — MLOps Fundamentals
MLOps is what happens when a machine learning model leaves the notebook and encounters the rather less forgiving world of production. The model may have an impressive score, a tidy plot, and an encouraging filename. None of those tell you whether it can be deployed, observed, reproduced, or safely replaced. MLOps supplies the operating habits that make those questions answerable.
Before this, the usual route was a manual handoff: someone trained a model, someone else packaged it, and everyone hoped the same data and settings could be found again later. The ML lifecycle turns that relay race into a loop: manage data, train, deploy, monitor, then retrain when the evidence says the old model no longer fits. The loop is less glamorous than a new algorithm, which is fortunate, because glamour makes poor incident response.
The awkward fact is that correct code can still produce a failing system. Data drift means the input data no longer resembles the data that taught the model its patterns. A schema can change. A feature can go missing. The world can move on with startling disregard for last year's training run. Monitoring therefore watches the data, the model's observed quality where labels exist, and the serving system's latency and errors. When a signal crosses its limit, the response is investigation, retraining, or rollback rather than hopeful staring.
A release record binds a candidate to its code, data, configuration, environment, metrics, and artifacts. That is why versioning only the model file is a half-built bridge. Progressive deployment adds the next useful restraint: a canary sends limited traffic to a new model, while shadow mode lets it make parallel predictions without affecting users. Both produce evidence before a full rollout; neither grants immunity from reading the evidence.
The tooling landscape is deliberately untidy. MLflow tracks experiments and model artifacts; DVC versions data; Kubeflow runs pipelines; Feast keeps feature definitions aligned; monitoring tools reveal drift; managed platforms assemble more of the path. No single tool performs the whole job, and a single analyst with one notebook does not need the entire parade. Start with tracking and a registry, then add automation and monitoring as production risk and collaboration make the work necessary.
Read the intro when you need the full map of the lifecycle and maturity levels. Use the cheatsheet when choosing a deployment strategy, a testing layer, or a monitoring signal. The practice reference turns those parts into a release gate, and the exercise asks you to assemble the evidence packet that a real candidate needs before it is allowed near live traffic.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
Supports
- MLOps maturity levels 0, 1, and 2 definitions
- Continuous training (CT) as distinct from CI/CD
- Automated pipeline architecture for ML systems
- Data and model validation as pipeline steps
- Monitoring and retraining triggers
- https://ml-ops.org/content/mlops-principles
Supports
- MLOps lifecycle phases and core principles
- Relationship between DevOps, DataOps, and ModelOps
- Feature store role in preventing training-serving skew
- Data drift as a cause of silent model degradation
- Reproducibility requirements for ML experiments
- https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/mlops-maturity-model
Supports
- Five-level maturity model for ML operations
- Progressive deployment strategies (canary, shadow, A/B)
- Governance and audit requirements for production ML
- Latency and SLA considerations in model deployment
- https://mlflow.org/docs/latest/
Supports
- Experiment tracking (parameters, metrics, artifacts)
- Model registry for versioned artifact management
- Reproducibility through environment and dependency tracking
- Integration with popular ML frameworks
- https://aws.amazon.com/what-is/mlops/
Supports
- MLOps as adaptation of DevOps for ML workloads
- Pipeline orchestration and automation patterns
- Model monitoring and operational health tracking
- https://mlflow.org/docs/latest/ml/tracking/
Supports
- MLflow Tracking records parameters, code versions, metrics, and output files for runs
- MLflow Tracking can associate datasets with training events
- https://dvc.org/doc
Supports
- DVC documentation for versioning data and ML artifacts alongside Git metadata
- https://docs.feast.dev/
Supports
- Feast documentation for feature definitions and offline and online feature access
- https://www.kubeflow.org/docs/components/pipelines/
Supports
- Kubeflow Pipelines documentation for portable ML workflow orchestration
- https://cloud.google.com/vertex-ai/docs/start/introduction-unified-platform
Supports
- Vertex AI documentation for a managed ML platform spanning model development and deployment
- https://docs.aws.amazon.com/sagemaker/latest/dg/mlops.html
Supports
- SageMaker AI documentation for production ML CI/CD practices
- https://research.google/pubs/hidden-technical-debt-in-machine-learning-systems/
Supports
- 2015 publication on technical debt in machine learning systems
- Production ML complexity beyond model code
- https://www.uber.com/us/en/blog/michelangelo-machine-learning-platform/
Supports
- 2017 public account of Uber's end-to-end ML platform
- Versioned training records, staged model switching, and prediction monitoring
- https://cloud.google.com/blog/products/ai-machine-learning/introducing-ai-hub-and-kubeflow-pipelines-making-ai-simpler-faster-and-more-useful-for-businesses/
Supports
- November 2018 announcement of Kubeflow Pipelines
- Reusable and deployable ML workflow pipelines
- https://cloud.google.com/blog/products/ai-machine-learning/introducing-feast-an-open-source-feature-store-for-machine-learning
Supports
- 2018 release of Feast as an open-source feature store
- Feature reuse and shared definitions across ML projects
- https://cloud.google.com/blog/products/ai-machine-learning/run-ml-workflows-in-production-with-cloud-native-toolkit-kubeflow-1-0
Supports
- March 2020 release of Kubeflow 1.0
- Kubernetes-native lifecycle platform for portable ML workloads
- https://aws.amazon.com/about-aws/whats-new/2020/12/introducing-amazon-sagemaker-pipelines-first-purpose-built-ci-cd-service-machine-learning/
Supports
- December 2020 general availability of SageMaker Pipelines
- Pipeline lineage, automation, and model registry workflow
- https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-launches-vertex-ai-unified-platform-for-mlops
Supports
- May 2021 general availability of Vertex AI
- Unified managed MLOps components including pipelines, metadata, and monitoring
- https://www.uber.com/gb/en/blog/scaling-michelangelo/
Supports
- Prediction-to-outcome joins for direct production accuracy measurement
- Distribution monitoring as a less precise substitute when labels cannot be joined
- Data access and feature pipelines as a major source of ML operational complexity
