CI/CD for Machine Learning
CI/CD for machine learning automates how changes to training code, data pipelines, and model-serving code are tested and released. It adds data and model quality gates to familiar software delivery so a new model reaches production only after its inputs, behavior, and serving contract pass defined checks.
itArtificial intelligence and machine learning | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Intro
CI/CD for Machine Learning
CI/CD for machine learning is the use of automated build, test, and release pipelines for an ML system. The system includes more than application code. Its releasable state can depend on training code, feature logic, data expectations, model artifacts, serving code, infrastructure, and configuration.
Traditional CI asks whether a software change builds and passes deterministic tests. ML CI keeps those checks and adds evidence about data and learned behavior. A pipeline might reject an unexpected input schema, a training run that produces invalid values, a candidate that misses a quality threshold, or a model that cannot load in its serving image. Continuous delivery then moves an accepted pipeline or model toward a production environment through explicit promotion gates.
Continuous training, or CT, is related but distinct. CI/CD releases changes to the ML pipeline and prediction service. CT runs the released training pipeline when a schedule, new data, drift signal, or other trigger calls for a new model. Keeping these loops separate makes failures easier to classify: a code change can break pipeline CI, while changed data can make a CT run fail validation without any code change.
The two pipelines
An operational design usually contains two connected pipelines.
The CI/CD pipeline starts from a version-control event. It checks source, packages pipeline components, runs tests, and deploys a new pipeline definition or serving implementation. Its output is executable production machinery.
The ML pipeline starts from approved code plus data. It validates data, transforms features, trains a candidate, evaluates it, and registers an accepted model. A later release stage deploys that registered model to a prediction service. Its output is a versioned model artifact and the evidence used to approve it.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
Supports
- CI/CD and continuous-training distinctions and maturity levels
- Source, test, deployment, registry, feature store, metadata, and orchestrator components
- Data and model validation, component tests, deployment compatibility, staged promotion, triggers, and monitoring
- https://learn.microsoft.com/en-us/azure/databricks/machine-learning/mlops/ci-cd-for-ml
Supports
- CI/CD scope across training data, input pipelines, training and serving code, predictions, and performance
- Configuration as code, infrastructure as code, logging, and troubleshooting roles
- https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/machine-learning-operations-v2
Supports
- End-to-end CI/CD and retraining architectures for classical ML, computer vision, and natural-language workloads
- Azure Machine Learning product placement in managed ML delivery
- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-setup-mlops-azureml?view=azureml-api-2
Supports
- Azure DevOps integration for infrastructure, data preparation, training, deployment, and monitoring
- Model evaluation and promotion through registration
- Environment identity and workload federation guidance
- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-models?view=azureml-api-2
Supports
- Azure Machine Learning model registration, versioning, lifecycle, and registry identities
- https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-projects.html
Supports
- SageMaker Projects CI/CD scope across images, data preparation, features, training, evaluation, deployment, monitoring, and updates
- Amazon SageMaker AI placement in the product landscape
- https://www.tensorflow.org/tfx/guide/understanding_tfx_pipelines
Supports
- TFX artifacts, parameters, components, pipeline graphs, and orchestrators
- Automated preparation, training, evaluation, deployment, and artifact dependency tracking
- https://www.kubeflow.org/docs/components/pipelines/concepts/pipeline/
Supports
- Kubeflow pipeline graphs, component execution, artifact flow, caching, retries, resources, and run history
- Kubeflow placement as a Kubernetes ML pipeline orchestrator
- https://www.kubeflow.org/docs/components/pipelines/interfaces/
Supports
- Kubeflow user interface, SDK, and API roles
- API use from continuous integration and deployment systems
- https://mlflow.org/docs/latest/ml/model-registry/workflow
Supports
- Registered model versions, aliases, tags, source runs, deployment, and cross-environment promotion
- Mutable alias versus immutable model-version behavior
- MLflow placement as a registry and lifecycle layer in an independent CI/CD stack
- https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/control-deployments
Supports
- Deployment environments, protection rules, required reviews, secrets, concurrency, triggers, and history
- Protected promotion-job and stage-specific credential guidance
- https://github.com/kelvins/awesome-mlops
Supports
- Discovery of ClearML, CML, and KitOps in the CI/CD for Machine Learning section
- https://clear.ml/docs/latest/docs/
Supports
- ClearML platform scope across compute, development, orchestration, artifacts, and deployment
- ClearML rationale and product placement
- https://github.com/clearml/clearml/blob/master/LICENSE
Supports
- ClearML open-source core licensing behind the open-core classification
- https://clear.ml/pricing/
Supports
- ClearML free and paid managed offerings behind the freemium classification
- https://cml.dev/doc
Supports
- CML use with repository CI for training, evaluation, reports, dataset monitoring, and cloud runners
- CML Awesome Links rationale and product placement
- https://github.com/iterative/cml/blob/master/LICENSE
Supports
- CML Apache-2.0 licensing and free open-source classification
- https://kitops.org/docs/integrations/cicd/
Supports
- ModelKit creation, contents, registry handoff, and use from CI/CD tools
- KitOps Awesome Links rationale and product placement
- https://github.com/kitops-ml/kitops/blob/main/LICENSE
Supports
- KitOps Apache-2.0 licensing and free open-source classification
- https://cloud.google.com/ai-platform/docs/technical-overview
Supports
- Vertex AI training, evaluation, registry, deployment, monitoring, and managed MLOps scope
- Vertex AI product placement
- https://cloud.google.com/vertex-ai/pricing
Supports
- Vertex AI paid managed-service classification
- https://azure.microsoft.com/en-us/pricing/details/machine-learning/
Supports
- Azure Machine Learning paid managed-service classification
- https://aws.amazon.com/sagemaker-ai/pricing/
Supports
- Amazon SageMaker AI paid managed-service classification
- https://www.databricks.com/product/machine-learning
Supports
- Databricks MLflow, feature, model, workflow, deployment, and monitoring product scope
- Databricks Machine Learning product placement
- https://www.databricks.com/product/pricing
Supports
- Databricks paid managed-platform classification
- https://github.com/mlflow/mlflow/blob/master/LICENSE.txt
Supports
- MLflow Apache-2.0 licensing and free open-source classification
- https://github.com/kubeflow/kubeflow/blob/master/LICENSE
Supports
- Kubeflow Apache-2.0 licensing and free open-source classification
- https://docs.gitlab.com/user/project/ml/model_registry/
Supports
- GitLab model versions, metadata, MLflow compatibility, and links from model versions to CI/CD jobs
- GitLab placement in the ML delivery landscape
- https://about.gitlab.com/pricing/
Supports
- GitLab free and paid tiers behind the freemium classification
- https://handbook.gitlab.com/handbook/engineering/open-source/
Supports
- GitLab open-core product model
- https://research.google/pubs/hidden-technical-debt-in-machine-learning-systems/
Supports
- 2015 publication and system-level ML technical-debt concerns
- https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/
Supports
- 2017 publication and 28 tests and monitoring needs for production ML
- https://blog.kubeflow.org/releases/2020/03/02/kubeflow-1-0-cloud-native-ml-for-everyone.html
Supports
- Kubeflow open sourcing in December 2017
- Kubeflow 1.0 release on March 2, 2020 and its stable production ML core
- https://www.databricks.com/sites/default/files/2023-08/accelerating-the-machine-learning-lifecycle-with-mlflow.pdf
Supports
- MLflow launch in June 2018
- MLflow's original experimentation, reproducibility, and deployment scope
- https://arxiv.org/abs/1903.00278
Supports
- March 2019 publication of an ML-specific continuous integration system
- Statistical model-quality decisions in ML integration
- https://dvc.org/blog/cml-release/
Supports
- CML release on July 7, 2020
- Training, evaluation, review reports, and compute in GitHub and GitLab CI
- https://aws.amazon.com/about-aws/whats-new/2020/12/introducing-amazon-sagemaker-pipelines-first-purpose-built-ci-cd-service-machine-learning/
Supports
- SageMaker Pipelines general availability on December 8, 2020
- Managed workflow orchestration, registry approval, lineage, and MLOps templates