MLflow for Experiment Tracking and Model Registry
MLflow is an open-source platform that records the details of every machine learning training run — its parameters, metrics, and output files — and keeps a versioned catalog of the models built from those runs, so teams can compare experiments and know exactly which model is running in production.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
MLflow for Experiment Tracking and Model Registry
MLflow is an open-source platform, now a Linux Foundation project, for managing the machine learning lifecycle. This course covers its two foundational components: Tracking, which records the parameters, metrics, and outputs of every training run, and the Model Registry, which catalogs the models built from those runs and manages their progression toward production. MLflow also includes Projects (packaging training code) and Deployment (serving models), which this course covers at the level needed to understand how tracking and the registry connect to the rest of the lifecycle.
The problem MLflow solves
Training a model produces more than a file of weights. It produces a specific combination of code version, hyperparameters, training data, library versions, and resulting metrics. Without a system to record that combination, teams lose track of which configuration produced which result. Two common failures follow: a team cannot reproduce a promising result because nobody recorded the exact hyperparameters used, and a team cannot tell which of twenty saved model files is the one actually serving production traffic. MLflow addresses both by making every training run and every registered model a queryable record instead of a file on someone's laptop.
Architecture: experiments, runs, and logged models
MLflow organizes work into a three-level hierarchy:
- An experiment is a named container for a set of related runs — for example, all runs that train a fraud-detection classifier.
- A run is one execution of training code, such as one invocation of
python train.py. Each run has a unique run ID and records everything logged during that execution. - A logged model is a trained artifact produced during a run. Starting with MLflow 3, logged models are first-class entities with their own identity, rather than files nested under a run's artifact folder — a logged model links back to the run that produced it, but you address and query it independently.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://mlflow.org/docs/latest/
Supports
- MLflow is an open-source AI platform / Linux Foundation project
- MLflow's two main functional areas (LLMs & Agents, and traditional Machine Learning: tracking, model packaging, registry, deployment)
- Availability of both open-source MLflow and managed MLflow on Databricks
- https://mlflow.org/docs/latest/ml/tracking/
Supports
- Experiment, run, and logged model hierarchy
- What a run logs (parameters, metrics, artifacts, tags, log_input for datasets)
- Backend store vs. artifact store distinction
- Tracking server as an optional standalone HTTP process
- mlflow ui vs. mlflow server distinction
- MLFLOW_TRACKING_URI as the client-side pointer to a tracking server
- https://mlflow.org/docs/latest/ml/tracking/autolog
Supports
- mlflow.autolog() behavior and what it captures
- Per-library autologging coverage and limitations (scikit-learn, XGBoost, LightGBM, Keras/TensorFlow, PyTorch Lightning, PySpark ML, Statsmodels, Paddle)
- https://mlflow.org/docs/latest/ml/model-registry/
Supports
- Registered models, model versions, and automatic version numbering
- models:/<name>/<version> URI format
- Model lineage linking a version back to its producing run
- Tags and aliases as registry metadata mechanisms
- https://mlflow.org/docs/latest/ml/model-registry/workflow
Supports
- Aliases (e.g. champion, challenger) as mutable named references to a model version
- Stage deprecation since MLflow 2.9 and its rationale
- Environment-scoped registered models (dev/staging/prod) as the current isolation pattern
- copy_model_version() for promoting a version across environment-scoped models
- models:/<name>@<alias> URI format
- https://mlflow.org/docs/latest/ml/model/
Supports
- MLmodel file structure and purpose
- Model flavors concept, including python_function as the universal flavor
- Model signature and input example
- serving_input_example.json generation
- Bundled dependency files (requirements.txt, conda.yaml, python_env.yaml)
- https://mlflow.org/docs/latest/ml/projects/
Supports
- MLproject file structure (name, environment, entry points)
- Environment types (virtualenv, Conda, Docker, system) and their tradeoffs
- mlflow run command and execution against local or remote Git projects
- https://mlflow.org/docs/latest/ml/deployment/
Supports
- mlflow models serve for local REST serving
- mlflow models build-docker for containerized deployment
- Managed deployment targets (SageMaker, Azure ML, Databricks Model Serving)
- Community-supported Kubernetes targets (KServe, Ray Serve)
- python_function flavor as the common interface across deployment targets
- https://mlflow.org/docs/latest/ml/mlflow-3/
Supports
- LoggedModel as a first-class entity introduced in MLflow 3
- Model artifacts moved from run-relative paths to experiments/<id>/models/<id>/artifacts/
- Impact on mlflow.get_artifact_uri("model") and other run-relative model access code
- log_model callable without mlflow.start_run() context in MLflow 3
- https://mlflow.org/releases/
Supports
- Current MLflow release version (3.15.1) and release date used for course freshness snapshot
- https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.MLflowCallback.html
Supports
- Optuna's MLflowCallback logging trial parameters and scores into an MLflow experiment (Awesome Links rationale)
- https://docs.bentoml.com/en/latest/reference/frameworks/mlflow.html
Supports
- BentoML's dedicated MLflow framework integration for importing and serving MLflow-format models (Awesome Links rationale)
- https://github.com/SeldonIO/MLServer/tree/master/runtimes/mlflow
Supports
- MLServer's dedicated MLflow runtime and V2 Inference Protocol signature conversion (Awesome Links rationale)
- https://doc.dvc.org/
Supports
- DVC's purpose: versioning datasets and models alongside code, reproducible pipelines (Awesome Links rationale)
- https://wandb.ai/site
Supports
- Weights & Biases as an experiment tracking and ML platform covering tracking, registry, and sweeps (Awesome Links rationale)
- https://www.kubeflow.org/
Supports
- Kubeflow as a Kubernetes-native AI platform
- Kubeflow Hub as a model registry component (Awesome Links rationale)
- https://www.evidentlyai.com/
Supports
- Evidently AI's purpose: monitoring data drift and predictive/AI system performance after deployment (Awesome Links rationale)