Model Registries
A model registry is a versioned catalog of trained models — storing artifacts, metadata, lineage, and stage labels — so that promoting, rolling back, and auditing models is systematic rather than an ad-hoc file-copying operation.
Why Does This Exist?
Without a registry, "deploying the new model" means finding the right pickle file on a shared drive, copying it to a server, and hoping nobody accidentally overwrites it. Rollback means finding the previous pickle file and hoping it was saved. Audit means asking the person who ran the training script what hyperparameters they used — if they still remember.
A model registry provides a structured catalog with version history, full metadata, approval workflows, and a deployment audit trail — the same rigor that software has had for decades, applied to model artifacts.
Think of It Like This
A library catalog with checkout records
A library catalog doesn't just store books — it records which books are available, who checked them out, when, and their condition history. You can find any version of a book, see its full history, and know exactly what's currently in circulation. A model registry is the library catalog for ML models: you know which version is in production, who approved it, what data trained it, and what metrics it achieved.
How It Actually Works
What gets stored
A model registry entry contains:
- Model artifact: the serialized model file (pickle, ONNX, SavedModel, PyTorch checkpoint, PMML)
- Metrics: validation accuracy, AUC, RMSE, F1 — whatever was logged during the training run
- Parameters: hyperparameters, feature list, preprocessing config, random seeds
- Training lineage: which dataset version, which code commit, which experiment run produced this model
- Stage label: None → Staging → Production → Archived
- Tags: arbitrary metadata (team, use case, regulatory status, model type)
Stage transitions
Models move through stages with human or automated approval:
- None: Model just logged, not yet reviewed
- Staging: Passed automated tests; ready for human review and A/B testing
- Production: Serving live traffic — exactly one model version should be here at a time
- Archived: Superseded by a newer version, preserved for audit and rollback
Webhooks on stage transitions can trigger CI/CD pipelines: "When a model moves to Production, automatically redeploy the serving endpoint."
Why one model in Production matters
If multiple model versions are simultaneously in Production, you don't know which one is serving any given request, comparison metrics become meaningless, and rollback becomes ambiguous. Enforce: when promoting a new version to Production, archive the existing one atomically.
Code
import mlflowfrom mlflow.tracking import MlflowClient
client = MlflowClient()
# ── Step 1: Log a model artifact during training ──────────────────────────────with mlflow.start_run() as run: mlflow.log_param("learning_rate", 0.01) mlflow.log_param("n_estimators", 200) mlflow.log_metric("val_auc", 0.923) mlflow.log_metric("val_accuracy", 0.891) # Log the model (replace with your actual model) from sklearn.ensemble import GradientBoostingClassifier model = GradientBoostingClassifier(n_estimators=200, learning_rate=0.01) # model.fit(X_train, y_train) # assume trained mlflow.sklearn.log_model(model, "model") run_id = run.info.run_id print(f"Run ID: {run_id}")
# ── Step 2: Register in the model registry ────────────────────────────────────model_uri = f"runs:/{run_id}/model"mv = mlflow.register_model(model_uri, "fraud-detector")print(f"Registered: name='fraud-detector', version={mv.version}")
# ── Step 3: Add description and tags ─────────────────────────────────────────client.update_model_version( name="fraud-detector", version=mv.version, description="GBM trained on 2024 transaction data. Val AUC=0.923.")client.set_model_version_tag("fraud-detector", mv.version, "team", "risk")client.set_model_version_tag("fraud-detector", mv.version, "data_cutoff", "2024-08-31")
# ── Step 4: Promote to Staging ────────────────────────────────────────────────client.transition_model_version_stage( name="fraud-detector", version=mv.version, stage="Staging", archive_existing_versions=False,)print(f"Version {mv.version} → Staging")
# ── Step 5: After review, promote to Production ───────────────────────────────client.transition_model_version_stage( name="fraud-detector", version=mv.version, stage="Production", archive_existing_versions=True, # archive the old production version)print(f"Version {mv.version} → Production (previous version archived)")
# ── Step 6: Load from registry at serving time ───────────────────────────────# This always loads whichever version is currently in Productionproduction_model = mlflow.sklearn.load_model("models:/fraud-detector/Production")# predictions = production_model.predict(X_inference)Watch Out For
Treating the registry as just file storage
A registry is only valuable if the metadata is maintained: metrics, parameters, lineage, stage labels, and descriptions. A registry with artifact blobs but no metadata is just a more expensive file share — you still can't answer "why did we deploy this model" or "what data trained it." Automate metadata logging as part of every training run, not as a manual afterthought.
Multiple models simultaneously in Production
Keeping multiple models in the Production stage creates ambiguity: which one is actually serving traffic? This happens when teams forget to archive the old version when promoting the new one. Enforce the invariant: Production contains exactly one model version at all times. Use archive_existing_versions=True when transitioning to Production.
The Quick Version
- A model registry stores model artifacts with version history, metrics, parameters, lineage, and stage labels.
- Stage labels (None → Staging → Production → Archived) formalize the promotion and rollback process.
- Webhooks on stage transitions enable automated CI/CD deployment pipelines.
- Loading models via registry aliases (
models:/fraud-detector/Production) decouples serving code from specific version numbers. - Production should contain exactly one model version at all times; use
archive_existing_versions=Truewhen promoting.