Calabi ML — Experiment Tracking
Professional+
Calabi ML provides end-to-end ML lifecycle management in Calabi: experiment tracking, run comparison, model registry, and artifact storage.
Accessing Calabi ML
Navigate to calabi.{domain}/mlflow. Your Calabi credentials work automatically.
Core Concepts
| Concept | Description |
|---|---|
| Experiment | A named group of runs (e.g., "psychometric-scoring-v3") |
| Run | One model training execution — logs params, metrics, and artifacts |
| Parameter | A training hyperparameter (e.g., learning_rate: 0.01) |
| Metric | A measured outcome (e.g., val_accuracy: 0.91) |
| Artifact | Files saved during training (model files, plots, data samples) |
| Model Registry | A versioned store of production-candidate models |
Logging from Python
import mlflow
mlflow.set_tracking_uri("https://calabi.{domain}/mlflow")
mlflow.set_experiment("psychometric-scoring-v3")
with mlflow.start_run():
mlflow.log_param("learning_rate", 0.01)
mlflow.log_param("max_depth", 6)
# ... train model ...
mlflow.log_metric("train_accuracy", 0.95)
mlflow.log_metric("val_accuracy", 0.91)
mlflow.log_metric("test_f1", 0.89)
# Save model
mlflow.sklearn.log_model(model, "model")
Backend Storage
| Component | Storage | Location |
|---|---|---|
| Experiments, runs, metrics, params | database | calabi_mlflow_{tenant} |
| Model files, plots, artifacts | Amazon S3 | {tenant}-mlflow-artifacts |