Machine Learning Associate · Domain 1: Databricks Machine Learning · 38% of the exam
Task 1.4: Identify the best run using the MLflow Client API; Manually log metrics, artifacts, and models in an MLflow Run; Identify information available in the MLflow UI
Experiments and runs, search_runs with filters and ordering, logging parameters, metrics, artifacts and models by hand, MLflow 3 logged models, and what the run and experiment pages show.
Study it
MLflow tracking: runs, manual logging, search_runs and the MLflow UI
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
Runs of several model families were logged to one experiment, each with the parameter model_family and the metric val_rmse. A data scientist needs the XGBoost run with the lowest val_rmse, with ties going to the most recent run. Which call returns that run in the first row of its result?
The filter keeps the XGBoost runs, ascending val_rmse puts the lowest error first, and descending start_time picks the newest run on a tie.
mlflow.search_runs returns the matching runs as a pandas DataFrame. Filters address parameters with params., metrics with metrics. and tags with tags., and order_by takes a list of columns with ASC or DESC; Databricks' own tutorial sorts by a metric and then by start_time DESC so that ties go to the most recent run.
With mlflow.sklearn.autolog() turned on, a data scientist trains a classifier inside a with mlflow.start_run() block, and autologging records the model, its parameters and training metrics. She also wants the test-set AUC and a confusion matrix saved locally as cm.png to appear on the same run. What should she add inside the block?
Amlflow.set_tag("test_auc", auc) and mlflow.log_artifact("cm.png")
Bmlflow.log_metric("test_auc", auc) and mlflow.log_artifact("cm.png")
CNothing more, because autologging also records test-set metrics and plots
Dmlflow.log_metric("test_auc", auc) and mlflow.log_input("cm.png")
Show the answer and why
Amlflow.set_tag("test_auc", auc) and mlflow.log_artifact("cm.png")
Incorrect
A tag is run metadata used for searching, not a metric value, so the AUC would not be recorded as a metric.
Bmlflow.log_metric("test_auc", auc) and mlflow.log_artifact("cm.png")
Correct
log_metric records the AUC as a metric of the active run, and log_artifact uploads the image file to the run's artifacts.
CNothing more, because autologging also records test-set metrics and plots
Incorrect
Autologging does not record the test-set AUC; Databricks' tutorial logs it manually.
Dmlflow.log_metric("test_auc", auc) and mlflow.log_input("cm.png")
Incorrect
log_input records the input dataset of a run for lineage; it does not upload an image file.
Autologging captures the model, parameters and training metrics. Values that it does not capture, such as a test-set AUC, are logged with mlflow.log_metric inside the run, and output files such as plots are logged with mlflow.log_artifact, after which they appear in the run's Artifacts tab.
Every run in an experiment logs val_loss after each epoch, and all the runs were logged in 2026. A data scientist wants to find, in the runs table on the experiment page, the runs that overfit late: val_loss went below 0.2 at some epoch, but the value logged at the end of training is above 0.3. Which search expression should she enter?
AMAX(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
Bmetrics.val_loss < 0.2 AND MAX(metrics.val_loss) > 0.3
CLATEST(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
DLATEST(metrics.val_loss) > 0.3 AND metrics.val_loss < 0.2
Show the answer and why
AMAX(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
Incorrect
MAX is the highest value logged at any epoch, so runs that started above 0.3 and ended at their best value also match.
Bmetrics.val_loss < 0.2 AND MAX(metrics.val_loss) > 0.3
Incorrect
A plain metric filter uses the last logged value, so this finds runs that ended below 0.2.
CLATEST(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
Correct
LATEST filters on the last logged value and MIN on the lowest value logged, which together describe a run that got worse after its best epoch.
DLATEST(metrics.val_loss) > 0.3 AND metrics.val_loss < 0.2
Incorrect
A plain metric filter also uses the last logged value, so both conditions test the same value and no run matches.
The runs table search filters metric values on the last logged value by default, and LATEST names that value explicitly. MIN and MAX search on the minimum or maximum value logged for a metric, and conditions can be combined with AND; only runs logged after August 2024 have minimum and maximum metric values.
A data scientist starts MLflow runs in a Databricks notebook without calling mlflow.set_experiment. Where are the runs logged?
AIn a new workspace experiment named after her user
BIn the notebook experiment of that notebook
CNowhere, until an experiment is set
DIn the Shared folder's default experiment
Show the answer and why
AIn a new workspace experiment named after her user
Incorrect
Without an active experiment, runs go to the notebook experiment; a workspace experiment must be set explicitly.
BIn the notebook experiment of that notebook
Correct
If no active experiment is set, runs are logged to the notebook experiment.
CNowhere, until an experiment is set
Incorrect
Runs are logged to the notebook experiment by default.
DIn the Shared folder's default experiment
Incorrect
There is no such default; the notebook experiment receives the runs.
All MLflow runs are logged to the active experiment. In a Databricks notebook, if no experiment is set as active, runs are logged to the notebook experiment; mlflow.set_experiment selects a workspace experiment instead.
A team moves its training code to MLflow 3. Its calls look like mlflow.sklearn.log_model(model, artifact_path="model") inside a run. What does Databricks recommend changing?
AUse name="model" instead of artifact_path; a run no longer has to be active to log a model
BNothing; artifact_path is still the recommended parameter in MLflow 3
CLog the model with mlflow.log_artifact instead of log_model
DKeep artifact_path and also pass the run_uuid of the active run
Show the answer and why
AUse name="model" instead of artifact_path; a run no longer has to be active to log a model
Correct
In MLflow 3, name replaces the deprecated artifact_path, and models can be logged without an active run.
BNothing; artifact_path is still the recommended parameter in MLflow 3
Incorrect
artifact_path is still supported but deprecated in MLflow 3.
CLog the model with mlflow.log_artifact instead of log_model
Incorrect
log_artifact uploads files; models are logged with the flavor's log_model.
DKeep artifact_path and also pass the run_uuid of the active run
Incorrect
run_uuid has been removed from RunInfo in MLflow 3; the change is to use name.
In MLflow 3, log models with name, which also lets you search for the model by name later; artifact_path is still supported but deprecated. Models are first-class objects, so MLflow no longer requires an active run when logging one.
On an MLflow run page, a reviewer wants to see the exact version of the notebook that produced the run. Where does she click?
AThe Artifacts tab
BThe Metrics chart
CThe link next to Source
DThe Tags field in the Details table
Show the answer and why
AThe Artifacts tab
Incorrect
The Artifacts tab lists files saved from the run, such as models.
BThe Metrics chart
Incorrect
Metric charts show logged metric values, not the source notebook.
CThe link next to Source
Correct
The Source link opens the version of the notebook that created the run, with the run's date and time highlighted.
DThe Tags field in the Details table
Incorrect
Tags are key-value metadata for searching runs.
The run screen shows the run ID, parameters, metrics and details including a link to the source notebook. Clicking the link next to Source shows the version of the notebook associated with the run, with a highlight bar for the run's date and time.
Runs carry a tag whose key contains spaces, data source, with values such as crm export. Which search expression in the experiment's runs table finds those runs?
Atags.data source = crm export
Btags.`data source` = "crm export"
Cparams.data_source = "crm export"
Dtags["data source"] == "crm export"
Show the answer and why
Atags.data source = crm export
Incorrect
A key with spaces needs backticks, and string values need quotes.
Btags.`data source` = "crm export"
Correct
Keys that include spaces are enclosed in backticks, and string values in quotes.
Cparams.data_source = "crm export"
Incorrect
The value was logged as a tag, not a parameter, and the key has a space.
Dtags["data source"] == "crm export"
Incorrect
Run search uses the tags.<key>="<value>" syntax, not brackets.
Runs can be searched by tag with tags.<key>="<value>", with string values in quotes. Keys and values can contain spaces; a key with spaces is enclosed in backticks, as in tags.`my custom tag` = "my value".
An analyst wants a CSV of 450 runs from an experiment, with their parameters and metrics. What does the experiment page offer?
ADownload <n> runs, which exports all 450 runs at once
BNothing; runs can be exported only through the REST API
CDownload all runs, which gives a code snippet to run in a notebook and download the rows
DAn export to a Unity Catalog volume from the run page
Show the answer and why
ADownload <n> runs, which exports all 450 runs at once
Incorrect
Download <n> runs exports at most 100 runs.
BNothing; runs can be exported only through the REST API
Incorrect
The experiment page offers both a direct download and a code snippet.
CDownload all runs, which gives a code snippet to run in a notebook and download the rows
Correct
For more than 100 runs, Download all runs opens a code snippet whose output can be downloaded in full.
DAn export to a Unity Catalog volume from the run page
Incorrect
Runs are downloaded from the experiment page's menu, not exported from a run page.
From the experiment page's menu, Download <n> runs creates a CSV of the runs shown, up to 100, with fields such as run ID, parameters and metrics. For more than 100 runs, or programmatic downloads, Download all runs shows a code snippet to run in a notebook, after which all rows can be downloaded.
In MLflow 3, mlflow.sklearn.log_model returns information about the logged model. Which model URI does Databricks recommend for loading it later with mlflow.sklearn.load_model?
Amodels:/<model_id>, as returned by log_model
Bruns:/<run_id>/artifacts/model, the run's artifact folder
Cdbfs:/mlflow/<experiment_name>/model
Dmodels:/<experiment_id>/latest
Show the answer and why
Amodels:/<model_id>, as returned by log_model
Correct
MLflow 3 stores model artifacts under the model, and the recommended URI is the models:/<model_id> URI that log_model returns.
Bruns:/<run_id>/artifacts/model, the run's artifact folder
Incorrect
In MLflow 3, model artifacts are no longer stored as run artifacts.
Cdbfs:/mlflow/<experiment_name>/model
Incorrect
This is not a URI that MLflow documents for loading a model.
Dmodels:/<experiment_id>/latest
Incorrect
Models are addressed by model ID or registered name, not experiment ID.
In MLflow 3, models are first-class objects and their artifacts are stored under the model's own path. Databricks recommends loading with the model URI returned by log_model, which has the form models:/<model_id>, rather than the MLflow 2.x runs:/<run_id>/<artifact_path>.