Skip to content

Machine Learning Associate · Domain 1: Databricks Machine Learning · 38% of the exam

Task 1.4: Identify the best run using the MLflow Client API; Manually log metrics, artifacts, and models in an MLflow Run; Identify information available in the MLflow UI

Experiments and runs, search_runs with filters and ordering, logging parameters, metrics, artifacts and models by hand, MLflow 3 logged models, and what the run and experiment pages show.

Study it

  • MLflow tracking: runs, manual logging, search_runs and the MLflow UI

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

Runs of several model families were logged to one experiment, each with the parameter model_family and the metric val_rmse. A data scientist needs the XGBoost run with the lowest val_rmse, with ties going to the most recent run. Which call returns that run in the first row of its result?

  1. Amlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse DESC", "start_time DESC"])
  2. Bmlflow.search_runs(filter_string="metrics.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time DESC"])
  3. Cmlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time ASC"])
  4. Dmlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time DESC"])
Show the answer and why
  • Amlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse DESC", "start_time DESC"])

    Incorrect

    Descending order on val_rmse puts the run with the highest error first.

  • Bmlflow.search_runs(filter_string="metrics.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time DESC"])

    Incorrect

    model_family was logged as a parameter, so the filter must use the params. prefix, not metrics.

  • Cmlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time ASC"])

    Incorrect

    Ascending start_time breaks ties in favour of the oldest run, not the most recent one.

  • Dmlflow.search_runs(filter_string="params.model_family = 'xgboost'", order_by=["metrics.val_rmse ASC", "start_time DESC"])

    Correct

    The filter keeps the XGBoost runs, ascending val_rmse puts the lowest error first, and descending start_time picks the newest run on a tie.

mlflow.search_runs returns the matching runs as a pandas DataFrame. Filters address parameters with params., metrics with metrics. and tags with tags., and order_by takes a list of columns with ASC or DESC; Databricks' own tutorial sorts by a metric and then by start_time DESC so that ties go to the most recent run.

Question 2 · choose 1

With mlflow.sklearn.autolog() turned on, a data scientist trains a classifier inside a with mlflow.start_run() block, and autologging records the model, its parameters and training metrics. She also wants the test-set AUC and a confusion matrix saved locally as cm.png to appear on the same run. What should she add inside the block?

  1. Amlflow.set_tag("test_auc", auc) and mlflow.log_artifact("cm.png")
  2. Bmlflow.log_metric("test_auc", auc) and mlflow.log_artifact("cm.png")
  3. CNothing more, because autologging also records test-set metrics and plots
  4. Dmlflow.log_metric("test_auc", auc) and mlflow.log_input("cm.png")
Show the answer and why
  • Amlflow.set_tag("test_auc", auc) and mlflow.log_artifact("cm.png")

    Incorrect

    A tag is run metadata used for searching, not a metric value, so the AUC would not be recorded as a metric.

  • Bmlflow.log_metric("test_auc", auc) and mlflow.log_artifact("cm.png")

    Correct

    log_metric records the AUC as a metric of the active run, and log_artifact uploads the image file to the run's artifacts.

  • CNothing more, because autologging also records test-set metrics and plots

    Incorrect

    Autologging does not record the test-set AUC; Databricks' tutorial logs it manually.

  • Dmlflow.log_metric("test_auc", auc) and mlflow.log_input("cm.png")

    Incorrect

    log_input records the input dataset of a run for lineage; it does not upload an image file.

Autologging captures the model, parameters and training metrics. Values that it does not capture, such as a test-set AUC, are logged with mlflow.log_metric inside the run, and output files such as plots are logged with mlflow.log_artifact, after which they appear in the run's Artifacts tab.

Question 3 · choose 1

Every run in an experiment logs val_loss after each epoch, and all the runs were logged in 2026. A data scientist wants to find, in the runs table on the experiment page, the runs that overfit late: val_loss went below 0.2 at some epoch, but the value logged at the end of training is above 0.3. Which search expression should she enter?

  1. AMAX(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
  2. Bmetrics.val_loss < 0.2 AND MAX(metrics.val_loss) > 0.3
  3. CLATEST(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2
  4. DLATEST(metrics.val_loss) > 0.3 AND metrics.val_loss < 0.2
Show the answer and why
  • AMAX(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2

    Incorrect

    MAX is the highest value logged at any epoch, so runs that started above 0.3 and ended at their best value also match.

  • Bmetrics.val_loss < 0.2 AND MAX(metrics.val_loss) > 0.3

    Incorrect

    A plain metric filter uses the last logged value, so this finds runs that ended below 0.2.

  • CLATEST(metrics.val_loss) > 0.3 AND MIN(metrics.val_loss) < 0.2

    Correct

    LATEST filters on the last logged value and MIN on the lowest value logged, which together describe a run that got worse after its best epoch.

  • DLATEST(metrics.val_loss) > 0.3 AND metrics.val_loss < 0.2

    Incorrect

    A plain metric filter also uses the last logged value, so both conditions test the same value and no run matches.

The runs table search filters metric values on the last logged value by default, and LATEST names that value explicitly. MIN and MAX search on the minimum or maximum value logged for a metric, and conditions can be combined with AND; only runs logged after August 2024 have minimum and maximum metric values.

Question 4 · choose 1

A data scientist starts MLflow runs in a Databricks notebook without calling mlflow.set_experiment. Where are the runs logged?

  1. AIn a new workspace experiment named after her user
  2. BIn the notebook experiment of that notebook
  3. CNowhere, until an experiment is set
  4. DIn the Shared folder's default experiment
Show the answer and why
  • AIn a new workspace experiment named after her user

    Incorrect

    Without an active experiment, runs go to the notebook experiment; a workspace experiment must be set explicitly.

  • BIn the notebook experiment of that notebook

    Correct

    If no active experiment is set, runs are logged to the notebook experiment.

  • CNowhere, until an experiment is set

    Incorrect

    Runs are logged to the notebook experiment by default.

  • DIn the Shared folder's default experiment

    Incorrect

    There is no such default; the notebook experiment receives the runs.

All MLflow runs are logged to the active experiment. In a Databricks notebook, if no experiment is set as active, runs are logged to the notebook experiment; mlflow.set_experiment selects a workspace experiment instead.

Question 5 · choose 1

A team moves its training code to MLflow 3. Its calls look like mlflow.sklearn.log_model(model, artifact_path="model") inside a run. What does Databricks recommend changing?

  1. AUse name="model" instead of artifact_path; a run no longer has to be active to log a model
  2. BNothing; artifact_path is still the recommended parameter in MLflow 3
  3. CLog the model with mlflow.log_artifact instead of log_model
  4. DKeep artifact_path and also pass the run_uuid of the active run
Show the answer and why
  • AUse name="model" instead of artifact_path; a run no longer has to be active to log a model

    Correct

    In MLflow 3, name replaces the deprecated artifact_path, and models can be logged without an active run.

  • BNothing; artifact_path is still the recommended parameter in MLflow 3

    Incorrect

    artifact_path is still supported but deprecated in MLflow 3.

  • CLog the model with mlflow.log_artifact instead of log_model

    Incorrect

    log_artifact uploads files; models are logged with the flavor's log_model.

  • DKeep artifact_path and also pass the run_uuid of the active run

    Incorrect

    run_uuid has been removed from RunInfo in MLflow 3; the change is to use name.

In MLflow 3, log models with name, which also lets you search for the model by name later; artifact_path is still supported but deprecated. Models are first-class objects, so MLflow no longer requires an active run when logging one.

Question 6 · choose 1

On an MLflow run page, a reviewer wants to see the exact version of the notebook that produced the run. Where does she click?

  1. AThe Artifacts tab
  2. BThe Metrics chart
  3. CThe link next to Source
  4. DThe Tags field in the Details table
Show the answer and why
  • AThe Artifacts tab

    Incorrect

    The Artifacts tab lists files saved from the run, such as models.

  • BThe Metrics chart

    Incorrect

    Metric charts show logged metric values, not the source notebook.

  • CThe link next to Source

    Correct

    The Source link opens the version of the notebook that created the run, with the run's date and time highlighted.

  • DThe Tags field in the Details table

    Incorrect

    Tags are key-value metadata for searching runs.

The run screen shows the run ID, parameters, metrics and details including a link to the source notebook. Clicking the link next to Source shows the version of the notebook associated with the run, with a highlight bar for the run's date and time.

Question 7 · choose 1

Runs carry a tag whose key contains spaces, data source, with values such as crm export. Which search expression in the experiment's runs table finds those runs?

  1. Atags.data source = crm export
  2. Btags.`data source` = "crm export"
  3. Cparams.data_source = "crm export"
  4. Dtags["data source"] == "crm export"
Show the answer and why
  • Atags.data source = crm export

    Incorrect

    A key with spaces needs backticks, and string values need quotes.

  • Btags.`data source` = "crm export"

    Correct

    Keys that include spaces are enclosed in backticks, and string values in quotes.

  • Cparams.data_source = "crm export"

    Incorrect

    The value was logged as a tag, not a parameter, and the key has a space.

  • Dtags["data source"] == "crm export"

    Incorrect

    Run search uses the tags.<key>="<value>" syntax, not brackets.

Runs can be searched by tag with tags.<key>="<value>", with string values in quotes. Keys and values can contain spaces; a key with spaces is enclosed in backticks, as in tags.`my custom tag` = "my value".

Question 8 · choose 1

An analyst wants a CSV of 450 runs from an experiment, with their parameters and metrics. What does the experiment page offer?

  1. ADownload <n> runs, which exports all 450 runs at once
  2. BNothing; runs can be exported only through the REST API
  3. CDownload all runs, which gives a code snippet to run in a notebook and download the rows
  4. DAn export to a Unity Catalog volume from the run page
Show the answer and why
  • ADownload <n> runs, which exports all 450 runs at once

    Incorrect

    Download <n> runs exports at most 100 runs.

  • BNothing; runs can be exported only through the REST API

    Incorrect

    The experiment page offers both a direct download and a code snippet.

  • CDownload all runs, which gives a code snippet to run in a notebook and download the rows

    Correct

    For more than 100 runs, Download all runs opens a code snippet whose output can be downloaded in full.

  • DAn export to a Unity Catalog volume from the run page

    Incorrect

    Runs are downloaded from the experiment page's menu, not exported from a run page.

From the experiment page's menu, Download <n> runs creates a CSV of the runs shown, up to 100, with fields such as run ID, parameters and metrics. For more than 100 runs, or programmatic downloads, Download all runs shows a code snippet to run in a notebook, after which all rows can be downloaded.

Question 9 · choose 1

In MLflow 3, mlflow.sklearn.log_model returns information about the logged model. Which model URI does Databricks recommend for loading it later with mlflow.sklearn.load_model?

  1. Amodels:/<model_id>, as returned by log_model
  2. Bruns:/<run_id>/artifacts/model, the run's artifact folder
  3. Cdbfs:/mlflow/<experiment_name>/model
  4. Dmodels:/<experiment_id>/latest
Show the answer and why
  • Amodels:/<model_id>, as returned by log_model

    Correct

    MLflow 3 stores model artifacts under the model, and the recommended URI is the models:/<model_id> URI that log_model returns.

  • Bruns:/<run_id>/artifacts/model, the run's artifact folder

    Incorrect

    In MLflow 3, model artifacts are no longer stored as run artifacts.

  • Cdbfs:/mlflow/<experiment_name>/model

    Incorrect

    This is not a URI that MLflow documents for loading a model.

  • Dmodels:/<experiment_id>/latest

    Incorrect

    Models are addressed by model ID or registered name, not experiment ID.

In MLflow 3, models are first-class objects and their artifacts are stored under the model's own path. Databricks recommends loading with the model URI returned by log_model, which has the form models:/<model_id>, rather than the MLflow 2.x runs:/<run_id>/<artifact_path>.

Practice domain 1 →Practice all domains →