Task 2.1: Orchestrate model training

• AKeep the hand-built URI but switch its protocol from azureml:// to https:// so the call goes through the private endpoint

  Incorrect

  The URI's whole format differs for private link workspaces, not just the protocol.

• BInstall mlflow and azureml-mlflow, get mlflow_tracking_uri from az ml workspace show or the SDK, then set it

  Correct

  With private link the tracking URI has a different format, so it has to come from the SDK or CLI.

• CSkip set_tracking_uri and let MLflow find the workspace automatically wherever the code runs

  Incorrect

  Automatic configuration applies only on Azure Machine Learning compute, not in Databricks.

• DLog the runs to the Databricks tracking server and import them into the workspace with the Azure Machine Learning SDK v2

  Incorrect

  SDK v2 doesn't provide logging or tracking; the workspace itself acts as the MLflow server.

Azure Machine Learning workspaces act as MLflow tracking servers. On Azure Machine Learning compute the tracking URI is configured automatically; elsewhere you install mlflow and the azureml-mlflow plugin and point MLflow at the workspace's tracking URI. For a private link workspace the URI format differs from the one you can build by hand, so get it with the SDK or az ml workspace show --query mlflow_tracking_uri.

• AUse order_by=["attributes.auc DESC"], since metrics are exposed as run attributes

  Incorrect

  auc is a logged metric, not a run attribute such as start_time or duration.

• BAdd max_results=10 to search_runs for the experiment so that MLflow returns the ten runs with the highest metric

  Incorrect

  max_results only caps the count; it doesn't order the runs by auc.

• CQuery the runs with the Azure Machine Learning SDK v2 jobs.list method, which returns metrics in order

  Incorrect

  SDK v2 doesn't provide logging or tracking queries; use MLflow.

• DCall search_runs for the experiment and sort the returned DataFrame by metrics.auc with pandas

  Correct

  order_by doesn't support metrics.*, params.* or tags.* expressions, so sort the results in pandas.

In Azure Machine Learning, mlflow.search_runs supports order_by on attributes such as start_time and duration, but not expressions containing metrics.*, params.* or tags.*. The documented approach is to retrieve the runs and use pandas sort_values on the metric column.

• ARaise the Bandit policy's slack_factor so that more trials fall outside the allowed range and stop

  Incorrect

  No trial reports the primary metric, so no slack setting can compare them.

• BLower delay_evaluation to 0 so that the policy starts evaluating trials from the first interval

  Incorrect

  Delay controls when evaluation starts; it can't evaluate a metric that is never logged.

• CMake the logged metric name and primary_metric match exactly, for example both accuracy

  Correct

  The sweep evaluates and terminates trials only on the metric with the exact name in primary_metric.

• DChange the primary metric's goal to minimize, since accuracy is compared from the lowest value

  Incorrect

  The goal direction doesn't fix a metric name that doesn't match.

Each sweep trial is evaluated on the primary metric, and the early termination policy uses that metric to find low performers. The name in primary_metric must exactly match the name the training script logs, for example mlflow.log_metric("accuracy", ...).

• AAutoML holds out 10% of the rows for validation, and accuracy is the right primary metric here

  Incorrect

  The 10% split applies above 20,000 rows, and accuracy doesn't optimize well with large class skew.

• BAutoML runs 10-fold cross-validation, and norm_macro_recall is the most reliable primary metric here

  Incorrect

  Ten folds apply below 1,000 rows, and norm_macro_recall is one of the threshold-dependent metrics.

• CAutoML refuses to start the job until validation data is supplied, and precision_score_weighted is required

  Incorrect

  AutoML chooses a validation technique itself when none is given.

• DAutoML runs 3-fold cross-validation, and AUC_weighted is a better primary metric than accuracy here

  Correct

  Datasets of 1,000-20,000 rows get three folds, and threshold metrics like accuracy suffer under class skew.

Without validation_data or n_cross_validations, AutoML picks validation by row count: a 10% holdout above 20,000 rows, three-fold cross-validation from 1,000 to 20,000 rows, and ten folds below 1,000. Threshold-dependent metrics such as accuracy, recall_score_weighted, norm_macro_recall and precision_score_weighted might not optimize well with large class skew, where AUC_weighted can be a better choice.

• Ainstance_type Standard_NC24s_v3, instance_count 2, distribution type pytorch with process_count_per_instance 4

  Incorrect

  The same GPUs, but without the r there's no InfiniBand for fast cross-node communication.

• Binstance_type Standard_NC24rs_v3, instance_count 2, distribution type pytorch with process_count_per_instance 4

  Correct

  The r SKU has InfiniBand, two instances give two nodes, and four processes use every GPU on a node.

• Cinstance_type Standard_NC24rs_v3, instance_count 2, distribution type pytorch without process_count_per_instance

  Incorrect

  Without it Azure Machine Learning starts one process per node, leaving three GPUs idle on each.

• Dinstance_type Standard_NC24rs_v3, instance_count 8, distribution type pytorch with process_count_per_instance 1

  Incorrect

  Eight instances would be eight four-GPU nodes, not two nodes with one process per GPU.

For distributed PyTorch, set the command's distribution type to PyTorch and process_count_per_instance to the number of processes per node, typically the number of GPUs; if you omit it, one process runs per node. instance_count is the number of nodes. Low-latency GPU-to-GPU communication across nodes needs InfiniBand, which typically only VM sizes with r in their names have: Standard_NC24rs_v3 does, Standard_NC24s_v3 doesn't.

• AA schedule without a start_time waits for a manual first run before it triggers any job

  Incorrect

  If start_time is omitted, it defaults to the schedule's creation time.

• BA recurrence set to the 31st runs on the last day of shorter months instead

  Incorrect

  The schedule doesn't trigger in months that have fewer days.

• CA schedule can't be triggered by an event such as new data arriving; it uses a recurrence pattern or a cron expression

  Correct

  v2 schedules don't support event-based triggers.

• DThe schedule's author must keep job run permissions even after a managed identity is assigned to the schedule

  Correct

  The documentation lists this as a requirement for the schedule to keep working.

• EStudio can edit any schedule, including complex recurrence patterns created from the CLI

  Incorrect

  Studio displays complex patterns but doesn't support editing them.

Azure Machine Learning v2 schedules trigger pipeline jobs on a recurrence pattern or a cron expression, not on events, and a schedule without a start_time starts from its creation time. A recurrence on the 30th or 31st doesn't trigger in months that have fewer days, studio can't edit complex patterns created with the CLI or SDK, and the author must retain job run permissions even when a managed identity is assigned to the schedule.

• AEdit it and run it on the owner's compute instance

  Incorrect

  Readers can't edit, and compute instances aren't shared.

• BOnly view the notebook

  Correct

  Readers can view notebooks; editing needs a role such as Contributor.

• CEdit it; notebooks don't follow workspace roles

  Incorrect

  Access to notebooks follows the workspace role.

• DNothing; notebooks can't be shared

  Incorrect

  Notebooks in workspace storage can be shared.

Notebooks are stored in the workspace's storage account and can be shared with others, depending on their access level to the workspace: a Contributor can edit the notebook, while a Reader can only view it. Other users run it on their own compute instances.

• A--learning-rate ${{inputs.learning_rate}}

  Correct

  Inputs are referenced with the ${{inputs.<input_name>}} expression.

• B--learning-rate $learning_rate

  Incorrect

  Shell variables aren't how job inputs are referenced.

• C--learning-rate {{learning_rate}} in Jinja style

  Incorrect

  The expression needs the $ and the inputs prefix.

• D--learning-rate %inputs.learning_rate%

  Incorrect

  That isn't the SDK and CLI v2 expression syntax.

In a command job, inputs is a dictionary of name-value pairs, and the command references each input with the ${{inputs.<input_name>}} expression, for example --learning-rate ${{inputs.learning_rate}}.

• ARandom sampling with a larger max_total_trials and no seed

  Incorrect

  More trials without a seed don't repeat the same trials on rerun.

• BBayesian sampling with a seed

  Incorrect

  Bayesian sampling isn't the space-filling, reproducible rule; Sobol is.

• CRandomSamplingAlgorithm with rule="sobol" and a seed

  Correct

  Sobol is a quasi-random sequence that improves space filling and reproducibility.

• DRandom sampling with logbase set to 10

  Incorrect

  logbase changes scaling, not coverage or reproducibility.

Sobol is a quasi-random sequence that improves space filling and reproducibility. To use it, provide a seed and set rule="sobol" on RandomSamplingAlgorithm.

• AThe cell fails until the user starts the instance

  Incorrect

  The instance is started for the user.

• BThe cell runs on serverless compute instead

  Incorrect

  Notebook cells run on a compute instance.

• CThe notebook switches to another user's instance

  Incorrect

  Compute instances aren't shared.

• DThe compute instance starts automatically

  Correct

  Selecting a stopped instance and running a cell starts it.

You can edit a notebook without connecting to a compute instance. To run cells you need a compute instance; if you select a stopped one, it starts automatically when you run the first cell.