AI-300 · Domain 2: Implement machine learning model lifecycle and operations · 30% of the exam
Task 2.2: Implement model registration and versioning
Packaging a feature retrieval specification with the model, registering MLflow models, evaluating models with responsible AI principles, and managing the model lifecycle including archiving.
Study it
Feature retrieval specifications and MLflow model registration
Lesson coming
Responsible AI dashboard and model archiving
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Microsoft documentation page that proves it.
Question 1 · choose 1
A model trained on features from a managed feature store is registered, but its details page shows no lineage to the feature sets, and the online scoring script can't find which features to look up. The training job saved the spec as features/spec.yaml next to model.pkl. What should the training job do?
ACopy the spec to the root of the model output folder as feature_retrieval_spec.yaml before registering
BRegister the spec as its own uri_file data asset and link it to the model with a tag
CKeep features/spec.yaml in its subfolder and pass its path to the scoring script as an environment variable
DRecreate the feature sets with the model's name so that the store records the lineage
Show the answer and why
ACopy the spec to the root of the model output folder as feature_retrieval_spec.yaml before registering
Correct
The spec must sit at the model artifact's root under its fixed name, for lineage and for inference.
BRegister the spec as its own uri_file data asset and link it to the model with a tag
Incorrect
Lineage is tracked only when the spec is inside the model artifact.
CKeep features/spec.yaml in its subfolder and pass its path to the scoring script as an environment variable
Incorrect
The file name can't be changed and it must be in the artifact's root folder.
DRecreate the feature sets with the model's name so that the store records the lineage
Incorrect
Lineage comes from the spec packaged with the model, not from feature set names.
When a model is trained on data with features from feature stores, the feature retrieval specification must be packaged with the model artifact, in its root folder, and its file name can't be changed. Lineage between the model and the feature sets is tracked only if the spec is in the model artifact, and the scoring code loads the feature list from it before looking up the online store.
A team marks the approved version of a model as Production with the MLflow client and wants its release pipeline to deploy whatever is in Production. What should it know?
AResolve the stage to its version number with the MLflow client, then deploy that version
BStages set with the client appear in studio and the CLI, and the pipeline can deploy models:/name/Production
CStages can be read with az ml model show for the pipeline to look up and deploy directly
DShare the Production version to a registry, where MLflow stages become deployable references
Show the answer and why
AResolve the stage to its version number with the MLflow client, then deploy that version
Correct
Azure Machine Learning can't deploy from a model stage.
BStages set with the client appear in studio and the CLI, and the pipeline can deploy models:/name/Production
Incorrect
Stages aren't visible in studio, the CLI, the SDK or the REST API, and stage deployment isn't supported.
CStages can be read with az ml model show for the pipeline to look up and deploy directly
Incorrect
Stages can't be retrieved with the Azure Machine Learning CLI, SDK or REST API.
DShare the Production version to a registry, where MLflow stages become deployable references
Incorrect
A registry doesn't make stages deployable; deployment from a stage isn't supported.
MLflow model stages are accessible only through the MLflow SDK; they aren't visible in Azure Machine Learning studio and can't be retrieved with the Azure Machine Learning SDK, CLI or REST API. Deployment from a specific model stage isn't currently supported, so the pipeline should resolve the stage to a specific version with the MLflow client and deploy that version.
A team wants a Responsible AI dashboard for an XGBoost classifier logged with mlflow.xgboost. Its training and test data are registered as uri_folder assets of Parquet files. The dashboard pipeline fails. What must change?
ARegister the model as custom_model and keep the Parquet uri_folder datasets
BKeep the mlflow.xgboost flavor but convert the datasets to uri_file CSVs
CRun the dashboard pipeline on a larger compute cluster so that the components can load the XGBoost model
DLog the model in MLflow format with a scikit-learn flavor and register the datasets as mltable
Show the answer and why
ARegister the model as custom_model and keep the Parquet uri_folder datasets
Incorrect
Only MLflow models with a sklearn flavor are supported, and the inputs must be mltable.
BKeep the mlflow.xgboost flavor but convert the datasets to uri_file CSVs
Incorrect
The flavor is still unsupported, and uri_file isn't mltable.
CRun the dashboard pipeline on a larger compute cluster so that the components can load the XGBoost model
Incorrect
Compute size isn't the problem; the components accept only sklearn-flavor MLflow models and mltable data.
DLog the model in MLflow format with a scikit-learn flavor and register the datasets as mltable
Correct
The RAI components need sklearn-flavor MLflow models and mltable inputs.
The Responsible AI components currently require models registered in MLflow format with a sklearn (scikit-learn) flavor, and dataset inputs in mltable format.
Model credit-risk has versions 1-5. Version 3 had a data issue and should disappear from listings, but an existing deployment still uses it, and the other versions must stay visible. What should you do?
AArchive the model without a version so all versions are hidden
BSet a tag hidden=true on version 3 so that model listings skip it
CArchive version 3 of the model only
DArchive versions 1 to 5, then register the good ones again as new versions
Show the answer and why
AArchive the model without a version so all versions are hidden
Incorrect
Without a version, every version of the model is archived.
BSet a tag hidden=true on version 3 so that model listings skip it
Incorrect
Tags are metadata; they don't change what list queries return.
CArchive version 3 of the model only
Correct
Archiving one version hides it from list queries while workflows can still use it.
DArchive versions 1 to 5, then register the good ones again as new versions
Incorrect
That hides the versions that must stay visible and gives them new version numbers.
Archiving a model hides it from list queries such as az ml model list by default, while you can continue to reference and use it. You can archive all versions or a specific version; without a version, all versions are archived.
A team manages its workspace models with the MLflow client and wants to manage models in its organization's Azure Machine Learning registry the same way. What applies?
AThe MLflow client works the same once pointed at the registry's tracking URI
BMLflow model management doesn't support registries; use the ml CLI or SDK
CThe MLflow client manages registry models fully, except their stages and tags
DThe registry has to be attached to the workspace as a datastore first
Show the answer and why
AThe MLflow client works the same once pointed at the registry's tracking URI
Incorrect
Registries aren't supported for model management with MLflow.
BMLflow model management doesn't support registries; use the ml CLI or SDK
Correct
The MLflow limitations list organizational registries as unsupported for model management.
CThe MLflow client manages registry models fully, except their stages and tags
Incorrect
Model management with MLflow doesn't work against registries at all.
DThe registry has to be attached to the workspace as a datastore first
Incorrect
Registries aren't attached as datastores; they hold assets for many workspaces.
With MLflow connected to Azure Machine Learning, you can manage the models registry of a workspace, but organizational registries aren't supported for model management with MLflow. To create and use models in a registry, use the Azure Machine Learning CLI or Python SDK, for example az ml model create with --registry-name.
In an MLflow model's MLmodel file, what does a flavor describe?
AThe compute target the model was trained on, recorded by MLflow
BThe deployment's traffic allocation
CThe model's registered version number
DHow models from a given framework are saved and loaded
Show the answer and why
AThe compute target the model was trained on, recorded by MLflow
Incorrect
Compute isn't recorded as a flavor.
BThe deployment's traffic allocation
Incorrect
Traffic belongs to endpoints.
CThe model's registered version number
Incorrect
Versions are set by the registry.
DHow models from a given framework are saved and loaded
Correct
Each flavor is a contract for one framework.
MLflow introduced flavors as a way to provide a unique contract for all machine learning frameworks: a flavor indicates what to expect for a model created with a specific framework, including how to persist and load it.
A tabular model's MLflow signature is column-based. What input does MLflow pass to the model at inference?
AA pandas DataFrame
BA numpy ndarray tensor
CA raw JSON string
DA Spark DataFrame
Show the answer and why
AA pandas DataFrame
Correct
Column-based signatures operate on tabular data supplied as DataFrames.
BA numpy ndarray tensor
Incorrect
That's what tensor-based signatures receive.
CA raw JSON string
Incorrect
MLflow parses input according to the signature.
DA Spark DataFrame
Incorrect
Column-based signatures use pandas DataFrames.
MLflow has two types of signatures. Column-based signatures operate on tabular data, and MLflow supplies pandas.DataFrame objects as inputs; tensor-based signatures operate on n-dimensional arrays, supplied as numpy.ndarray or a dictionary of them.