Professional Machine Learning Engineer · Domain 3: Scaling prototypes into ML models · 20% of the exam
Task 3.1: Building models given the task considering cost, complexity, latency, and scalability.
Choosing the model type (ARIMA, DNN, LLM), the product (AutoML, BigQuery ML, Agent Platform Pipelines), the deployment strategy and an interpretable modeling technique when explanations are required.
Study it
Choosing the model type and the product: ARIMA, DNN, LLM; AutoML, BigQuery ML or a pipeline
Interpretable models when explanations are a requirement
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 1
A lender's credit-risk team must show a regulator the statistical significance of each input's effect on the approval model, and must also give every rejected applicant the factors behind that decision. The data is a structured BigQuery table, and a small loss of accuracy against more complex models is acceptable. Which modeling approach should the ML engineer choose?
ABigQuery ML logistic regression, with ML.ADVANCED_WEIGHTS for p-values and ML.EXPLAIN_PREDICT per applicant
BA BigQuery ML boosted tree model, with ML.EXPLAIN_PREDICT for each applicant and ML.GLOBAL_EXPLAIN for the regulator
CA BigQuery ML DNN classifier, sharing its ML.GLOBAL_EXPLAIN results with the regulator
DAn Agent Platform AutoML tabular model, with its sampled Shapley feature importance shown to the regulator
Show the answer and why
ABigQuery ML logistic regression, with ML.ADVANCED_WEIGHTS for p-values and ML.EXPLAIN_PREDICT per applicant
Correct
Linear models in BigQuery ML provide standard errors and p-values for their weights and local Shapley attributions for each prediction.
BA BigQuery ML boosted tree model, with ML.EXPLAIN_PREDICT for each applicant and ML.GLOBAL_EXPLAIN for the regulator
Incorrect
Tree SHAP gives per-row attributions, but tree models have no standard errors or p-values for significance testing.
CA BigQuery ML DNN classifier, sharing its ML.GLOBAL_EXPLAIN results with the regulator
Incorrect
Global feature importance ranks the inputs; it is not a significance test, and the regulator asked for one.
DAn Agent Platform AutoML tabular model, with its sampled Shapley feature importance shown to the regulator
Incorrect
AutoML offers sampled Shapley importance, not standard errors or p-values for each input's effect.
BigQuery Explainable AI uses a different method for each model type. Linear and logistic regression support Shapley values through ML.EXPLAIN_PREDICT and ML.GLOBAL_EXPLAIN, plus standard errors and p-values for significance testing through ML.ADVANCED_WEIGHTS. Boosted trees and random forests use Tree SHAP and Gini importance, and AutoML Tables uses sampled Shapley.
A manufacturer has 25,000 photos of circuit boards labeled with its own seven defect classes. It has no data scientists, wants a model it can call from an endpoint within a few weeks, and does not want to write training code. Which product should the ML engineer choose?
AThe Cloud Vision API's label detection feature
BA custom PyTorch training job on the labeled photos in serverless training
CA BigQuery ML logistic regression model over an object table
DAgent Platform AutoML image classification
Show the answer and why
AThe Cloud Vision API's label detection feature
Incorrect
Pretrained label detection returns general labels, not the manufacturer's own seven defect classes.
BA custom PyTorch training job on the labeled photos in serverless training
Incorrect
Custom training needs training code and ML skills, which the team does not have.
CA BigQuery ML logistic regression model over an object table
Incorrect
Logistic regression in BigQuery ML trains on tabular features, not on the raw image pixels of the photos.
DAgent Platform AutoML image classification
Correct
AutoML trains an image classification model from labeled images without training code and can deploy it to an endpoint.
AutoML on Agent Platform trains image classification and object detection models from labeled images with no code. Pretrained APIs such as Vision label detection only return general labels, and custom training is for teams that write their own training code.
Inspection cameras in a mine must classify rock samples in under 50 ms. The site has no reliable network connection, so images cannot be sent to the cloud. The team already trains AutoML image models on Agent Platform and wants to keep that workflow. Which deployment strategy should the ML engineer choose?
ADeploy the AutoML model to an Agent Platform endpoint and call it from the cameras over the site's link
BRun batch inference on the images each night after they are copied to Cloud Storage
CTrain an AutoML Edge model and export it as a TF Lite package for the camera devices
DDeploy the model to a private endpoint that uses Private Service Connect
Show the answer and why
ADeploy the AutoML model to an Agent Platform endpoint and call it from the cameras over the site's link
Incorrect
An endpoint needs a network round trip for every image, which the site cannot provide.
BRun batch inference on the images each night after they are copied to Cloud Storage
Incorrect
Nightly batch results arrive hours later, far outside the 50 ms requirement.
CTrain an AutoML Edge model and export it as a TF Lite package for the camera devices
Correct
AutoML Edge models export as TF Lite, Edge TPU TF Lite or a container, so inference runs on the device with no network.
DDeploy the model to a private endpoint that uses Private Service Connect
Incorrect
A private endpoint still requires connectivity to Google Cloud, which the site does not have.
AutoML Edge image models can be exported as a TF Lite package for edge or mobile devices, as Edge TPU TF Lite, or as a container. That lets inference run where the data is produced when there is no connectivity or the latency budget rules out a network call.
A marketplace with 30 million products wants to retrieve a few hundred personalized candidates per page view within 50 ms. About 20,000 new products are listed every day and have no purchase history yet but do have titles, categories and images. Which modeling approach should the ML engineer choose for candidate generation?
AA BigQuery ML matrix factorization model trained on the purchase history of all users and refreshed with every night's new orders
BA Gemini model that scores every product against the user's profile on each page view
CA k-means model that groups products by category and recommends the user's favorite cluster
DA two-tower model whose item tower uses content features, with item embeddings served from an approximate nearest neighbor index
Show the answer and why
AA BigQuery ML matrix factorization model trained on the purchase history of all users and refreshed with every night's new orders
Incorrect
Matrix factorization learns embeddings only for items with interactions, so new products cannot be recommended.
BA Gemini model that scores every product against the user's profile on each page view
Incorrect
Scoring 30 million products per request with an LLM cannot fit a 50 ms budget or a sensible cost.
CA k-means model that groups products by category and recommends the user's favorite cluster
Incorrect
Clusters are coarse, so recommendations are barely personalized and ignore similarity within a cluster.
DA two-tower model whose item tower uses content features, with item embeddings served from an approximate nearest neighbor index
Correct
The item tower can embed items never seen in training, and ANN search over precomputed embeddings meets the latency budget.
Two-tower retrieval learns a query tower and a candidate tower. Because the candidate tower takes item features, it can compute embeddings for items not seen during training, which addresses cold start. Candidate embeddings are precomputed and served from a Vector Search index configured for approximate nearest neighbor search.
A utility forecasts daily electricity demand per region in BigQuery. The forecast must account for each day's temperature forecast and for planned price promotions, both stored as columns next to the demand history. The team works in SQL. Which BigQuery ML model type should it use?
AK-means clustering on the demand, temperature and promotions columns of each region
BA matrix factorization model over regions and days
CARIMA_PLUS_XREG, with temperature and promotions as regressors
DA PCA model that reduces the columns before forecasting
Show the answer and why
AK-means clustering on the demand, temperature and promotions columns of each region
Incorrect
K-means groups similar days; it does not forecast demand over a horizon.
BA matrix factorization model over regions and days
Incorrect
Matrix factorization is for recommendations from user and item interactions, not for forecasting.
CARIMA_PLUS_XREG, with temperature and promotions as regressors
Correct
ARIMA_PLUS_XREG extends the ARIMA_PLUS pipeline with external regressors, so extra variables shape the forecast.
DA PCA model that reduces the columns before forecasting
Incorrect
PCA reduces dimensions; it does not produce a forecast.
BigQuery ML forecasts with ARIMA_PLUS for a single variable, ARIMA_PLUS_XREG when you need to forecast with additional variables (external regressors), or the pretrained TimesFM model. ML.FORECAST returns the forecast for the horizon.
A team likes AutoML's accuracy on its tabular classification data but needs more control: it wants to limit the architecture search space, choose the training hardware and shrink the final ensemble to cut serving latency. Which option fits, using only generally available features?
AThe Tabular Workflow for End-to-End AutoML
BStandard AutoML tabular training with a larger training budget
CThe Feature Transform Engine run on its own
DA BigQuery ML boosted tree model with default options
Show the answer and why
AThe Tabular Workflow for End-to-End AutoML
Correct
This generally available workflow exposes each step of AutoML, including architecture search, hardware and ensemble size.
BStandard AutoML tabular training with a larger training budget
Incorrect
More budget gives AutoML more time but no control over search space, hardware or ensemble size.
CThe Feature Transform Engine run on its own
Incorrect
Feature Transform Engine handles feature selection and transforms, and it is in Preview.
DA BigQuery ML boosted tree model with default options
Incorrect
A boosted tree model is a different model type, not AutoML with more control over its steps.
Tabular Workflow for End-to-End AutoML is a managed Agent Platform Pipelines workflow that lets you choose what to control and what to automate at each step: data splitting, feature engineering, architecture search, model training and ensembling, including hardware and ensemble size. It is generally available, while Feature Transform Engine and the forecasting workflow are in Preview.
A streaming service shows each of its 5 million users a "recommended tonight" row that is the same all evening. Predictions must appear instantly when the home page loads, and the team wants to check predictions before users see them. Which deployment strategy fits best?
ADynamic inference on every page load from an online endpoint that autoscales
BCalling a large language model for each user at page load
CRunning the model on each user's device after the page loads
DStatic inference that precomputes and caches each user's row in advance
Show the answer and why
ADynamic inference on every page load from an online endpoint that autoscales
Incorrect
On-demand prediction adds latency and cost for results that do not change during the evening, and they cannot be checked in advance.
BCalling a large language model for each user at page load
Incorrect
An LLM call per page load is slow and costly for a fixed nightly list.
CRunning the model on each user's device after the page loads
Incorrect
On-device inference still computes at page load and cannot be reviewed beforehand.
DStatic inference that precomputes and caches each user's row in advance
Correct
Precomputed predictions are served instantly and can be verified before anyone sees them.
Static inference generates predictions in advance and caches them, which suits a known set of inputs when prediction speed, cost and the ability to verify predictions matter. Dynamic inference predicts on demand and suits diverse inputs that cannot be enumerated in advance.
A data science team trained a PyTorch model outside Google Cloud. The analytics team needs to score 300 million rows in BigQuery with it every week, in SQL, without moving the data or running an endpoint. Which option should the ML engineer choose?
ARetrain the model as a BigQuery ML DNN classifier with default options on the same data
BConvert the model to ONNX and import it with CREATE MODEL in BigQuery ML
CDeploy the model to an endpoint and call it from a remote function
DExport the table to Cloud Storage and score it in a notebook
Show the answer and why
ARetrain the model as a BigQuery ML DNN classifier with default options on the same data
Incorrect
Retraining discards the team's validated PyTorch model and its architecture.
BConvert the model to ONNX and import it with CREATE MODEL in BigQuery ML
Correct
BigQuery ML imports ONNX models, such as those from PyTorch, and runs prediction in BigQuery with ML.PREDICT.
CDeploy the model to an endpoint and call it from a remote function
Incorrect
This adds an endpoint and per-row calls, which the team wants to avoid.
DExport the table to Cloud Storage and score it in a notebook
Incorrect
Exporting and scoring outside BigQuery moves the data and leaves SQL.
BigQuery ML can import models trained outside BigQuery, including ONNX (from frameworks such as PyTorch and scikit-learn), TensorFlow, TensorFlow Lite and XGBoost, and run prediction in BigQuery. The CREATE MODEL statement for an imported model processes no bytes.