Professional Machine Learning Engineer · Domain 4: Serving and scaling models · 20% of the exam
Task 4.1: Serving models.
Batch and online inference on Agent Platform, Model Garden, Cloud Run and GKE, prebuilt and custom serving containers, versioning in Model Registry, A/B and canary rollouts, and inference preprocessing and postprocessing.
Study it
Batch and online inference on Agent Platform, Cloud Run and GKE
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 2
A model trained and registered in us-central1 must now serve EU customers from europe-west4 for data residency, with exactly the same model artifacts so that behavior matches. Which TWO steps should the ML engineer take? (Choose TWO.)
ACopy the model from Model Registry into europe-west4
BDeploy the us-central1 model directly to a europe-west4 endpoint
CRetrain the model from scratch on the same data in europe-west4
DServe EU traffic from the us-central1 endpoint through a global load balancer
EDeploy the copied model to an endpoint created in europe-west4
Show the answer and why
ACopy the model from Model Registry into europe-west4
Correct
Model Registry can copy a model to another location, so the same artifacts are available in the target region.
BDeploy the us-central1 model directly to a europe-west4 endpoint
Incorrect
The model resource lives in us-central1; it has to be copied before a europe-west4 endpoint can serve it.
CRetrain the model from scratch on the same data in europe-west4
Incorrect
Retraining does not guarantee identical behavior, which is why the docs recommend copying the model.
DServe EU traffic from the us-central1 endpoint through a global load balancer
Incorrect
Inference would still run in us-central1, outside the residency requirement.
EDeploy the copied model to an endpoint created in europe-west4
Correct
Endpoints serve models from their own region, so the copy is deployed to a europe-west4 endpoint.
Training a model again in each region is not a reliable way to keep behavior consistent. Model Registry's copy model feature copies a model to another location in the same project or to another project, where it can be deployed to endpoints and used for batch inference in that region.
A team deploys a model in its own serving container to an Agent Platform endpoint. The container runs a FastAPI server on port 5000 with routes /healthz and /score. The deployment never becomes ready and the logs show failed health checks. What should the team change in the container?
AAdd a HEALTHCHECK to the Dockerfile that calls the /healthz health route every 10 seconds
BReplace the image with a prebuilt inference container
CListen on AIP_HTTP_PORT and answer requests on the AIP_HEALTH_ROUTE and AIP_PREDICT_ROUTE paths
DRaise the deployment's minimum replica count so that more replicas start
Show the answer and why
AAdd a HEALTHCHECK to the Dockerfile that calls the /healthz health route every 10 seconds
Incorrect
Agent Platform sends its own health checks to its configured route; a Dockerfile probe does not change where they go.
BReplace the image with a prebuilt inference container
Incorrect
A prebuilt container drops the team's own server and its logic instead of fixing the port and routes it uses.
CListen on AIP_HTTP_PORT and answer requests on the AIP_HEALTH_ROUTE and AIP_PREDICT_ROUTE paths
Correct
Agent Platform sends liveness, health and inference requests to the port and routes it passes in these environment variables.
DRaise the deployment's minimum replica count so that more replicas start
Incorrect
More replicas start the same container, and each one fails the same health check on the wrong port and route.
A custom inference container must run an HTTP server that listens on the port Agent Platform sets in AIP_HTTP_PORT (8080 by default, or the first containerSpec.ports entry) and must respond to health checks on AIP_HEALTH_ROUTE and to inference requests on AIP_PREDICT_ROUTE.
A recommendation model is served from an Agent Platform endpoint that many mobile app versions call by its URL. A retrained model must first receive 5% of live traffic, then more over two weeks, and the team must be able to send all traffic back to the old model within minutes. The app cannot be changed. What should the ML engineer do?
ADeploy the new model to a new endpoint and switch the DNS name over once the tests pass
BMove the default alias in Model Registry to the new version and let the endpoint pick it up
CDeploy the new model to the same endpoint at a 5% traffic split, then raise its share
DShip an app update that sends 5% of users' traffic to a second endpoint at random
Show the answer and why
ADeploy the new model to a new endpoint and switch the DNS name over once the tests pass
Incorrect
A DNS switch moves all traffic at once, so there is no 5% stage, and the app calls the endpoint URL directly.
BMove the default alias in Model Registry to the new version and let the endpoint pick it up
Incorrect
An alias names a version in the registry; it does not change which deployed model an endpoint sends traffic to.
CDeploy the new model to the same endpoint at a 5% traffic split, then raise its share
Correct
Several models on one endpoint share its URL, and the traffic split can be changed at any time, including back to 100% for the old one.
DShip an app update that sends 5% of users' traffic to a second endpoint at random
Incorrect
Splitting traffic in the client requires an app change, which the stem rules out.
Deploying two models to the same endpoint lets you gradually replace one with the other without changing the URL that clients call: the new model serves a small percentage of traffic, and you raise its traffic split until it serves 100%. Changing the split back is the rollback.
A scikit-learn model must normalize raw inputs, look up a currency rate from an internal service, and map the predicted class index to a label before returning it. The team wants to write only that Python logic, not an HTTP server or a Dockerfile. How should the ML engineer package the model for an Agent Platform endpoint?
AAs a custom inference routine with a Predictor that implements preprocess and postprocess
BIn the prebuilt scikit-learn container with the model artifact only
CIn a custom container with a FastAPI server and Dockerfile written by the team for this model
DWith the preprocessing copied into every client that calls the endpoint
Show the answer and why
AAs a custom inference routine with a Predictor that implements preprocess and postprocess
Correct
Custom inference routines let you supply preprocessing and postprocessing code while the SDK builds the model server and image.
BIn the prebuilt scikit-learn container with the model artifact only
Incorrect
The prebuilt container serves the model as is and runs no custom preprocessing or postprocessing.
CIn a custom container with a FastAPI server and Dockerfile written by the team for this model
Incorrect
This works but requires the HTTP server and Dockerfile the team does not want to write.
DWith the preprocessing copied into every client that calls the endpoint
Incorrect
Client-side copies drift apart, and the label mapping would still be missing from the endpoint.
Custom inference routines let you build custom containers with preprocessing and postprocessing code, such as normalizing inputs or calling external services, without writing a model server or a Dockerfile. You implement the Predictor interface and the SDK builds the container.
A fraud system runs a PyTorch model that embeds transaction text and passes the embedding to an XGBoost classifier. Today the client calls two separate endpoints in turn, which doubles network latency. The team wants both models behind one GPU-backed endpoint, chained on the server, with requests batched on the server side. What should the ML engineer use?
ATwo prebuilt containers on one endpoint with a 50/50 traffic split
BAn NVIDIA Triton serving container with the two models in an ensemble
CBoth models deployed to the same endpoint and called one after the other
DA batch inference job that runs both models over each hour's transactions
Show the answer and why
ATwo prebuilt containers on one endpoint with a 50/50 traffic split
Incorrect
A traffic split sends each request to one model or the other; it does not pass one model's output to the next.
BAn NVIDIA Triton serving container with the two models in an ensemble
Correct
Triton serves models from several frameworks, chains them as an ensemble and batches requests dynamically on the server.
CBoth models deployed to the same endpoint and called one after the other
Incorrect
Sharing an endpoint does not chain the models; the client still makes two calls and two network trips.
DA batch inference job that runs both models over each hour's transactions
Incorrect
Batch jobs run asynchronously over stored inputs; they cannot answer each transaction online.
The NVIDIA Triton inference server on Agent Platform supports models from multiple frameworks (including PyTorch and XGBoost through its backends), concurrent model execution, ensembles that pipe one model's outputs into another, and dynamic batching of inference requests on the server.
An ML engineer wants to serve an XGBoost model with an Agent Platform prebuilt inference container instead of writing a server. Which TWO steps are required? (Choose TWO.)
ABuild a serving image of the team's own and push it to Artifact Registry
BSave the model with the exact file name the container expects, such as model.bst
CWrite a Dockerfile that installs XGBoost on a base container image
DConvert the model to a TensorFlow SavedModel first
EPoint the model's artifact URI at the Cloud Storage directory holding the file
Show the answer and why
ABuild a serving image of the team's own and push it to Artifact Registry
Incorrect
Google already publishes the prebuilt images; only a custom container has to be built and pushed.
BSave the model with the exact file name the container expects, such as model.bst
Correct
Prebuilt containers load artifacts by name, and an XGBoost Booster is exported as model.bst.
CWrite a Dockerfile that installs XGBoost on a base container image
Incorrect
A Dockerfile is the custom-container path the team wanted to avoid.
DConvert the model to a TensorFlow SavedModel first
Incorrect
There are prebuilt containers for XGBoost, so no conversion to TensorFlow is needed.
EPoint the model's artifact URI at the Cloud Storage directory holding the file
Correct
When the model is imported, the artifact URI tells the prebuilt container where to load the artifacts from.
To use a prebuilt container for XGBoost, export the model artifact with a file name the container recognizes (model.bst from Booster.save_model, or model.joblib or model.pkl), upload it to a Cloud Storage directory, and import the model with that directory as the artifact URI and the matching prebuilt container image.
An automated rollout shifts an endpoint's traffic split to 100% for a new model version and then undeploys the old version in the same step. A few in-flight requests fail at that moment. What should the pipeline do instead?
ADelete the endpoint and create a new endpoint for each release of the model
BUndeploy the old version first, then shift the traffic split
CWait 10 to 15 minutes after the split reaches 100% before undeploying
DKeep both versions deployed permanently at 100% and 0%
Show the answer and why
ADelete the endpoint and create a new endpoint for each release of the model
Incorrect
Deleting the endpoint drops all traffic and changes nothing about in-flight requests to the old version.
BUndeploy the old version first, then shift the traffic split
Incorrect
Undeploying before the shift sends requests to a model that is no longer there.
CWait 10 to 15 minutes after the split reaches 100% before undeploying
Correct
The hold window lets in-flight requests finish and routing settle before the old model's resources are released.
DKeep both versions deployed permanently at 100% and 0%
Incorrect
A version left deployed keeps its replicas, and their cost, without serving traffic.
When replacing a model on an endpoint after shifting traffic to a new version, wait at least 10 to 15 minutes after the traffic split reaches 100% before undeploying the previous model, so that in-flight inference requests complete and regional routing fully synchronizes. An endpoint can be deleted only after all its models are undeployed.
A deployed model's latency is too high on its current machine type, and the team wants it on a larger machine with a GPU. It looks for a way to edit the machine type of the deployed model. What should the ML engineer do?
AEdit the deployed model's machine type in place from the console
BRedeploy the model with the new machine type, then move traffic and undeploy the old one
CRaise maxReplicaCount on the existing deployment to let more replicas start
DChange the endpoint's region to one with GPU capacity
Show the answer and why
AEdit the deployed model's machine type in place from the console
Incorrect
Compute settings of a deployed model cannot be changed after deployment.
BRedeploy the model with the new machine type, then move traffic and undeploy the old one
Correct
Deployment settings are fixed once deployed, so the model is deployed again with the new compute and traffic is moved.
CRaise maxReplicaCount on the existing deployment to let more replicas start
Incorrect
More replicas of the same machine type add throughput, not per-request speed or a GPU.
DChange the endpoint's region to one with GPU capacity
Incorrect
An endpoint's region is set at creation and does not change the deployed machine type.
When you deploy a model you choose the endpoint (with its location), the model and its container, and the compute resources of the deployed model. After deployment these settings cannot be changed; to change them, redeploy the model.
A team serves TensorFlow SavedModels from the open-source TensorFlow Serving prebuilt container and wants faster, cheaper inference without rewriting the model or building its own container. What should the ML engineer try?
AExport the model to TensorFlow Lite and serve it from a custom container
BMove the model to a larger machine type
CSwitch to a scikit-learn prebuilt container
DThe optimized TensorFlow runtime container
Show the answer and why
AExport the model to TensorFlow Lite and serve it from a custom container
Incorrect
This means building and maintaining a custom container, which the team wants to avoid.
BMove the model to a larger machine type
Incorrect
A bigger machine may lower latency but raises cost.
CSwitch to a scikit-learn prebuilt container
Incorrect
A scikit-learn container cannot serve a TensorFlow SavedModel.
DThe optimized TensorFlow runtime container
Correct
The optimized runtime serves TensorFlow models faster and at lower cost than the open-source TensorFlow Serving containers.
The optimized TensorFlow runtime is a prebuilt container that optimizes TensorFlow models for faster and lower-cost inference than the open source based prebuilt TensorFlow Serving containers, with minimal effort. The gain varies by model type.
Each month a team retrains its churn model and uploads it to Model Registry as a separate model with a new name, so the registry now holds 14 unrelated models. The team wants one model with a history of its versions. What should it do for future uploads?
AImport each retrained model as a new version of the existing model
BAdd a month label to each separately named model to group them in searches
CCopy each new model into another region
DDelete the old models after each new upload
Show the answer and why
AImport each retrained model as a new version of the existing model
Correct
Model Registry can import a model as a new version of an existing model by naming its parent model.
BAdd a month label to each separately named model to group them in searches
Incorrect
Labels help filter, but the models remain unrelated resources without a shared version history.
CCopy each new model into another region
Incorrect
Copying moves a model between locations; it does not group versions.
DDelete the old models after each new upload
Incorrect
Deleting loses the history the team wants to keep.
In Model Registry you can import a model as a new version of an existing model, by choosing Import as new version in the console or setting the parent model in the upload request. Versions share one model resource, and one of them is the default version.