Professional Machine Learning Engineer · Domain 6: Monitoring AI solutions · 13% of the exam
Task 6.1: Identifying risks to AI solutions.
Protecting AI systems from data exfiltration, malicious prompting and sensitive data sent to LLMs with regex checks, safety filters and Model Armor, monitoring for bias, and explaining model predictions.
Study it
Securing AI systems: exfiltration, malicious prompts, sensitive data, safety filters and Model Armor
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 1
A chat product calls two models: Gemini on Agent Platform, and an open model that the team serves itself on GKE. Security requires the same prompt injection, jailbreak and sensitive data screening on prompts and responses for both models, managed as one policy. What should the ML engineer use?
AGemini's configurable safety filters at their strictest thresholds on both models
BA Model Armor template, with the app calling its sanitize methods for prompts and responses to either model
CModel Armor floor settings enabled for the Agent Platform integration in the project
DA system instruction on each model telling it to refuse injected prompts and keep personal data out of responses
Show the answer and why
AGemini's configurable safety filters at their strictest thresholds on both models
Incorrect
Safety filters are part of the Gemini API; they do not apply to the model served on GKE, and they do not detect injection.
BA Model Armor template, with the app calling its sanitize methods for prompts and responses to either model
Correct
Model Armor screens prompts and responses for LLMs in Google Cloud or elsewhere, so one template covers both models.
CModel Armor floor settings enabled for the Agent Platform integration in the project
Incorrect
The Agent Platform integration screens Gemini calls; requests to the GKE-hosted model never pass through it.
DA system instruction on each model telling it to refuse injected prompts and keep personal data out of responses
Incorrect
Instructions can be overridden by an injection, and nothing checks the responses that come back.
Model Armor screens LLM prompts and responses whether the model runs on Google Cloud or elsewhere, using templates that combine prompt injection and jailbreak detection, responsible AI filters, malicious URL detection and Sensitive Data Protection. Applications call the sanitize methods for user prompts and model responses; the Agent Platform integration covers only Gemini calls.
Serverless training jobs in a shared project run with the default identity. An audit finds that a job's code could read every Cloud Storage bucket in the project, including HR exports that no model should touch. Each team's jobs must reach only their own dataset and output buckets, and the projects cannot be split this quarter. What should the ML engineer do?
APut the shared project inside a VPC Service Controls perimeter that protects Cloud Storage and Agent Platform
BEncrypt the HR buckets with a customer-managed key in Cloud KMS
CRun each team's jobs as its own service account with roles only on that team's buckets
DTurn on Data Access audit logs for Cloud Storage in the project
Show the answer and why
APut the shared project inside a VPC Service Controls perimeter that protects Cloud Storage and Agent Platform
Incorrect
A perimeter stops data from leaving the project; the HR buckets are inside the same project and stay reachable.
BEncrypt the HR buckets with a customer-managed key in Cloud KMS
Incorrect
CMEK changes who controls the key; any identity allowed to use the key can still read the data.
CRun each team's jobs as its own service account with roles only on that team's buckets
Correct
A custom service account limits the job to the permissions granted to it, instead of the service agent's project-wide bucket access.
DTurn on Data Access audit logs for Cloud Storage in the project
Incorrect
Audit logs record who read which bucket; they do not stop a job from reading the HR exports.
By default, a serverless training job can access any Cloud Storage bucket in the same project through the Custom Code Service Agent. To control access, run the job with a custom service account that has only the roles it needs, following least privilege.
A lender is launching a PyTorch credit model on an Agent Platform endpoint in 2027. Regulators require per-prediction feature attributions returned with each score for the life of the model. The team plans to configure Vertex Explainable AI on the deployed model. What should the ML engineer recommend?
AConfigure Vertex Explainable AI with sampled Shapley attributions on the deployed model
BCompute SHAP values in the serving code and return them with each prediction
CEnable request-response logging and explain the logged predictions once a month
DPublish global feature importance from training as the explanation for every applicant
Show the answer and why
AConfigure Vertex Explainable AI with sampled Shapley attributions on the deployed model
Incorrect
The capability is deprecated and becomes unavailable on March 16, 2027, before the model's life ends.
BCompute SHAP values in the serving code and return them with each prediction
Correct
Vertex Explainable AI is deprecated and its APIs end in March 2027; open-source SHAP keeps attributions under the team's control.
CEnable request-response logging and explain the logged predictions once a month
Incorrect
Monthly explanations of logs are not attributions returned with each score.
DPublish global feature importance from training as the explanation for every applicant
Incorrect
Global importance describes the model overall, not why one applicant got one score.
Vertex Explainable AI was deprecated on March 16, 2026; no new features are added, and on March 16, 2027 its APIs will no longer be available. Google suggests open-source libraries such as SHAP and LIME instead, computed by the team's own serving code and returned with each prediction.
A sentiment model is trained on reviews from a website where people mostly post when they loved or hated a product. In production it misreads the many mildly worded reviews from a survey tool. Which type of bias best explains the problem?
AAutomation bias, from trusting the model's output over human judgment
BHistorical bias, from past inequities captured in the data
CGroup attribution bias, from generalizing one person's traits to a group
DReporting bias, from a gap between recorded and real-world frequencies
Show the answer and why
AAutomation bias, from trusting the model's output over human judgment
Incorrect
Automation bias is about people favoring automated results, not about how the training data was produced.
BHistorical bias, from past inequities captured in the data
Incorrect
Historical bias reflects inequities of the time the data was recorded; mild versus extreme opinions are not that.
CGroup attribution bias, from generalizing one person's traits to a group
Incorrect
This bias concerns generalizing about groups of people, not the mix of opinions people chose to post.
DReporting bias, from a gap between recorded and real-world frequencies
Correct
The training data over-represents memorable, extreme opinions compared with how often they occur in the real world.
Reporting bias occurs when the frequency of events, properties or outcomes captured in a dataset does not reflect their real-world frequency, often because people record what is unusual or memorable. Reviews written mostly by people with strong opinions are a classic example.
A medical-education app on Gemini finds that some clinical descriptions are blocked because they score high on severity even though the probability of harm is low. The product owner wants blocking decided by the probability score alone, with the thresholds unchanged. What should the ML engineer change?
AThe harm block method, from the default SEVERITY to PROBABILITY
BEvery threshold set to BLOCK_NONE, whatever the probability or severity score
CThe temperature of each request, lowered to 0
DA system instruction that asks the model to avoid graphic detail
Show the answer and why
AThe harm block method, from the default SEVERITY to PROBABILITY
Correct
The PROBABILITY method uses only the probability score, while the default SEVERITY method uses both scores.
BEvery threshold set to BLOCK_NONE, whatever the probability or severity score
Incorrect
This removes blocking entirely instead of basing it on probability, and the thresholds were to stay as they are.
CThe temperature of each request, lowered to 0
Incorrect
Temperature changes how varied the output is; it is not part of content filtering.
DA system instruction that asks the model to avoid graphic detail
Incorrect
Instructions change what the model writes; they do not change how the filter scores and blocks content.
Gemini's content filters assign a probability score and a severity score for each harm category. The Gemini API offers two harm block methods: SEVERITY, the default, uses both scores, and PROBABILITY uses the probability score only. Thresholds such as BLOCK_MEDIUM_AND_ABOVE then apply to the chosen scores.
A security reviewer asks what happens to Gemini prompts on Agent Platform if Google's automated safety classifiers flag the project's traffic as possibly abusive. The project uses CMEK for its own data. What is accurate for a customer under the standard Google Cloud Platform Terms of Service?
AFlagged prompts are used to retrain Google's safety models
BFlagged prompts are deleted immediately and never seen by anyone
CFlagged prompts may be logged for up to 90 days for review, and those logs are not CMEK-encrypted
DFlagged prompts are encrypted with the project's CMEK key like the rest of its data and kept for one year
Show the answer and why
AFlagged prompts are used to retrain Google's safety models
Incorrect
Prompts logged for abuse monitoring are not used to train or fine-tune any models.
BFlagged prompts are deleted immediately and never seen by anyone
Incorrect
Authorized Google employees may review flagged prompts as part of the investigation.
CFlagged prompts may be logged for up to 90 days for review, and those logs are not CMEK-encrypted
Correct
Prompts can be logged solely to check for policy violations, kept up to 90 days in the chosen region, without CMEK encryption.
DFlagged prompts are encrypted with the project's CMEK key like the rest of its data and kept for one year
Incorrect
Abuse monitoring prompt logs are not encrypted with customer-managed keys.
Google uses automated safety classifiers to detect potential abuse. If further investigation is needed, Google may log customer prompts solely to examine possible violations; the data is not used to train models, is stored for up to 90 days in the selected region, is not encrypted with CMEK, and may be reviewed by authorized employees. Customers with a Google Cloud Master Agreement are exempt from this prompt logging by default. These terms cover Google models not designated as Google Frontier Models or Advanced AI; the page lists separate, 30-day terms for those, also without CMEK.
A model that predicts which customers will enjoy a new product line was trained only on survey answers from customers who had already bought similar products. It performs poorly on the general customer base. Which type of bias does this describe?
AAutomation bias, from favoring automated results over people
BExperimenter's bias, from training until the model matches the builder's hypothesis
CImplicit bias, from the assumptions of the people labeling data
DSelection bias, because the examples do not represent the population served
Show the answer and why
AAutomation bias, from favoring automated results over people
Incorrect
Automation bias concerns how people trust automated systems, not how the data was collected.
BExperimenter's bias, from training until the model matches the builder's hypothesis
Incorrect
Experimenter's bias comes from how a model builder keeps training toward an expected result, not from which customers the survey reached.
CImplicit bias, from the assumptions of the people labeling data
Incorrect
Implicit bias comes from people's own mental models, not from whom the survey reached.
DSelection bias, because the examples do not represent the population served
Correct
The data came from a subset of customers that does not reflect everyone the model will make predictions for.
Selection bias occurs when a dataset's examples are chosen in a way that does not reflect their real-world distribution. Its forms include coverage bias (data not selected representatively), non-response bias and sampling bias. Training only on past buyers of similar products is a coverage problem.
Planners ask why a BigQuery ML ARIMA_PLUS model forecasts a sharp rise in sales for next month. The analyst wants to show how much of each forecast value comes from trend, seasonality and holiday effects. Which function should the analyst use?
AML.EVALUATE on the ARIMA_PLUS model and last month's data
BML.EXPLAIN_FORECAST on the ARIMA_PLUS model
CML.EXPLAIN_PREDICT on the forecast table
DML.FORECAST again with a longer horizon and a confidence level of 0.99
Show the answer and why
AML.EVALUATE on the ARIMA_PLUS model and last month's data
Incorrect
ML.EVALUATE reports accuracy metrics, not the components behind each forecast value.
BML.EXPLAIN_FORECAST on the ARIMA_PLUS model
Correct
ML.EXPLAIN_FORECAST returns the forecast with its trend, seasonal and holiday components for each time point.
CML.EXPLAIN_PREDICT on the forecast table
Incorrect
ML.EXPLAIN_PREDICT gives feature attributions for supervised models, not time series decomposition.
DML.FORECAST again with a longer horizon and a confidence level of 0.99
Incorrect
More forecast values and wider intervals still do not split each value into its components.
For ARIMA_PLUS and ARIMA_PLUS_XREG models, BigQuery Explainable AI uses time series decomposition: ML.EXPLAIN_FORECAST returns the forecast along with components such as trend, seasonal periods and holiday effects, so the change in a forecast can be attributed to them.