Skip to content

Professional Machine Learning Engineer · Domain 6: Monitoring AI solutions · 13% of the exam

Task 6.1: Identifying risks to AI solutions.

Protecting AI systems from data exfiltration, malicious prompting and sensitive data sent to LLMs with regex checks, safety filters and Model Armor, monitoring for bias, and explaining model predictions.

Study it

  • Securing AI systems: exfiltration, malicious prompts, sensitive data, safety filters and Model Armor

    Partly covered by: Guardrails, Finding and Masking PII

  • Responsible AI: bias and explainability

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.

Question 1 · choose 1

A chat product calls two models: Gemini on Agent Platform, and an open model that the team serves itself on GKE. Security requires the same prompt injection, jailbreak and sensitive data screening on prompts and responses for both models, managed as one policy. What should the ML engineer use?

  1. AGemini's configurable safety filters at their strictest thresholds on both models
  2. BA Model Armor template, with the app calling its sanitize methods for prompts and responses to either model
  3. CModel Armor floor settings enabled for the Agent Platform integration in the project
  4. DA system instruction on each model telling it to refuse injected prompts and keep personal data out of responses
Show the answer and why
  • AGemini's configurable safety filters at their strictest thresholds on both models

    Incorrect

    Safety filters are part of the Gemini API; they do not apply to the model served on GKE, and they do not detect injection.

  • BA Model Armor template, with the app calling its sanitize methods for prompts and responses to either model

    Correct

    Model Armor screens prompts and responses for LLMs in Google Cloud or elsewhere, so one template covers both models.

  • CModel Armor floor settings enabled for the Agent Platform integration in the project

    Incorrect

    The Agent Platform integration screens Gemini calls; requests to the GKE-hosted model never pass through it.

  • DA system instruction on each model telling it to refuse injected prompts and keep personal data out of responses

    Incorrect

    Instructions can be overridden by an injection, and nothing checks the responses that come back.

Model Armor screens LLM prompts and responses whether the model runs on Google Cloud or elsewhere, using templates that combine prompt injection and jailbreak detection, responsible AI filters, malicious URL detection and Sensitive Data Protection. Applications call the sanitize methods for user prompts and model responses; the Agent Platform integration covers only Gemini calls.

Question 2 · choose 1

Serverless training jobs in a shared project run with the default identity. An audit finds that a job's code could read every Cloud Storage bucket in the project, including HR exports that no model should touch. Each team's jobs must reach only their own dataset and output buckets, and the projects cannot be split this quarter. What should the ML engineer do?

  1. APut the shared project inside a VPC Service Controls perimeter that protects Cloud Storage and Agent Platform
  2. BEncrypt the HR buckets with a customer-managed key in Cloud KMS
  3. CRun each team's jobs as its own service account with roles only on that team's buckets
  4. DTurn on Data Access audit logs for Cloud Storage in the project
Show the answer and why
  • APut the shared project inside a VPC Service Controls perimeter that protects Cloud Storage and Agent Platform

    Incorrect

    A perimeter stops data from leaving the project; the HR buckets are inside the same project and stay reachable.

  • BEncrypt the HR buckets with a customer-managed key in Cloud KMS

    Incorrect

    CMEK changes who controls the key; any identity allowed to use the key can still read the data.

  • CRun each team's jobs as its own service account with roles only on that team's buckets

    Correct

    A custom service account limits the job to the permissions granted to it, instead of the service agent's project-wide bucket access.

  • DTurn on Data Access audit logs for Cloud Storage in the project

    Incorrect

    Audit logs record who read which bucket; they do not stop a job from reading the HR exports.

By default, a serverless training job can access any Cloud Storage bucket in the same project through the Custom Code Service Agent. To control access, run the job with a custom service account that has only the roles it needs, following least privilege.

Question 3 · choose 1

A lender is launching a PyTorch credit model on an Agent Platform endpoint in 2027. Regulators require per-prediction feature attributions returned with each score for the life of the model. The team plans to configure Vertex Explainable AI on the deployed model. What should the ML engineer recommend?

  1. AConfigure Vertex Explainable AI with sampled Shapley attributions on the deployed model
  2. BCompute SHAP values in the serving code and return them with each prediction
  3. CEnable request-response logging and explain the logged predictions once a month
  4. DPublish global feature importance from training as the explanation for every applicant
Show the answer and why
  • AConfigure Vertex Explainable AI with sampled Shapley attributions on the deployed model

    Incorrect

    The capability is deprecated and becomes unavailable on March 16, 2027, before the model's life ends.

  • BCompute SHAP values in the serving code and return them with each prediction

    Correct

    Vertex Explainable AI is deprecated and its APIs end in March 2027; open-source SHAP keeps attributions under the team's control.

  • CEnable request-response logging and explain the logged predictions once a month

    Incorrect

    Monthly explanations of logs are not attributions returned with each score.

  • DPublish global feature importance from training as the explanation for every applicant

    Incorrect

    Global importance describes the model overall, not why one applicant got one score.

Vertex Explainable AI was deprecated on March 16, 2026; no new features are added, and on March 16, 2027 its APIs will no longer be available. Google suggests open-source libraries such as SHAP and LIME instead, computed by the team's own serving code and returned with each prediction.

Question 4 · choose 1

A sentiment model is trained on reviews from a website where people mostly post when they loved or hated a product. In production it misreads the many mildly worded reviews from a survey tool. Which type of bias best explains the problem?

  1. AAutomation bias, from trusting the model's output over human judgment
  2. BHistorical bias, from past inequities captured in the data
  3. CGroup attribution bias, from generalizing one person's traits to a group
  4. DReporting bias, from a gap between recorded and real-world frequencies
Show the answer and why
  • AAutomation bias, from trusting the model's output over human judgment

    Incorrect

    Automation bias is about people favoring automated results, not about how the training data was produced.

  • BHistorical bias, from past inequities captured in the data

    Incorrect

    Historical bias reflects inequities of the time the data was recorded; mild versus extreme opinions are not that.

  • CGroup attribution bias, from generalizing one person's traits to a group

    Incorrect

    This bias concerns generalizing about groups of people, not the mix of opinions people chose to post.

  • DReporting bias, from a gap between recorded and real-world frequencies

    Correct

    The training data over-represents memorable, extreme opinions compared with how often they occur in the real world.

Reporting bias occurs when the frequency of events, properties or outcomes captured in a dataset does not reflect their real-world frequency, often because people record what is unusual or memorable. Reviews written mostly by people with strong opinions are a classic example.

Question 5 · choose 1

A medical-education app on Gemini finds that some clinical descriptions are blocked because they score high on severity even though the probability of harm is low. The product owner wants blocking decided by the probability score alone, with the thresholds unchanged. What should the ML engineer change?

  1. AThe harm block method, from the default SEVERITY to PROBABILITY
  2. BEvery threshold set to BLOCK_NONE, whatever the probability or severity score
  3. CThe temperature of each request, lowered to 0
  4. DA system instruction that asks the model to avoid graphic detail
Show the answer and why
  • AThe harm block method, from the default SEVERITY to PROBABILITY

    Correct

    The PROBABILITY method uses only the probability score, while the default SEVERITY method uses both scores.

  • BEvery threshold set to BLOCK_NONE, whatever the probability or severity score

    Incorrect

    This removes blocking entirely instead of basing it on probability, and the thresholds were to stay as they are.

  • CThe temperature of each request, lowered to 0

    Incorrect

    Temperature changes how varied the output is; it is not part of content filtering.

  • DA system instruction that asks the model to avoid graphic detail

    Incorrect

    Instructions change what the model writes; they do not change how the filter scores and blocks content.

Gemini's content filters assign a probability score and a severity score for each harm category. The Gemini API offers two harm block methods: SEVERITY, the default, uses both scores, and PROBABILITY uses the probability score only. Thresholds such as BLOCK_MEDIUM_AND_ABOVE then apply to the chosen scores.

Question 6 · choose 1

A security reviewer asks what happens to Gemini prompts on Agent Platform if Google's automated safety classifiers flag the project's traffic as possibly abusive. The project uses CMEK for its own data. What is accurate for a customer under the standard Google Cloud Platform Terms of Service?

  1. AFlagged prompts are used to retrain Google's safety models
  2. BFlagged prompts are deleted immediately and never seen by anyone
  3. CFlagged prompts may be logged for up to 90 days for review, and those logs are not CMEK-encrypted
  4. DFlagged prompts are encrypted with the project's CMEK key like the rest of its data and kept for one year
Show the answer and why
  • AFlagged prompts are used to retrain Google's safety models

    Incorrect

    Prompts logged for abuse monitoring are not used to train or fine-tune any models.

  • BFlagged prompts are deleted immediately and never seen by anyone

    Incorrect

    Authorized Google employees may review flagged prompts as part of the investigation.

  • CFlagged prompts may be logged for up to 90 days for review, and those logs are not CMEK-encrypted

    Correct

    Prompts can be logged solely to check for policy violations, kept up to 90 days in the chosen region, without CMEK encryption.

  • DFlagged prompts are encrypted with the project's CMEK key like the rest of its data and kept for one year

    Incorrect

    Abuse monitoring prompt logs are not encrypted with customer-managed keys.

Google uses automated safety classifiers to detect potential abuse. If further investigation is needed, Google may log customer prompts solely to examine possible violations; the data is not used to train models, is stored for up to 90 days in the selected region, is not encrypted with CMEK, and may be reviewed by authorized employees. Customers with a Google Cloud Master Agreement are exempt from this prompt logging by default. These terms cover Google models not designated as Google Frontier Models or Advanced AI; the page lists separate, 30-day terms for those, also without CMEK.

Google Cloud documentation

Question 7 · choose 1

A model that predicts which customers will enjoy a new product line was trained only on survey answers from customers who had already bought similar products. It performs poorly on the general customer base. Which type of bias does this describe?

  1. AAutomation bias, from favoring automated results over people
  2. BExperimenter's bias, from training until the model matches the builder's hypothesis
  3. CImplicit bias, from the assumptions of the people labeling data
  4. DSelection bias, because the examples do not represent the population served
Show the answer and why
  • AAutomation bias, from favoring automated results over people

    Incorrect

    Automation bias concerns how people trust automated systems, not how the data was collected.

  • BExperimenter's bias, from training until the model matches the builder's hypothesis

    Incorrect

    Experimenter's bias comes from how a model builder keeps training toward an expected result, not from which customers the survey reached.

  • CImplicit bias, from the assumptions of the people labeling data

    Incorrect

    Implicit bias comes from people's own mental models, not from whom the survey reached.

  • DSelection bias, because the examples do not represent the population served

    Correct

    The data came from a subset of customers that does not reflect everyone the model will make predictions for.

Selection bias occurs when a dataset's examples are chosen in a way that does not reflect their real-world distribution. Its forms include coverage bias (data not selected representatively), non-response bias and sampling bias. Training only on past buyers of similar products is a coverage problem.

Question 8 · choose 1

Planners ask why a BigQuery ML ARIMA_PLUS model forecasts a sharp rise in sales for next month. The analyst wants to show how much of each forecast value comes from trend, seasonality and holiday effects. Which function should the analyst use?

  1. AML.EVALUATE on the ARIMA_PLUS model and last month's data
  2. BML.EXPLAIN_FORECAST on the ARIMA_PLUS model
  3. CML.EXPLAIN_PREDICT on the forecast table
  4. DML.FORECAST again with a longer horizon and a confidence level of 0.99
Show the answer and why
  • AML.EVALUATE on the ARIMA_PLUS model and last month's data

    Incorrect

    ML.EVALUATE reports accuracy metrics, not the components behind each forecast value.

  • BML.EXPLAIN_FORECAST on the ARIMA_PLUS model

    Correct

    ML.EXPLAIN_FORECAST returns the forecast with its trend, seasonal and holiday components for each time point.

  • CML.EXPLAIN_PREDICT on the forecast table

    Incorrect

    ML.EXPLAIN_PREDICT gives feature attributions for supervised models, not time series decomposition.

  • DML.FORECAST again with a longer horizon and a confidence level of 0.99

    Incorrect

    More forecast values and wider intervals still do not split each value into its components.

For ARIMA_PLUS and ARIMA_PLUS_XREG models, BigQuery Explainable AI uses time series decomposition: ML.EXPLAIN_FORECAST returns the forecast along with components such as trend, seasonal periods and holiday effects, so the change in a forecast can be attributed to them.

Practice domain 6 →Practice all domains →