Professional Machine Learning Engineer · Domain 5: Automating and orchestrating ML pipelines · 18% of the exam
Task 5.1: Developing end-to-end ML pipelines.
Validating data and models, building and orchestrating pipelines with Agent Platform Pipelines, Managed Service for Apache Airflow and Ray, and keeping preprocessing consistent between training and serving.
Study it
Validating data and models inside a pipeline
Lesson coming
Agent Platform Pipelines, Managed Service for Apache Airflow and Ray
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 1
A daily Agent Platform Pipelines run starts with a task that queries "yesterday's orders" from BigQuery, using the same table name parameter every day. The team finds that the model has been trained on the same data for a week, while every run reports success. An expensive feature step later in the pipeline should still reuse earlier results when its inputs are unchanged. What should the ML engineer do?
ATurn off execution caching for the whole pipeline job
BGive the extraction task a machine type with more CPUs and memory
CTurn off execution caching for the extraction task only
DConfigure automatic retries for the extraction task in the pipeline definition
Show the answer and why
ATurn off execution caching for the whole pipeline job
Incorrect
This fixes the stale data but also makes the expensive feature step rerun every time, which the team wants to avoid.
BGive the extraction task a machine type with more CPUs and memory
Incorrect
A bigger machine does not stop the cached output from being reused; the task never actually runs.
CTurn off execution caching for the extraction task only
Correct
The task's inputs and spec are identical each day, so its cached output is reused; disabling caching for that task alone forces a fresh query.
DConfigure automatic retries for the extraction task in the pipeline definition
Incorrect
Retries rerun a task that failed; a cached task did not fail, so retries never trigger.
Agent Platform Pipelines looks up each step's cache key (its inputs, output definition and component specification) in ML Metadata and reuses a matching execution's outputs. A step whose inputs never change is skipped every run. Caching can be turned off per task with set_caching_options(False) or for a whole run with enable_caching.
A team's pipeline uses hand-written Python components that call the Agent Platform API to upload a model, create an endpoint and deploy the model. The code breaks with SDK changes and records no lineage for the endpoint. What should the ML engineer use instead?
AThe prebuilt ModelUploadOp, EndpointCreateOp and ModelDeployOp components
BA Cloud Build step after the pipeline finishes that runs the equivalent gcloud commands
CA single custom container component that runs all three calls
DA pipeline template from Artifact Registry with the same components
Show the answer and why
AThe prebuilt ModelUploadOp, EndpointCreateOp and ModelDeployOp components
Correct
Google Cloud Pipeline Components wrap these Agent Platform operations and record their artifacts in ML Metadata.
BA Cloud Build step after the pipeline finishes that runs the equivalent gcloud commands
Incorrect
This moves deployment out of the pipeline, so the run no longer records the steps or their lineage.
CA single custom container component that runs all three calls
Incorrect
One bigger custom component keeps the same fragile API code.
DA pipeline template from Artifact Registry with the same components
Incorrect
A template stores the pipeline definition; it does not change the hand-written components inside it.
Google Cloud Pipeline Components are prebuilt KFP components for Agent Platform and other Google Cloud services, including ModelUploadOp for Model Registry and EndpointCreateOp and ModelDeployOp for endpoints. Their inputs and outputs are tracked as artifacts, which gives lineage.
A pipeline fans out into 20 expensive GPU training tasks that start after a data check. When one task fails, the team wants the pipeline to stop scheduling the tasks that have not started yet, while the running ones may finish. Today every remaining task still starts. What should the ML engineer set?
AThe pipeline's failure policy to fast
BRetries with a backoff on each training task
CExecution caching turned off for the whole pipeline run
DA Cloud Monitoring alert that pages someone to cancel the run
Show the answer and why
AThe pipeline's failure policy to fast
Correct
With fail fast, the pipeline stops scheduling new tasks after a task fails; tasks already scheduled finish.
BRetries with a backoff on each training task
Incorrect
Retries rerun a failed task; they do not stop the other tasks from being scheduled.
CExecution caching turned off for the whole pipeline run
Incorrect
Caching decides whether outputs are reused, not what happens after a failure.
DA Cloud Monitoring alert that pages someone to cancel the run
Incorrect
A manual cancel is slow and still lets tasks start before someone reacts.
Agent Platform Pipelines supports two failure policies. Fail fast stops scheduling new tasks after a task fails, while tasks already scheduled run to completion. Fail slow, the default, keeps scheduling tasks after a failure.
A team's feature generation and distributed training code is written with Ray and runs on a self-managed Ray cluster on VMs that the team patches itself. It reads from BigQuery and logs to Agent Platform. The team wants a managed Ray cluster in Agent Platform without rewriting the code. What should the ML engineer use?
AA Dataflow pipeline rewritten with Apache Beam
BAn Agent Platform pipeline with one component per Ray task
CKubeRay on a GKE cluster that the team operates
DA Ray cluster on Gemini Enterprise Agent Platform
Show the answer and why
AA Dataflow pipeline rewritten with Apache Beam
Incorrect
Dataflow is managed but would require rewriting the Ray code.
BAn Agent Platform pipeline with one component per Ray task
Incorrect
Pipelines orchestrate steps; turning Ray tasks into components is a rewrite and loses Ray's distributed runtime.
CKubeRay on a GKE cluster that the team operates
Incorrect
KubeRay runs Ray, but the team would still operate the cluster, which is what it wants to stop doing.
DA Ray cluster on Gemini Enterprise Agent Platform
Correct
Ray on Agent Platform provides managed Ray clusters that run existing Ray code and can read from BigQuery.
Ray on Gemini Enterprise Agent Platform lets you run Ray applications on managed Ray clusters that integrate with Agent Platform services, and it can read data from and write data to BigQuery. Existing Ray code runs with minimal changes.
An automated training pipeline must not train on data whose schema or value distributions break expectations, and must not deploy a model whose evaluation metrics miss the agreed thresholds. Which TWO steps should the ML engineer add to the pipeline? (Choose TWO.)
AA step before training that validates the new data's schema and statistics
BA gate that releases the trained model only if it passes its quality bar
CModel Monitoring on the endpoint after deployment, with the training schema as baseline
DA retry setting on the training step
ESpot VMs for the training step
Show the answer and why
AA step before training that validates the new data's schema and statistics
Correct
Data validation stops the pipeline before training when the input data deviates from what the model expects.
BA gate that releases the trained model only if it passes its quality bar
Correct
Model validation gates deployment on the evaluation results, so a weak model is never served.
CModel Monitoring on the endpoint after deployment, with the training schema as baseline
Incorrect
Monitoring watches production traffic after the model is live; it does not stop a bad model from being deployed.
DA retry setting on the training step
Incorrect
Retries rerun a task that failed; they add no checks on the data or the metrics.
ESpot VMs for the training step
Incorrect
Spot VMs make training cheaper; they do not check data or metrics.
In an automated ML pipeline, data validation runs before model training to decide whether to retrain or stop on unexpected data, and model validation runs after training, deploying only when the evaluation metrics meet the defined thresholds.
A TensorFlow model performs well offline but poorly in production. The training pipeline normalizes and buckets features in a Beam job, and the serving team re-implemented the same logic in Java inside the application. Small differences between the two give training-serving skew. Which design removes the cause?
AUse tf.Transform and export its transform graph with the model for serving
BWrite unit tests that compare the Java and Beam implementations on sample rows at each release
CTurn on Model Monitoring skew detection for the deployed serving model
DRetrain the model more often on the latest data from the pipeline
Show the answer and why
AUse tf.Transform and export its transform graph with the model for serving
Correct
tf.Transform exports the preprocessing as a graph that is attached to the served model, so both paths run the same logic.
BWrite unit tests that compare the Java and Beam implementations on sample rows at each release
Incorrect
Tests can catch some differences, but two implementations remain and will drift again.
CTurn on Model Monitoring skew detection for the deployed serving model
Incorrect
Skew detection reports the symptom after deployment; it does not remove the duplicated logic.
DRetrain the model more often on the latest data from the pipeline
Incorrect
Fresh data does not fix a mismatch between how features are computed in training and in serving.
Training-serving skew often comes from implementing preprocessing twice. With TensorFlow Transform, the full-pass analysis runs in the Beam pipeline and produces a transform graph that is exported with the model, so the same transformations are applied at serving time.
A pipeline's evaluation component builds a custom report with tables of per-segment metrics. Reviewers want to open it directly from the run in the Agent Platform Pipelines console instead of downloading files. How should the component publish the report?
AWrite it to an Output[HTML] artifact of the component
BEmail it from the component with an SMTP library
CReturn it as a string output parameter of the component
DWrite it to Cloud Logging as one long log entry
Show the answer and why
AWrite it to an Output[HTML] artifact of the component
Correct
HTML and Markdown output artifacts are custom visualizations that the Pipelines console can display for the run.
BEmail it from the component with an SMTP library
Incorrect
Email sends a copy elsewhere; it does not show the report in the console.
CReturn it as a string output parameter of the component
Incorrect
A string parameter is shown as a value, not rendered as a formatted report.
DWrite it to Cloud Logging as one long log entry
Incorrect
A log entry is plain text in Logging, not a report rendered with the run.
Besides predefined visualizations, Agent Platform Pipelines supports custom visualization artifacts: a component can declare an Output[HTML] or Output[Markdown] artifact and write content to its path. The files are stored under the pipeline root in Cloud Storage and can be viewed with the run.
The ML team wants an email with the final status every time its nightly Agent Platform pipeline finishes, whether it succeeds or fails, without building a separate alerting system. What should the ML engineer add?
AA Cloud Monitoring uptime check that probes the pipeline's run page every few minutes
BThe email notification component from Google Cloud Pipeline Components
CA Model Monitoring job with email alerts
DA print statement in the last component
Show the answer and why
AA Cloud Monitoring uptime check that probes the pipeline's run page every few minutes
Incorrect
Uptime checks probe URLs; they do not report pipeline run status.
BThe email notification component from Google Cloud Pipeline Components
Correct
The component sends a final status email to the given addresses when the pipeline exits.
CA Model Monitoring job with email alerts
Incorrect
Model Monitoring alerts on deployed model inputs, not pipeline runs.
DA print statement in the last component
Incorrect
Output in the logs reaches no one's inbox, and it is skipped when an earlier task fails.
Agent Platform Pipelines can notify you of the success or failure of a pipeline run: the email notification component in Google Cloud Pipeline Components sends a final status email to the addresses you specify when the pipeline exits.
Before a weekly batch scoring step, a pipeline should check whether this week's input data for a BigQuery ML model differs significantly from the data the model was trained on. The team wants the check in SQL. Which function should the step run?
AML.FORECAST on the training data
BML.EVALUATE on the model and this week's data
CML.EXPLAIN_PREDICT on this week's data
DML.VALIDATE_DATA_SKEW on the model and this week's data
Show the answer and why
AML.FORECAST on the training data
Incorrect
ML.FORECAST predicts future values for time series models.
BML.EVALUATE on the model and this week's data
Incorrect
ML.EVALUATE needs labels and measures model quality, not differences in the input data.
CML.EXPLAIN_PREDICT on this week's data
Incorrect
This explains individual predictions; it does not compare data sets.
DML.VALIDATE_DATA_SKEW on the model and this week's data
Correct
The function computes statistics for the serving data and compares them with those stored for the training data at model creation.
ML.VALIDATE_DATA_SKEW computes statistics for serving data, compares them with the statistics computed for the training data when the BigQuery ML model was created, and identifies anomalous differences, which makes it a data validation step before scoring.
A custom pipeline component outputs the BigQuery table it creates, and the next Google Cloud Pipeline Component must consume that table. The team also wants the table to appear correctly in the run's lineage. How should the custom component declare its output?
AAs a plain string parameter holding the Google Cloud table name
BAs a file written to Cloud Storage with the table name inside
CAs an artifact of type google.BQTable
DAs an environment variable set for the next task
Show the answer and why
AAs a plain string parameter holding the Google Cloud table name
Incorrect
A string passes the name but not a typed artifact that lineage and the next component recognize.
BAs a file written to Cloud Storage with the table name inside
Incorrect
A file is an untyped workaround that the next component would have to parse.
CAs an artifact of type google.BQTable
Correct
google.BQTable is a first-party artifact type for a BigQuery table, tracked in ML Metadata and understood by the components.
DAs an environment variable set for the next task
Incorrect
Pipeline tasks do not pass outputs through environment variables.
Google Cloud Pipeline Components define standard ML Metadata artifact types such as google.VertexModel, google.VertexEndpoint, google.VertexDataset, google.BQMLModel and google.BQTable. Declaring outputs with these types lets components consume each other's resources and records them in the run's lineage.