Machine Learning Associate · Domain 4: Model Deployment · 12% of the exam
Task 4.1: Identify the differences and advantages of model serving approaches: batch, realtime, and streaming; Use pandas to perform batch inference; Identify how streaming inference is performed with Delta Live Tables
Choosing batch, streaming or real-time inference, scoring with mlflow.pyfunc.spark_udf and pandas UDFs, and applying a model inside a Lakeflow pipeline (formerly Delta Live Tables).
Batch inference with pandas and Spark UDFs; models in Lakeflow pipelines
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
A retailer computes product recommendations for its 2 million customers from data that changes once a day. Its mobile app must show a customer's recommendations within 20 milliseconds of opening, and the team wants the most cost-effective design that meets this. What should it build?
AA Model Serving endpoint that the app calls to score the customer in real time each time the app opens
BNightly batch scoring that publishes the predictions to an online key-value store the app reads
CA nightly batch scoring job that writes a Delta table, which the app queries through a SQL warehouse
DAn ai_query call to the model for each app request
Show the answer and why
AA Model Serving endpoint that the app calls to score the customer in real time each time the app opens
Incorrect
Real-time serving is for predictions that cannot be computed in advance; here they can, and batch scoring is the more cost-effective option.
BNightly batch scoring that publishes the predictions to an online key-value store the app reads
Correct
When low latency is needed but predictions can be computed offline, Databricks suggests publishing batch predictions to an online key-value store.
CA nightly batch scoring job that writes a Delta table, which the app queries through a SQL warehouse
Incorrect
Batch predictions written to tables serve higher-latency consumers; for low-latency reads they are published to an online key-value store.
DAn ai_query call to the model for each app request
Incorrect
AI Functions batch inference is in Public Preview and is a SQL batch tool, not a per-request path for an app.
Batch or streaming inference is generally the most cost-effective option for higher throughput, higher latency use cases. When predictions must be returned with low latency but can be computed offline, as with daily recommendations, the batch predictions can be published to an online key-value store. Model Serving REST endpoints are for real-time scoring.
A scikit-learn model, saved with joblib, scores 500 million rows through a Series-to-Series pandas UDF whose body loads the model file and then calls predict. The job is slow because the model is loaded again for every batch. Which change addresses this?
AReplace it with a regular Python UDF that scores one row at a time
BRewrite it as a Series-to-scalar pandas UDF used with groupBy().agg()
CScore each customer group with groupBy().applyInPandas()
DRewrite it as an Iterator[pd.Series] to Iterator[pd.Series] pandas UDF that loads the model once before looping over the batches
Show the answer and why
AReplace it with a regular Python UDF that scores one row at a time
Incorrect
Row-at-a-time Python UDFs are slower than pandas UDFs, which vectorize the work.
BRewrite it as a Series-to-scalar pandas UDF used with groupBy().agg()
Incorrect
A Series-to-scalar UDF is an aggregation that loads each group into memory; it returns one value per group, not a score per row.
CScore each customer group with groupBy().applyInPandas()
Incorrect
applyInPandas loads all data of a group into memory, which risks out of memory errors, and still runs the function per group.
DRewrite it as an Iterator[pd.Series] to Iterator[pd.Series] pandas UDF that loads the model once before looping over the batches
Correct
An iterator UDF suits work that needs state initialized once, such as loading a model file, before processing every input batch.
pandas UDFs use Apache Arrow and vectorized operations. A Series-to-Series UDF is called once per batch, so anything it loads is loaded per batch. The Iterator of Series to Iterator of Series form receives all batches through an iterator, so the model can be loaded once and applied to each batch.
A Lakeflow pipeline (formerly Delta Live Tables) runs in triggered mode, started
by a job every 15 minutes. It scores new card transactions with an MLflow model
loaded as fraud_udf = mlflow.pyfunc.spark_udf(spark, model_uri):
@dp.table
def scored_transactions():
feats = spark.read.table("customer_features")
return (spark.readStream.table("transactions")
.join(feats, "customer_id", "left")
.withColumn("score", fraud_udf(*FEATURE_COLS)))
Another job updates customer_features in place a few times a day, and one of
its updates commits while a pipeline update is running. Which two statements
are true? (Choose TWO.)
ATransactions scored in earlier updates are scored again with the new feature values in the next update
BThe running update joins its transactions with customer_features as it was when that update started
CThe next update joins its new transactions with the updated customer_features
DThe running update switches to the new customer_features as soon as the other job commits it
Ecustomer_features must be read with spark.readStream for any transaction to get updated feature values
Show the answer and why
ATransactions scored in earlier updates are scored again with the new feature values in the next update
Incorrect
Records already processed are not recalculated when the static table changes, unless the table is fully refreshed.
BThe running update joins its transactions with customer_features as it was when that update started
Correct
In a triggered pipeline, the static side of the join returns results as of the time the update started.
CThe next update joins its new transactions with the updated customer_features
Correct
Each pipeline update joins its new stream records with the most current snapshot of the static table.
DThe running update switches to the new customer_features as soon as the other job commits it
Incorrect
A triggered update reads the static table as of its start; it does not pick up a version committed while it runs.
Ecustomer_features must be read with spark.readStream for any transaction to get updated feature values
Incorrect
A static read already gives each update the current snapshot, and a streaming read expects an append-only source, which this table is not.
Joining a stream of append-only records with a mostly static table is a stream-static join. In each pipeline update, the new stream records are joined with the most current snapshot of the static table; in a triggered pipeline that snapshot is the one current when the update started, and in a continuous pipeline the latest version is read each time the table processes an update. Rows joined earlier are not recalculated when the static table changes later, unless a full refresh is run. Streaming reads expect append-only sources.
A data scientist has a small pandas DataFrame of new records and an MLflow model URI, and wants predictions in the notebook without Spark. Which code scores the DataFrame?
get_model_dependencies returns the path of the model's dependencies file, not a model.
Bmlflow.pyfunc.load_model(model_uri).predict(pdf)
Correct
load_model loads the model as a generic Python function, and predict scores the pandas DataFrame.
Cmlflow.log_artifact(model_uri).predict(pdf)
Incorrect
log_artifact uploads a file to a run; it does not load a model.
Dmlflow.search_runs(model_uri).predict(pdf)
Incorrect
search_runs returns run metadata, not a model.
To load a logged model for inference, use mlflow.<flavor>.load_model or, for Python models, mlflow.pyfunc.load_model, which exposes a generic predict method; model.predict(model_input) then scores a pandas DataFrame.
A data scientist writes a Series-to-Series pandas UDF that scores rows. What must the Python function return for each batch it receives?
AA Series of the same length as the input
BA single scalar value for the whole batch
CA pandas DataFrame of any length
DA Python list of Row objects
Show the answer and why
AA Series of the same length as the input
Correct
A Series-to-Series pandas UDF must return a pandas Series of the same length as its input.
BA single scalar value for the whole batch
Incorrect
Returning one value per batch is the Series-to-scalar (aggregate) form.
CA pandas DataFrame of any length
Incorrect
Output of arbitrary length is what mapInPandas allows, not a Series-to-Series UDF.
DA Python list of Row objects
Incorrect
The function works with pandas Series, specified with type hints.
A Series-to-Series pandas UDF takes a pandas Series and returns a pandas Series of the same length, declared with Python type hints. Spark splits the data into batches, calls the function for each batch and concatenates the results.
A batch job must apply a pandas function to every partition of a Spark DataFrame, receiving pandas DataFrames and possibly returning a different number of rows than it received. Which API fits?
AA Series-to-Series pandas UDF
BDataFrame.describe()
CDataFrame.crosstab()
DDataFrame.mapInPandas()
Show the answer and why
AA Series-to-Series pandas UDF
Incorrect
A Series-to-Series UDF must return output of the same length as its input.
BDataFrame.describe()
Incorrect
describe computes statistics; it does not apply a function.
CDataFrame.crosstab()
Incorrect
crosstab builds a frequency table of two columns.
DDataFrame.mapInPandas()
Correct
mapInPandas transforms an iterator of pandas DataFrames and can return output of arbitrary length.
DataFrame.mapInPandas() maps an iterator of pandas.DataFrame to another iterator of pandas.DataFrame and returns the result as a PySpark DataFrame. Its output can have any length, unlike pandas UDFs such as Series to Series.
A model logged with specific library versions is loaded for batch scoring as a PySpark UDF on Databricks Runtime 17.3 LTS ML. What restores the model's dependencies inside the UDF?
Aget_model_dependencies called inside the UDF
Bmodel_uri pointing at the model's conda.yaml file
Cenv_manager="virtualenv"
Dnested=True on the loading call
Show the answer and why
Aget_model_dependencies called inside the UDF
Incorrect
get_model_dependencies returns the path of a dependencies file to install with %pip in a notebook; it is not how the UDF restores them.
Bmodel_uri pointing at the model's conda.yaml file
Incorrect
model_uri points at the model itself; dependencies are restored with env_manager.
Cenv_manager="virtualenv"
Correct
When a model is loaded as a PySpark UDF, env_manager="virtualenv" restores the model's dependencies.
Dnested=True on the loading call
Incorrect
nested is an argument for starting MLflow runs, not for loading a UDF.
In Databricks Runtime 11.0 ML and above, mlflow.pyfunc.get_model_dependencies helps restore a pyfunc model's dependencies in a notebook. When you load a model as a PySpark UDF, specify env_manager="virtualenv" to restore the model dependencies in the context of the UDF.
A team wants to apply an MLflow model inside a Lakeflow pipeline (formerly Delta Live Tables). How does the Databricks pipelines documentation describe the role of an MLflow model there?
AAs a transformation that acts on a Spark DataFrame and returns a Spark DataFrame
BAs a data source that the pipeline ingests with Auto Loader
CAs a separate serving endpoint that the pipeline must call over REST
DAs a sink that receives the pipeline's final table
Show the answer and why
AAs a transformation that acts on a Spark DataFrame and returns a Spark DataFrame
Correct
MLflow models are treated as transformations, so a pipeline dataset can be defined with a query that applies the model.
BAs a data source that the pipeline ingests with Auto Loader
Incorrect
Models are applied to data as transformations, not ingested as data.
CAs a separate serving endpoint that the pipeline must call over REST
Incorrect
Pipelines can apply MLflow models directly as transformations.
DAs a sink that receives the pipeline's final table
Incorrect
Models transform DataFrames; they are not pipeline sinks.
Pipelines can define a dataset against any query that returns a DataFrame, using built-in Spark operations, UDFs, custom logic and MLflow models as transformations. MLflow models act on a Spark DataFrame input and return the results as a Spark DataFrame, so existing scoring code converts to a pipeline with a few lines.
A team chooses Model Serving for real-time predictions. How does Model Serving handle changes in request volume?
AThe team must add and remove servers by hand
BIt scales up or down automatically with demand
CIt queues requests until a nightly batch job runs
DIt rejects requests above a fixed rate set at creation
Show the answer and why
AThe team must add and remove servers by hand
Incorrect
Model Serving scales automatically.
BIt scales up or down automatically with demand
Correct
Model Serving automatically scales to demand, saving cost while keeping latency low.
CIt queues requests until a nightly batch job runs
Incorrect
Model Serving answers requests in real time through REST APIs.
DIt rejects requests above a fixed rate set at creation
Incorrect
Model Serving scales with demand instead of using a fixed capacity.
Model Serving exposes each served model as a REST API and provides a highly available, low-latency service that automatically scales up or down to meet demand changes, saving infrastructure costs while optimizing latency.