Skip to content

Machine Learning Associate · Domain 4: Model Deployment · 12% of the exam

Task 4.1: Identify the differences and advantages of model serving approaches: batch, realtime, and streaming; Use pandas to perform batch inference; Identify how streaming inference is performed with Delta Live Tables

Choosing batch, streaming or real-time inference, scoring with mlflow.pyfunc.spark_udf and pandas UDFs, and applying a model inside a Lakeflow pipeline (formerly Delta Live Tables).

Study it

  • Batch, streaming and real-time inference

    Partly covered by: Training vs Inference

  • Batch inference with pandas and Spark UDFs; models in Lakeflow pipelines

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

A retailer computes product recommendations for its 2 million customers from data that changes once a day. Its mobile app must show a customer's recommendations within 20 milliseconds of opening, and the team wants the most cost-effective design that meets this. What should it build?

  1. AA Model Serving endpoint that the app calls to score the customer in real time each time the app opens
  2. BNightly batch scoring that publishes the predictions to an online key-value store the app reads
  3. CA nightly batch scoring job that writes a Delta table, which the app queries through a SQL warehouse
  4. DAn ai_query call to the model for each app request
Show the answer and why
  • AA Model Serving endpoint that the app calls to score the customer in real time each time the app opens

    Incorrect

    Real-time serving is for predictions that cannot be computed in advance; here they can, and batch scoring is the more cost-effective option.

  • BNightly batch scoring that publishes the predictions to an online key-value store the app reads

    Correct

    When low latency is needed but predictions can be computed offline, Databricks suggests publishing batch predictions to an online key-value store.

  • CA nightly batch scoring job that writes a Delta table, which the app queries through a SQL warehouse

    Incorrect

    Batch predictions written to tables serve higher-latency consumers; for low-latency reads they are published to an online key-value store.

  • DAn ai_query call to the model for each app request

    Incorrect

    AI Functions batch inference is in Public Preview and is a SQL batch tool, not a per-request path for an app.

Batch or streaming inference is generally the most cost-effective option for higher throughput, higher latency use cases. When predictions must be returned with low latency but can be computed offline, as with daily recommendations, the batch predictions can be published to an online key-value store. Model Serving REST endpoints are for real-time scoring.

Question 2 · choose 1

A scikit-learn model, saved with joblib, scores 500 million rows through a Series-to-Series pandas UDF whose body loads the model file and then calls predict. The job is slow because the model is loaded again for every batch. Which change addresses this?

  1. AReplace it with a regular Python UDF that scores one row at a time
  2. BRewrite it as a Series-to-scalar pandas UDF used with groupBy().agg()
  3. CScore each customer group with groupBy().applyInPandas()
  4. DRewrite it as an Iterator[pd.Series] to Iterator[pd.Series] pandas UDF that loads the model once before looping over the batches
Show the answer and why
  • AReplace it with a regular Python UDF that scores one row at a time

    Incorrect

    Row-at-a-time Python UDFs are slower than pandas UDFs, which vectorize the work.

  • BRewrite it as a Series-to-scalar pandas UDF used with groupBy().agg()

    Incorrect

    A Series-to-scalar UDF is an aggregation that loads each group into memory; it returns one value per group, not a score per row.

  • CScore each customer group with groupBy().applyInPandas()

    Incorrect

    applyInPandas loads all data of a group into memory, which risks out of memory errors, and still runs the function per group.

  • DRewrite it as an Iterator[pd.Series] to Iterator[pd.Series] pandas UDF that loads the model once before looping over the batches

    Correct

    An iterator UDF suits work that needs state initialized once, such as loading a model file, before processing every input batch.

pandas UDFs use Apache Arrow and vectorized operations. A Series-to-Series UDF is called once per batch, so anything it loads is loaded per batch. The Iterator of Series to Iterator of Series form receives all batches through an iterator, so the model can be loaded once and applied to each batch.

Question 3 · choose 2

A Lakeflow pipeline (formerly Delta Live Tables) runs in triggered mode, started by a job every 15 minutes. It scores new card transactions with an MLflow model loaded as fraud_udf = mlflow.pyfunc.spark_udf(spark, model_uri): @dp.table def scored_transactions(): feats = spark.read.table("customer_features") return (spark.readStream.table("transactions") .join(feats, "customer_id", "left") .withColumn("score", fraud_udf(*FEATURE_COLS))) Another job updates customer_features in place a few times a day, and one of its updates commits while a pipeline update is running. Which two statements are true? (Choose TWO.)

  1. ATransactions scored in earlier updates are scored again with the new feature values in the next update
  2. BThe running update joins its transactions with customer_features as it was when that update started
  3. CThe next update joins its new transactions with the updated customer_features
  4. DThe running update switches to the new customer_features as soon as the other job commits it
  5. Ecustomer_features must be read with spark.readStream for any transaction to get updated feature values
Show the answer and why
  • ATransactions scored in earlier updates are scored again with the new feature values in the next update

    Incorrect

    Records already processed are not recalculated when the static table changes, unless the table is fully refreshed.

  • BThe running update joins its transactions with customer_features as it was when that update started

    Correct

    In a triggered pipeline, the static side of the join returns results as of the time the update started.

  • CThe next update joins its new transactions with the updated customer_features

    Correct

    Each pipeline update joins its new stream records with the most current snapshot of the static table.

  • DThe running update switches to the new customer_features as soon as the other job commits it

    Incorrect

    A triggered update reads the static table as of its start; it does not pick up a version committed while it runs.

  • Ecustomer_features must be read with spark.readStream for any transaction to get updated feature values

    Incorrect

    A static read already gives each update the current snapshot, and a streaming read expects an append-only source, which this table is not.

Joining a stream of append-only records with a mostly static table is a stream-static join. In each pipeline update, the new stream records are joined with the most current snapshot of the static table; in a triggered pipeline that snapshot is the one current when the update started, and in a continuous pipeline the latest version is read each time the table processes an update. Rows joined earlier are not recalculated when the static table changes later, unless a full refresh is run. Streaming reads expect append-only sources.

Question 4 · choose 1

A data scientist has a small pandas DataFrame of new records and an MLflow model URI, and wants predictions in the notebook without Spark. Which code scores the DataFrame?

  1. Amlflow.pyfunc.get_model_dependencies(model_uri).predict(pdf)
  2. Bmlflow.pyfunc.load_model(model_uri).predict(pdf)
  3. Cmlflow.log_artifact(model_uri).predict(pdf)
  4. Dmlflow.search_runs(model_uri).predict(pdf)
Show the answer and why
  • Amlflow.pyfunc.get_model_dependencies(model_uri).predict(pdf)

    Incorrect

    get_model_dependencies returns the path of the model's dependencies file, not a model.

  • Bmlflow.pyfunc.load_model(model_uri).predict(pdf)

    Correct

    load_model loads the model as a generic Python function, and predict scores the pandas DataFrame.

  • Cmlflow.log_artifact(model_uri).predict(pdf)

    Incorrect

    log_artifact uploads a file to a run; it does not load a model.

  • Dmlflow.search_runs(model_uri).predict(pdf)

    Incorrect

    search_runs returns run metadata, not a model.

To load a logged model for inference, use mlflow.<flavor>.load_model or, for Python models, mlflow.pyfunc.load_model, which exposes a generic predict method; model.predict(model_input) then scores a pandas DataFrame.

Question 5 · choose 1

A data scientist writes a Series-to-Series pandas UDF that scores rows. What must the Python function return for each batch it receives?

  1. AA Series of the same length as the input
  2. BA single scalar value for the whole batch
  3. CA pandas DataFrame of any length
  4. DA Python list of Row objects
Show the answer and why
  • AA Series of the same length as the input

    Correct

    A Series-to-Series pandas UDF must return a pandas Series of the same length as its input.

  • BA single scalar value for the whole batch

    Incorrect

    Returning one value per batch is the Series-to-scalar (aggregate) form.

  • CA pandas DataFrame of any length

    Incorrect

    Output of arbitrary length is what mapInPandas allows, not a Series-to-Series UDF.

  • DA Python list of Row objects

    Incorrect

    The function works with pandas Series, specified with type hints.

A Series-to-Series pandas UDF takes a pandas Series and returns a pandas Series of the same length, declared with Python type hints. Spark splits the data into batches, calls the function for each batch and concatenates the results.

Question 6 · choose 1

A batch job must apply a pandas function to every partition of a Spark DataFrame, receiving pandas DataFrames and possibly returning a different number of rows than it received. Which API fits?

  1. AA Series-to-Series pandas UDF
  2. BDataFrame.describe()
  3. CDataFrame.crosstab()
  4. DDataFrame.mapInPandas()
Show the answer and why
  • AA Series-to-Series pandas UDF

    Incorrect

    A Series-to-Series UDF must return output of the same length as its input.

  • BDataFrame.describe()

    Incorrect

    describe computes statistics; it does not apply a function.

  • CDataFrame.crosstab()

    Incorrect

    crosstab builds a frequency table of two columns.

  • DDataFrame.mapInPandas()

    Correct

    mapInPandas transforms an iterator of pandas DataFrames and can return output of arbitrary length.

DataFrame.mapInPandas() maps an iterator of pandas.DataFrame to another iterator of pandas.DataFrame and returns the result as a PySpark DataFrame. Its output can have any length, unlike pandas UDFs such as Series to Series.

Question 7 · choose 1

A model logged with specific library versions is loaded for batch scoring as a PySpark UDF on Databricks Runtime 17.3 LTS ML. What restores the model's dependencies inside the UDF?

  1. Aget_model_dependencies called inside the UDF
  2. Bmodel_uri pointing at the model's conda.yaml file
  3. Cenv_manager="virtualenv"
  4. Dnested=True on the loading call
Show the answer and why
  • Aget_model_dependencies called inside the UDF

    Incorrect

    get_model_dependencies returns the path of a dependencies file to install with %pip in a notebook; it is not how the UDF restores them.

  • Bmodel_uri pointing at the model's conda.yaml file

    Incorrect

    model_uri points at the model itself; dependencies are restored with env_manager.

  • Cenv_manager="virtualenv"

    Correct

    When a model is loaded as a PySpark UDF, env_manager="virtualenv" restores the model's dependencies.

  • Dnested=True on the loading call

    Incorrect

    nested is an argument for starting MLflow runs, not for loading a UDF.

In Databricks Runtime 11.0 ML and above, mlflow.pyfunc.get_model_dependencies helps restore a pyfunc model's dependencies in a notebook. When you load a model as a PySpark UDF, specify env_manager="virtualenv" to restore the model dependencies in the context of the UDF.

Question 8 · choose 1

A team wants to apply an MLflow model inside a Lakeflow pipeline (formerly Delta Live Tables). How does the Databricks pipelines documentation describe the role of an MLflow model there?

  1. AAs a transformation that acts on a Spark DataFrame and returns a Spark DataFrame
  2. BAs a data source that the pipeline ingests with Auto Loader
  3. CAs a separate serving endpoint that the pipeline must call over REST
  4. DAs a sink that receives the pipeline's final table
Show the answer and why
  • AAs a transformation that acts on a Spark DataFrame and returns a Spark DataFrame

    Correct

    MLflow models are treated as transformations, so a pipeline dataset can be defined with a query that applies the model.

  • BAs a data source that the pipeline ingests with Auto Loader

    Incorrect

    Models are applied to data as transformations, not ingested as data.

  • CAs a separate serving endpoint that the pipeline must call over REST

    Incorrect

    Pipelines can apply MLflow models directly as transformations.

  • DAs a sink that receives the pipeline's final table

    Incorrect

    Models transform DataFrames; they are not pipeline sinks.

Pipelines can define a dataset against any query that returns a DataFrame, using built-in Spark operations, UDFs, custom logic and MLflow models as transformations. MLflow models act on a Spark DataFrame input and return the results as a Spark DataFrame, so existing scoring code converts to a pipeline with a few lines.

Question 9 · choose 1

A team chooses Model Serving for real-time predictions. How does Model Serving handle changes in request volume?

  1. AThe team must add and remove servers by hand
  2. BIt scales up or down automatically with demand
  3. CIt queues requests until a nightly batch job runs
  4. DIt rejects requests above a fixed rate set at creation
Show the answer and why
  • AThe team must add and remove servers by hand

    Incorrect

    Model Serving scales automatically.

  • BIt scales up or down automatically with demand

    Correct

    Model Serving automatically scales to demand, saving cost while keeping latency low.

  • CIt queues requests until a nightly batch job runs

    Incorrect

    Model Serving answers requests in real time through REST APIs.

  • DIt rejects requests above a fixed rate set at creation

    Incorrect

    Model Serving scales with demand instead of using a fixed capacity.

Model Serving exposes each served model as a REST API and provides a highly available, low-latency service that automatically scales up or down to meet demand changes, saving infrastructure costs while optimizing latency.

Practice domain 4 →Practice all domains →