Machine Learning Associate · Domain 4: Model Deployment · 12% of the exam
Task 4.2: Deploy a custom model to a model endpoint; Deploy and query a model for realtime inference; Split data between endpoints for realtime interference
Serving a Unity Catalog model on a Model Serving endpoint, workload size and scale to zero, querying with dataframe_split or dataframe_records, and traffic splits between served entities.
Study it
Model Serving endpoints: deploy, query and split traffic
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
A text classifier needs a custom tokenization step before its predict call, and clients of its real-time endpoint will send raw text. How should the team deploy it with Model Serving?
AA pyfunc PythonModel whose predict tokenizes the text before scoring, logged, registered and served
BRegister the model in Unity Catalog and attach the tokenizer code to the model version as a tag that the endpoint reads
CLog the bare scikit-learn model and record the tokenization steps with mlflow.log_param
DServe the model in a Feature Serving endpoint, which applies the model's preprocessing
Show the answer and why
AA pyfunc PythonModel whose predict tokenizes the text before scoring, logged, registered and served
Correct
Custom pyfunc logging deploys arbitrary Python code with the model, so the preprocessing runs inside the endpoint.
BRegister the model in Unity Catalog and attach the tokenizer code to the model version as a tag that the endpoint reads
Incorrect
Tags are key-value labels for organizing models; they are not code that the endpoint runs.
CLog the bare scikit-learn model and record the tokenization steps with mlflow.log_param
Incorrect
Parameters record values on a run; the endpoint would serve the model without any tokenization.
DServe the model in a Feature Serving endpoint, which applies the model's preprocessing
Incorrect
Feature Serving endpoints serve feature values; custom models are deployed on Model Serving endpoints.
Custom logging with pyfunc lets you deploy arbitrary Python code alongside a model, such as preprocessing and postprocessing logic around its predict function, so the endpoint can accept raw inputs. Tags and logged parameters only describe a model; they add no code to it.
A Python client posts {"dataframe_records": [...]} to a custom tabular model's serving endpoint and then reads response.json()[0], which fails. The model also depends on receiving its feature columns in the training order. What should the client change?
AKeep dataframe_records and read the predictions key of the response
BSwitch to dataframe_split and keep reading the first element of the response array
CSwitch to dataframe_split and read the predictions key of the response
DSwitch to instances and read the predictions key of the response
Show the answer and why
AKeep dataframe_records and read the predictions key of the response
Incorrect
The records format does not guarantee that column order is preserved.
BSwitch to dataframe_split and keep reading the first element of the response array
Incorrect
The endpoint wraps the model output in a predictions key, so the response is not a bare array.
CSwitch to dataframe_split and read the predictions key of the response
Correct
The split format is the recommended one and keeps the column order, and the output is read from the predictions key.
DSwitch to instances and read the predictions key of the response
Incorrect
instances is a tensor input format for models that expect tensors, not a DataFrame format for tabular features.
Custom models accept a pandas DataFrame as JSON or tensor input. The recommended dataframe_split format sends columns and data in the split orientation; dataframe_records does not guarantee that column order is preserved. instances and inputs are tensor formats. The response wraps the model's output in a predictions key.
A serving endpoint named churn routes 90% of traffic to the served entity current and 10% to challenger. Before raising the challenger's share, QA must send a fixed set of 5,000 test requests to the challenger only, without changing what production callers receive and without creating another endpoint. What should QA do?
ASend the requests to /serving-endpoints/churn/served-models/challenger/invocations
BSet challenger's traffic_percentage to 100 while QA runs, then restore the 90/10 split
CSend 50,000 requests to /serving-endpoints/churn/invocations so that about 5,000 reach the challenger
DAdd a third served entity that serves the challenger version and route the QA requests to it
Show the answer and why
ASend the requests to /serving-endpoints/churn/served-models/challenger/invocations
Correct
Querying an individual served model sends every request to it, and the endpoint's traffic settings are ignored for those requests.
BSet challenger's traffic_percentage to 100 while QA runs, then restore the 90/10 split
Incorrect
Changing the traffic configuration also changes what production callers receive during the test.
CSend 50,000 requests to /serving-endpoints/churn/invocations so that about 5,000 reach the challenger
Incorrect
Requests to the endpoint follow the traffic split, so QA cannot control which test requests reach the challenger.
DAdd a third served entity that serves the challenger version and route the QA requests to it
Incorrect
A new served entity has to be added to the traffic configuration, which changes the endpoint that production callers use.
An endpoint can serve several models with a traffic split set in traffic_config routes (for example 90/10 between current and challenger). To query one model behind the endpoint, send requests to /serving-endpoints/{endpoint-name}/served-models/{served-model-name}/invocations; the request format is the same, and the traffic settings are ignored.
A team wraps a PyTorch model in a custom mlflow.pyfunc PythonModel, logs it, and serves it on a Model Serving endpoint with the GPU_SMALL workload type. Monitoring shows that the predictions still run on the CPU. What does the Custom models overview say?
AThe pyfunc code must run the predictions on the GPU with PyTorch's own methods
BA larger GPU workload type moves the predictions to the GPU on its own
CNothing is needed, since a GPU workload type runs every model on the GPU
DLog it with the scikit-learn flavor instead, which handles the GPU automatically
Show the answer and why
AThe pyfunc code must run the predictions on the GPU with PyTorch's own methods
Correct
On GPU compute, your code must make sure predictions run on the GPU; MLflow does this automatically only for the PyTorch and Transformers flavors.
BA larger GPU workload type moves the predictions to the GPU on its own
Incorrect
Every GPU workload type still needs code that runs the predictions on the GPU.
CNothing is needed, since a GPU workload type runs every model on the GPU
Incorrect
The page says your code must be set up to run predictions on the GPU, unless the model's flavor handles it.
DLog it with the scikit-learn flavor instead, which handles the GPU automatically
Incorrect
MLflow handles the GPU automatically for models logged with the PyTorch or Transformers flavors, not for scikit-learn.
Model Serving offers CPU and GPU workload types for custom models. When you deploy with a GPU, you must make sure that your code runs predictions on the GPU, using the methods your framework provides. MLflow does this automatically for models logged with the PyTorch or Transformers flavors, so a custom pyfunc wrapper has to do it in its own code.
A custom model endpoint is expected to handle about 12 concurrent requests. Which workload size covers that concurrency?
ASmall
BExtra small
CLarge
DMedium
Show the answer and why
ASmall
Incorrect
Small is for 0-4 concurrent requests.
BExtra small
Incorrect
The available sizes are Small, Medium and Large.
CLarge
Incorrect
Large is for 16-64 requests, more than needed here.
DMedium
Correct
Medium covers 8-16 concurrent requests.
When creating a custom model serving endpoint you choose the compute size: Small for 0-4 requests, Medium for 8-16 requests and Large for 16-64 requests, on CPU or GPU workload types.
In the Serving UI, a data scientist opens Query endpoint for a model that was logged with an input example. How can she fill in a valid request quickly?
ADownload the model's artifacts and edit the conda.yaml file
BCopy the endpoint's build logs into the request box
CUse Show Example to load the logged input example
DType a SQL query against the model's training table
Show the answer and why
ADownload the model's artifacts and edit the conda.yaml file
Incorrect
conda.yaml lists dependencies; it is not a request body.
BCopy the endpoint's build logs into the request box
Incorrect
Build logs describe the container build, not a request.
CUse Show Example to load the logged input example
Correct
If the model has an input example logged, Show Example loads it into the query box.
DType a SQL query against the model's training table
Incorrect
The Query endpoint page takes JSON model input data.
From the Serving endpoint page, Query endpoint accepts JSON model input data and sends it with Send Request. If the model was logged with an input example, Show Example loads it.
An old script creates serving endpoints with a config that lists served_models. What does the current Create a new serving endpoint API reference say?
Aserved_models is required and served_entities is ignored by the API
BBoth fields are equivalent and neither is deprecated
Cserved_models must be paired with an inference table
Dserved_models is deprecated; use served_entities instead
Show the answer and why
Aserved_models is required and served_entities is ignored by the API
Incorrect
served_models is deprecated; served_entities is the field to use.
BBoth fields are equivalent and neither is deprecated
Incorrect
The API marks served_models as deprecated.
Cserved_models must be paired with an inference table
Incorrect
Inference tables are unrelated to choosing between the two fields.
Dserved_models is deprecated; use served_entities instead
Correct
The API reference marks served_models as deprecated in favour of served_entities.
In the Model Serving API, an endpoint config lists the entities to serve in served_entities (entity name, version, workload size and so on). The older served_models field is marked Deprecated, use served_entities instead.
A team wants one serving endpoint that splits traffic between its own custom churn model and an external model from a third-party provider. What does the Databricks documentation say?
AThis works, with traffic_config routes between the two
BNot possible; the two model types cannot share one endpoint
CThis works only if both models are registered in the same schema
DThis works only on GPU compute
Show the answer and why
AThis works, with traffic_config routes between the two
Incorrect
Different model types cannot be served in a single endpoint.
BNot possible; the two model types cannot share one endpoint
Correct
You cannot serve different model types, such as a custom model and an external model, in the same endpoint.
CThis works only if both models are registered in the same schema
Incorrect
The restriction is on model types, not on where models are registered.
DThis works only on GPU compute
Incorrect
Compute type does not lift the restriction on mixing model types.
A serving endpoint can serve several models with a traffic split, but all of the same type: custom models, Foundation Model APIs provisioned throughput models or external models. You cannot serve, for example, a custom model and an external model in the same endpoint.
An endpoint currently routes 90% of traffic to current and 10% to challenger. The team wants a 50/50 split without recreating the endpoint. How can it change the split?
ADelete the endpoint and create it again with the new routes
BRegister a new model version with the desired percentages as tags
CSend PUT /api/2.0/serving-endpoints/{name}/config with the new traffic_config, or use Edit configuration in the Serving UI
DChange the challenger's workload size to Large
Show the answer and why
ADelete the endpoint and create it again with the new routes
Incorrect
The traffic split of an existing endpoint can be updated in place.
BRegister a new model version with the desired percentages as tags
Incorrect
Model version tags do not configure endpoint traffic.
CSend PUT /api/2.0/serving-endpoints/{name}/config with the new traffic_config, or use Edit configuration in the Serving UI
Correct
The endpoint's config, including traffic_config routes, is updated with a PUT to its config or from the UI.
DChange the challenger's workload size to Large
Incorrect
Workload size sets compute capacity, not the share of traffic.
The traffic split between served models is set in traffic_config routes and can be updated with PUT /api/2.0/serving-endpoints/{name}/config, or from the Serving tab with the Edit configuration button.