Skip to content

Data Engineer Professional · Domain 3: Data Manipulation · 12% of the exam

Task 3.3: Apply AI functions, including ai_query, to perform model inference within data pipelines for enrichment and classification tasks.

Task-specific AI functions, ai_query with endpoints, return types and failOnError, batch inference on tables, and putting results into pipelines.

Study it

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

A Lakeflow pipeline enriches support tickets by calling a custom model serving endpoint with ai_query. A few tickets make the endpoint return an error, and today that fails the whole update. The team wants the other rows to keep flowing and the failed tickets kept for a later retry. What should the data engineer do?

  1. AWrap the ai_query call in try_cast
  2. BAdd an expectation with ON VIOLATION FAIL UPDATE on the ai_query result
  3. CRaise the job's task retries
  4. DCall ai_query with failOnError => false and send rows whose errorMessage is not null to a retry table
Show the answer and why
  • AWrap the ai_query call in try_cast

    Incorrect

    try_cast handles values that cannot be cast; it does not catch errors returned by the endpoint call.

  • BAdd an expectation with ON VIOLATION FAIL UPDATE on the ai_query result

    Incorrect

    An expectation checks rows; it does not let failed inference rows continue to a retry table.

  • CRaise the job's task retries

    Incorrect

    Retrying the whole update hits the same failing rows again.

  • DCall ai_query with failOnError => false and send rows whose errorMessage is not null to a retry table

    Correct

    With failOnError set to false, ai_query returns a struct with the response and an error message, so failed rows can be separated.

failOnError defaults to true. When it is false, ai_query returns a STRUCT with the parsed response and an error message string; the message is null for rows that succeeded, so failures can be routed elsewhere.

Question 2 · choose 1

An analyst runs SELECT ai_classify(...) on a Classic SQL warehouse and gets an error. What should they use instead?

  1. AA larger Classic SQL warehouse
  2. BA Classic warehouse with Photon turned off
  3. CA pro or serverless SQL warehouse
  4. DDatabricks Runtime 12.2 LTS on a cluster
Show the answer and why
  • AA larger Classic SQL warehouse

    Incorrect

    Size does not matter; Classic warehouses are not supported.

  • BA Classic warehouse with Photon turned off

    Incorrect

    Photon is not the reason; the warehouse type is.

  • CA pro or serverless SQL warehouse

    Correct

    AI Functions are not available on Classic SQL warehouses.

  • DDatabricks Runtime 12.2 LTS on a cluster

    Incorrect

    That runtime is too old for AI Functions.

AI Functions are not available on Classic SQL warehouses; they run on pro and serverless SQL warehouses and on clusters with a supported runtime.

Question 3 · choose 1

A team wants to tag each support ticket as billing, technical or account without choosing a model, writing a prompt or creating an endpoint. Which function fits?

  1. Aai_classify(ticket_text, labels)
  2. Bai_query with a custom endpoint and prompt
  3. Cai_summarize(ticket_text)
  4. Dvector_search over the tickets
Show the answer and why
  • Aai_classify(ticket_text, labels)

    Correct

    ai_classify classifies text into labels you provide, using models maintained by Databricks.

  • Bai_query with a custom endpoint and prompt

    Incorrect

    ai_query is the general-purpose function for your own endpoints and prompts, which the team wants to avoid.

  • Cai_summarize(ticket_text)

    Incorrect

    ai_summarize returns a short summary of the text, not one of the given labels.

  • Dvector_search over the tickets

    Incorrect

    vector_search queries an AI Search index for similar items; it does not assign labels.

Task-specific AI Functions, such as ai_classify and ai_summarize, need no model selection or prompt and are recommended for getting started. ai_query is the general-purpose function for custom models and prompts.

Question 4 · choose 1

A pipeline must pull the invoice number, invoice date and total from the text of each invoice into separate fields. Which function is built for this?

  1. Aai_mask with the three field names as the entities
  2. Bai_summarize with a word limit that fits three fields
  3. Cparse_json on the invoice text
  4. Dai_extract with a schema that names the three fields
Show the answer and why
  • Aai_mask with the three field names as the entities

    Incorrect

    ai_mask returns the text with the listed entities masked, not their values as separate fields.

  • Bai_summarize with a word limit that fits three fields

    Incorrect

    ai_summarize returns a summary of the text as one string, not a structured result.

  • Cparse_json on the invoice text

    Incorrect

    parse_json parses JSON strings, and invoice text is not JSON.

  • Dai_extract with a schema that names the three fields

    Correct

    ai_extract extracts structured data from text according to a schema, from simple field names to nested objects and types.

ai_extract returns structured data from text or parsed documents according to a schema you provide, from simple field names to rich schemas for documents such as invoices.

Question 5 · choose 1

ai_query calls a chat foundation model to summarize reviews, and the next step needs the answer as JSON with fixed fields. Which ai_query argument enforces the output structure?

  1. AreturnType with a STRING type for the reply
  2. BresponseFormat set to a json_schema
  3. CfailOnError => false
  4. DmodelParameters with temperature set to 0
Show the answer and why
  • AreturnType with a STRING type for the reply

    Incorrect

    returnType describes the expected type, mainly for custom endpoints; it does not make a chat model follow a schema.

  • BresponseFormat set to a json_schema

    Correct

    responseFormat specifies the format a chat foundation model must follow, such as a JSON schema.

  • CfailOnError => false

    Incorrect

    failOnError does not shape the content of the response.

  • DmodelParameters with temperature set to 0

    Incorrect

    Temperature affects randomness, not structure.

For chat foundation models, the responseFormat argument enforces an output format, as text, json_object or json_schema, or as a DDL-style string.

Practice domain 3 →Practice all domains →