Skip to content

Data Engineer Professional · Domain 3: Data Manipulation · 12% of the exam

Task 3.2: Develop a model and query semi-structured data using the VARIANT data type and related functions (e.g., parse_json, variant_get).

parse_json and try_parse_json, path extraction and casting, variant_get and try_variant_get, variant_explode, schema_of_variant, and when to shred fields into columns.

Study it

  • VARIANT: parse_json, paths, variant_get and shredding

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

A silver query extracts price from the VARIANT column payload as DECIMAL(10,2). Most events carry a number, but some partner feeds send the string "N/A", and the job currently fails on those rows. The query must keep running and leave price NULL for such rows. Which expression fits?

  1. Atry_variant_get(payload, '$.price', 'decimal(10,2)')
  2. Bvariant_get(payload, '$.price', 'decimal(10,2)')
  3. Cpayload:price::decimal(10,2)
  4. Dtry_parse_json(payload:price) to reparse the extracted field
Show the answer and why
  • Atry_variant_get(payload, '$.price', 'decimal(10,2)')

    Correct

    try_variant_get returns NULL when the path is missing or the value cannot be cast to the requested type.

  • Bvariant_get(payload, '$.price', 'decimal(10,2)')

    Incorrect

    variant_get raises an error when the cast fails, which is the current failure.

  • Cpayload:price::decimal(10,2)

    Incorrect

    The :: cast also raises an error for a value that cannot be cast.

  • Dtry_parse_json(payload:price) to reparse the extracted field

    Incorrect

    Reparsing the extracted value does not cast it to DECIMAL, so price would still not be a number.

variant_get and the :: operator raise errors when a value cannot be cast. try_variant_get takes the same path and type arguments and returns NULL instead.

Question 2 · choose 1

Before promoting frequently used fields of a VARIANT column into typed columns, a data engineer wants one schema that merges the structure of all rows in the table, including fields that appear only in some rows. Which function returns it?

  1. Aschema_of_variant_agg(payload) over the table
  2. Bschema_of_variant(payload) on the first row
  3. Cschema_of_json(to_json(payload)) on the first row
  4. DDESCRIBE TABLE on the table
Show the answer and why
  • Aschema_of_variant_agg(payload) over the table

    Correct

    This aggregate merges the schemas of all VARIANT values by field name and returns one DDL schema.

  • Bschema_of_variant(payload) on the first row

    Incorrect

    This returns the schema of a single value, so fields missing from that row are left out.

  • Cschema_of_json(to_json(payload)) on the first row

    Incorrect

    This also infers from one value and misses fields of other rows.

  • DDESCRIBE TABLE on the table

    Incorrect

    DESCRIBE shows the column type VARIANT, not the structure inside it.

schema_of_variant_agg returns the combined schema of all VARIANT values in a group, merging fields by name and using the least common type when types differ. schema_of_variant works on a single value.

Question 3 · choose 1

In a VARIANT column v, some documents contain "discount": null. The filter WHERE v:discount IS NULL misses them. Which predicate finds them?

  1. Ais_variant_null(v:discount)
  2. Bv:discount = 'null'
  3. Cisnull(v:discount)
  4. Dv:discount <=> NULL
Show the answer and why
  • Ais_variant_null(v:discount)

    Correct

    is_variant_null tests for a VARIANT-encoded null, which IS NULL does not treat as SQL NULL.

  • Bv:discount = 'null'

    Incorrect

    That compares with the string 'null', not a JSON null value.

  • Cisnull(v:discount)

    Incorrect

    isnull checks for SQL NULL, like IS NULL.

  • Dv:discount <=> NULL

    Incorrect

    Null-safe equality with NULL is true only for a SQL NULL, and a VARIANT-encoded null is not a SQL NULL.

A JSON null inside a VARIANT is a VARIANT-encoded null, not a SQL NULL. is_variant_null returns true for it, while IS NULL checks whether the expression itself is SQL NULL.

Question 4 · choose 1

A bronze STRING column contains JSON documents, and a few of them are malformed. A silver query must convert the column to VARIANT without failing. Which function should it use?

  1. Aparse_json(raw)
  2. Bschema_of_json(raw)
  3. Ctry_parse_json(raw)
  4. Dto_json(raw)
Show the answer and why
  • Aparse_json(raw)

    Incorrect

    parse_json raises an error on malformed JSON.

  • Bschema_of_json(raw)

    Incorrect

    schema_of_json returns the schema of a JSON string in DDL format, not a VARIANT value.

  • Ctry_parse_json(raw)

    Correct

    try_parse_json returns NULL for strings that cannot be parsed.

  • Dto_json(raw)

    Incorrect

    to_json converts values to JSON strings, the opposite direction.

parse_json returns an error when the string is not valid JSON, while try_parse_json returns NULL.

Question 5 · choose 1

A VARIANT column order holds an items array. A query needs one row per item, with the item as a VARIANT value. Which function produces the rows?

  1. Aexplode(order:items)
  2. Bvariant_explode(order:items)
  3. Cschema_of_variant(order:items)
  4. Dvariant_get(order, '$.items', 'string')
Show the answer and why
  • Aexplode(order:items)

    Incorrect

    explode works on ARRAY and MAP types, not on a VARIANT.

  • Bvariant_explode(order:items)

    Correct

    variant_explode returns pos, key and value columns, one row per array element or object field.

  • Cschema_of_variant(order:items)

    Incorrect

    That returns a schema string, not rows.

  • Dvariant_get(order, '$.items', 'string')

    Incorrect

    That extracts one value per row as a string.

variant_explode un-nests a VARIANT array or object into rows with pos, key and value. For arrays the key is NULL; variant_explode_outer keeps a NULL row when the input is not an array or object.

Question 6 · choose 1

The VARIANT column v holds {"items": [{"sku": "A1"}, {"sku": "B2"}]}. Which expression returns the sku of the first item?

  1. Av:items[1].sku
  2. Bv.items.sku
  3. Cv:items:sku
  4. Dv:items[0].sku
Show the answer and why
  • Av:items[1].sku

    Incorrect

    Array indexes are zero-based, so [1] is the second item.

  • Bv.items.sku

    Incorrect

    Dot notation on the column is for STRUCT fields, not VARIANT paths.

  • Cv:items:sku

    Incorrect

    A field inside an array element needs the element index.

  • Dv:items[0].sku

    Correct

    Colon notation follows field names with dots and array elements with zero-based brackets.

The : operator extracts a path from a VARIANT or JSON string; nested fields use dots and array elements use zero-based brackets.

Practice domain 3 →Practice all domains →