Data Engineer Associate · Domain 3: Data Transformation and Modeling · 22% of the exam
Task 3.6: Understand the difference between, and how to build, Gold layer objects such as materialized views, views, streaming tables, and tables for BI and analytics teams in Unity Catalog.
When a view, a materialized view, a streaming table or a plain table fits a gold dataset, how each is created and refreshed, and who can query it.
Study it
Gold objects: views, materialized views, streaming tables and tables
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
BI teams query a gold dataset of daily revenue by region from dashboards many times an hour. It joins and aggregates two silver tables whose rows are sometimes updated or deleted. The results must stay correct after such changes, and dashboards should read precomputed results. Which object should hold the gold dataset?
AA view defined on the two silver tables
BA streaming table that reads the two silver tables as streams
CA temporary view created in the dashboard query
DA materialized view defined on the two silver tables
Show the answer and why
AA view defined on the two silver tables
Incorrect
A view stores only the query text and recomputes the join and aggregation on every dashboard query.
BA streaming table that reads the two silver tables as streams
Incorrect
A streaming table handles each source row once and does not recompute its joins, so later updates and deletes in the silver tables are missed.
CA temporary view created in the dashboard query
Incorrect
A temporary view holds no precomputed results either; the join and aggregation would run again on every dashboard query.
DA materialized view defined on the two silver tables
Correct
A materialized view caches precomputed results and keeps them consistent with updated or deleted source rows, often refreshing incrementally.
Materialized views fit gold aggregations and joins that many consumers read: they precompute results and stay correct when the sources change.
The gold materialized view main.gold.daily_sales was created in Databricks SQL with no refresh schedule and feeds a dashboard. An hour ago, several rows in its silver source table were corrected. Which TWO statements are true? (Choose TWO.)
AThe dashboard shows the fixes, as the view's query runs on each read
BA refresh must run on the SQL warehouse that ran the CREATE statement
CThe dashboard shows the data as it was at the view's last refresh
DA refresh is billed as usage of the warehouse that submits it
EREFRESH MATERIALIZED VIEW main.gold.daily_sales picks the fixes up now
Show the answer and why
AThe dashboard shows the fixes, as the view's query runs on each read
Incorrect
A materialized view serves precomputed results; its data changes when the view is refreshed, not each time it is queried.
BA refresh must run on the SQL warehouse that ran the CREATE statement
Incorrect
The owner, and any user granted the REFRESH privilege, can refresh the view from the SQL editor, a notebook attached to a SQL warehouse, the Databricks SQL CLI or the Databricks SQL API.
CThe dashboard shows the data as it was at the view's last refresh
Correct
A refresh updates the view to reflect the base table at the time of the refresh, so until the next refresh the view keeps the earlier results.
DA refresh is billed as usage of the warehouse that submits it
Incorrect
A refresh runs on a serverless pipeline that Databricks creates for the view, billed as serverless Lakeflow pipelines DBUs; the warehouse size does not limit it.
EREFRESH MATERIALIZED VIEW main.gold.daily_sales picks the fixes up now
Correct
REFRESH MATERIALIZED VIEW updates the view on demand without supplying its query again, and by default it blocks until the refresh completes.
A standalone materialized view stores the result of its query as of its last refresh, which is what makes dashboard reads fast. Changes in the sources reach it only when it is refreshed: on demand with REFRESH MATERIALIZED VIEW, or on a schedule or trigger set in its definition. The refresh itself runs on a serverless pipeline, not on the submitting warehouse.
Analysts need a version of a silver table with a few columns renamed and one filtered out. It must always reflect the current silver data and must not store a copy. Which object should the data engineer create?
AA materialized view
BA table created with CREATE TABLE AS SELECT
CA streaming table
DA view
Show the answer and why
AA materialized view
Incorrect
A materialized view stores a copy of its query results, which the requirement rules out.
BA table created with CREATE TABLE AS SELECT
Incorrect
A CTAS table is a copy taken at creation time and becomes stale as silver changes.
CA streaming table
Incorrect
A streaming table stores data and processes append-only sources incrementally; it is not a light projection.
DA view
Correct
A view stores only its query text, so it processes no data at creation and always reads the current silver rows.
A view is the cheapest gold object when the logic is simple and results must always be current. Materialize only when query cost or reuse justifies it.
A gold dataset is an append-only log of enriched click events. Downstream consumers read new events within minutes, and events are never updated after they arrive. Which object type fits it best?
AA materialized view
BA streaming table
CA view
DA temporary view
Show the answer and why
AA materialized view
Incorrect
Materialized views update in seconds or minutes and are not designed for low-latency processing of each appended event.
BA streaming table
Correct
Streaming tables process each appended record once, handle large volumes and suit low-latency incremental processing.
CA view
Incorrect
A view computes its result on every query instead of storing processed events for consumers.
DA temporary view
Incorrect
A temporary view lives only in the session or pipeline that defines it and is not published for consumers.
Choose a streaming table for append-only data that is processed incrementally, each record once.
A standalone materialized view in Databricks SQL must refresh at 03:30 UTC each night, after the nightly loads finish. Which clause belongs in its CREATE OR REPLACE MATERIALIZED VIEW statement?
ATRIGGER ON UPDATE
BSCHEDULE EVERY 1 DAY
CSCHEDULE CRON '0 30 3 * * ?' AT TIME ZONE 'UTC'
DNo clause; run REFRESH MATERIALIZED VIEW once after creating it
Show the answer and why
ATRIGGER ON UPDATE
Incorrect
This refreshes the view whenever its source tables change, not at a fixed time.
BSCHEDULE EVERY 1 DAY
Incorrect
An interval schedule repeats every day but does not pin the refresh to 03:30 UTC.
CSCHEDULE CRON '0 30 3 * * ?' AT TIME ZONE 'UTC'
Correct
A CRON schedule with a time zone runs the refresh at an exact clock time.
DNo clause; run REFRESH MATERIALIZED VIEW once after creating it
Incorrect
A refresh run once after creation does not repeat, so nothing would run at 03:30 UTC on the following nights.
The definition of a standalone materialized view can carry a schedule, as an interval (EVERY) or a CRON expression with a time zone, or a trigger that fires when its sources update (TRIGGER ON UPDATE). Only CRON pins the refresh to an exact clock time.
A standalone streaming table in Databricks SQL reads new orders with STREAM and joins each one to the dimension table customers to add a segment. Some customer rows land in customers only after their orders were processed by an earlier refresh. What happens to those already-processed orders on later refreshes?
AThey keep the join result from when they were first processed
BThe next refresh joins them again with the updated customers table
CThe next refresh fails, because the dimension table has changed
DThey wait in the source stream until their customer row arrives
Show the answer and why
AThey keep the join result from when they were first processed
Correct
A refresh evaluates only rows that arrived after the last update, and a join with a static snapshot uses the snapshot state at the time of the initial processing, so late dimension rows are ignored for them.
BThe next refresh joins them again with the updated customers table
Incorrect
A refresh evaluates only the new rows and appends only new data; already-processed rows are not re-queried.
CThe next refresh fails, because the dimension table has changed
Incorrect
Changes in the static side are not an error; a refresh fails when a change to the table's definition is incompatible with existing data, such as a changed data type.
DThey wait in the source stream until their customer row arrives
Incorrect
The stream does not hold rows back for a late match; the rows are processed in the refresh in which they arrive, using the snapshot state at that time.
A streaming table processes each source row once. Its stream-static joins read the static table as it is at processing time, so dimension rows that arrive late do not change rows already written, and facts can be dropped if dimensions are late. A full refresh is the way to reprocess everything when that is needed.