Data Engineer Professional Task 5.4: Apply Change Data Feed (CDF) to expose row-level changes (updates/deletes) for efficient incremental downstream processing.

Question 1

A job reads the change data feed of main.sales.orders weekly, from the version after the last one it processed, and merges the rows into a downstream table. The job was broken for 40 days, and now the read fails because those versions are no longer available. The downstream table must become correct again and keep updating incrementally. What should the data engineer do?

Accepted Answer

Rebuild the downstream table from a full read of orders, then continue from the version current at that read

Answer

Read the change data feed again from the old starting version, with failOnDataLoss set to false

Answer

Use RESTORE on orders to bring back the missing versions

Answer

Turn change data feed off and on again on orders

Question 2

A job reads a table's change data feed and applies updates downstream. It gets two rows for every updated record. Which rows should it skip to keep only the new values?

Accepted Answer

Rows with _change_type = 'update_preimage'

Answer

Rows with _change_type = 'update_postimage'

Answer

Rows with _change_type = 'insert'

Answer

Rows with an even _commit_version

Question 3

An analyst needs all row-level changes of main.sales.orders since version 5, using SQL. Which query returns them?

Accepted Answer

SELECT * FROM table_changes('main.sales.orders', 5)

Answer

SELECT * FROM main.sales.orders VERSION AS OF 5

Answer

DESCRIBE HISTORY main.sales.orders

Answer

SELECT * FROM main.sales.orders WHERE _commit_version >= 5

Question 4

On Databricks Runtime 19, a team wants change data feed on a Unity Catalog managed Delta table without setting delta.enableChangeDataFeed. What does the table need?

Accepted Answer

Row tracking, which automatic change data feed relies on

Answer

Deletion vectors disabled on the table first

Answer

Both legacy and automatic change data feed turned on together

Answer

Partitioning by the date the change was made

Question 5

A target table already holds data processed up to version 75 of a source table. A new stream must apply the source's changes from version 76 onward into the same target. How is the stream set up?

Accepted Answer

readChangeFeed with startingVersion 76 and a new checkpoint location

Answer

readChangeFeed with startingVersion 76 and the target's old checkpoint

Answer

A batch read of the source with VERSION AS OF 76

Answer

readChangeFeed with startingVersion 0 and a new checkpoint location

Task 5.4: Apply Change Data Feed (CDF) to expose row-level changes (updates/deletes) for efficient incremental downstream processing.

Study it

Sample questions

A job reads a table's change data feed and applies updates downstream. It gets two rows for every updated record. Which rows should it skip to keep only the new values?

An analyst needs all row-level changes of main.sales.orders since version 5, using SQL. Which query returns them?

On Databricks Runtime 19, a team wants change data feed on a Unity Catalog managed Delta table without setting delta.enableChangeDataFeed. What does the table need?

A target table already holds data processed up to version 75 of a source table. A new stream must apply the source's changes from version 76 onward into the same target. How is the stream set up?