Task 5.2: Choose the appropriate Delta optimization technique (deletion vectors, Liquid Clustering, CLUSTER BY AUTO) for a given table access pattern.

• AReplace REORG with VACUUM using a zero-hour retention

  Incorrect

  VACUUM removes unreferenced files; it does not rewrite files whose rows are only soft-deleted.

• BDisable deletion vectors on the table

  Incorrect

  That gives up the faster deletes for every write instead of fixing the purge.

• CRun OPTIMIZE FULL on the table every night instead

  Incorrect

  Rewriting the whole table nightly costs far more than a targeted purge.

• DSet spark.databricks.delta.reorg.purgeMode to rows

  Correct

  The rows mode handles only files with soft-deleted rows, which helps when many files have none and no columns were dropped.

By default, purgeMode is all, so a purge scans every Parquet footer for soft-deleted rows and dropped column data. On large tables where many files have no soft-deleted rows and no columns were dropped, rows limits the work to files with soft-deletes.

• AAdd device_id to dataSkippingStatsColumns, recompute stats, then cluster

  Correct

  Clustering keys need statistics, which by default cover only the first 32 columns; changing the property does not recompute existing statistics.

• BSet delta.dataSkippingNumIndexedCols to 32 on the table

  Incorrect

  32 columns is already the default, and device_id is the 45th column.

• CPartition the table by device_id instead of clustering it

  Incorrect

  Partitioning by a high-cardinality column creates huge numbers of small partitions.

• DRun OPTIMIZE ... ZORDER BY (device_id) together with the new clustering key

  Incorrect

  Liquid clustering is not compatible with ZORDER, and Z-ordering also relies on column statistics.

Clustering keys must be columns with statistics, and Delta tables collect statistics for the first 32 columns by default. Without predictive optimization, delta.dataSkippingStatsColumns names the columns to collect; existing statistics are not recomputed until you compute them again.

• AThe client keeps reading the table, but it sees deleted rows

  Incorrect

  The client cannot read the table at all after the upgrade.

• BDatabricks writes a second, compatible copy of the table for the client

  Incorrect

  No copy is created.

• CNothing changes for the client until the next VACUUM runs

  Incorrect

  The protocol changes when the feature is enabled.

• DThe protocol is upgraded, and that client can no longer read it

  Correct

  Enabling deletion vectors upgrades the table protocol, so clients without deletion vector support cannot read it.

Enabling deletion vectors upgrades the table protocol, after which clients without deletion vector support cannot read the table. On Databricks Runtime 16.4 LTS and above, the feature can be dropped to restore compatibility.

• AUse eight keys, the maximum allowed, so that every filter is covered

  Incorrect

  The limit is four clustering keys.

• BUse four keys, because more clustering keys always help pruning

  Incorrect

  On smaller tables, more keys can hurt single-column queries.

• CUse fewer keys; under 10 TB, extra keys slow single-column filters

  Correct

  Up to four keys are allowed, but for smaller tables extra keys can degrade filtering on a single column.

• DUse no clustering keys and partition the table by all four columns

  Incorrect

  Partitioning by several columns creates many small partitions.

You can specify up to four clustering keys. For tables under 10 TB, filtering on a single column performs worse with four keys than with two; the difference fades as tables grow.

• ADeletion vectors

  Correct

  On Databricks Runtime 14.2 and above, tables with deletion vectors support row-level concurrency.

• BChange data feed

  Incorrect

  The change feed records changes; it does not resolve write conflicts.

• CColumn mapping

  Incorrect

  Column mapping allows renames and drops of columns.

• DType widening

  Incorrect

  Type widening changes column types without rewrites.

Row-level concurrency lets concurrent writes to different rows succeed even when they touch the same files. Tables with deletion vectors support it on Databricks Runtime 14.2 and above.

• ARun REORG TABLE ... APPLY (PURGE) first, then the ALTER

  Incorrect

  REORG rewrites files; it does not enable the feature.

• BSet the property in the streaming table's CREATE statement

  Correct

  Deletion vectors cannot be enabled on materialized views or streaming tables with ALTER; use the CREATE statement.

• CRun OPTIMIZE on the table and then the same ALTER statement

  Incorrect

  The ALTER statement is not supported for these objects.

• DEnable deletion vectors on the source table instead

  Incorrect

  Source table settings do not change the streaming table's features.

The enableDeletionVectors property turns deletion vectors on or off. For materialized views and streaming tables, it cannot be changed with ALTER; it must be set when the object is created.