Skip to content

Data Engineer Associate · Domain 6: Troubleshooting, Monitoring, and Optimization · 10% of the exam

Task 6.1: Identify trends in job performance using the Lakeflow Jobs run history view to compare current execution times against historical baselines.

The runs list and matrix view, run durations over time, and the system tables for job history.

Study it

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

A data engineer wants to see, in the Lakeflow Jobs UI and without writing queries, how each task's duration in the latest run compares with that task's durations over the past weeks. Where should the engineer look?

  1. AThe Spark UI of the latest run's cluster
  2. BThe compute event log of the job cluster
  3. CThe Timeline view of the latest job run
  4. DThe matrix view on the job's Runs tab
Show the answer and why
  • AThe Spark UI of the latest run's cluster

    Incorrect

    The Spark UI shows one cluster's jobs and stages; it has no history of earlier job runs.

  • BThe compute event log of the job cluster

    Incorrect

    The event log records cluster events such as starts, resizes and terminations, not task durations across runs.

  • CThe Timeline view of the latest job run

    Incorrect

    The Timeline view shows where time went inside one run; it does not compare against earlier runs.

  • DThe matrix view on the job's Runs tab

    Correct

    The matrix view shows the history of runs with a cell per task, and bar heights indicate run durations, so trends across runs stand out.

The Runs tab's matrix view is the built-in history: each column is a run and each task's cell shows its status and duration.

Question 2 · choose 1

A data engineer needs to compare a job's run durations over the past six months to see when it started slowing down. The Runs tab no longer shows the older runs. Where can the engineer get this history?

  1. AThe matrix view on the job's Runs tab, scrolled back further
  2. BThe system.lakeflow.job_run_timeline system table
  3. CThe Spark UI of the job's most recent cluster
  4. DThe compute event log of the job cluster
Show the answer and why
  • AThe matrix view on the job's Runs tab, scrolled back further

    Incorrect

    The Jobs UI keeps run history for 60 days, so runs from six months ago are no longer listed.

  • BThe system.lakeflow.job_run_timeline system table

    Correct

    The jobs system tables record job runs across the account with a 365-day free retention period, and they can be queried for trends.

  • CThe Spark UI of the job's most recent cluster

    Incorrect

    The Spark UI shows one cluster's Spark jobs and stages, not earlier job runs.

  • DThe compute event log of the job cluster

    Incorrect

    The event log records cluster lifecycle events, not job run durations over months.

Use the Runs tab for recent history and the system.lakeflow tables for long-term analysis, such as duration trends against a baseline.

Question 3 · choose 1

Auditors require that the rendered output of each monthly notebook run be available for a year. What should the data engineer do?

  1. ANothing, since Databricks keeps the details of each job run indefinitely
  2. BSet a long timeout on the job so its runs stay open
  3. CClone the job each month so that each copy keeps its own run history
  4. DExport each run's notebook results, such as to HTML, within 60 days
Show the answer and why
  • ANothing, since Databricks keeps the details of each job run indefinitely

    Incorrect

    Databricks maintains job run history for up to 60 days.

  • BSet a long timeout on the job so its runs stay open

    Incorrect

    A timeout limits how long a run may take; it does not extend how long results are kept.

  • CClone the job each month so that each copy keeps its own run history

    Incorrect

    Run history is kept for up to 60 days whichever job a run belongs to, so the copies lose their older runs too.

  • DExport each run's notebook results, such as to HTML, within 60 days

    Correct

    Databricks recommends exporting run results before they expire; notebook runs can be exported to HTML.

Run history in the Jobs UI lasts 60 days. Persist what you must keep: export notebook output or write results to tables.

Question 4 · choose 1

A classic jobs compute run now takes 25 minutes instead of 15. Before opening the Spark UI, the data engineer wants to know whether the extra time is spent waiting for compute, installing libraries or actually running. Where should the engineer look?

  1. AThe run's Duration breakdown by phase
  2. BThe jobs timeline in the Spark UI of the job cluster
  3. CThe compute event log of the cluster
  4. DThe matrix view of the job's runs
Show the answer and why
  • AThe run's Duration breakdown by phase

    Correct

    Hovering over a run's Duration shows the time in the Queued, Waiting for resources, Library installation and Running phases.

  • BThe jobs timeline in the Spark UI of the job cluster

    Incorrect

    The Spark UI covers only what happens once Spark is running, not queueing or library installation.

  • CThe compute event log of the cluster

    Incorrect

    The event log lists cluster events; it does not split a job run's duration into phases.

  • DThe matrix view of the job's runs

    Incorrect

    The matrix view does not split a run into its queueing, waiting, installation and running phases.

The run breakdown shows which phase grew, so the fix can target queueing, compute start-up, library installs or the work itself.

Question 5 · choose 1

A platform lead wants to see, across all jobs in the workspace, how many runs finished in the last 48 hours and which error types caused the most failures. Which view shows this?

  1. AThe matrix view on each job's page, opened one job at a time
  2. BThe system.billing.usage table
  3. CThe Runs tab under Jobs & Pipelines, filtered to Jobs
  4. DFailure notifications configured on each job
Show the answer and why
  • AThe matrix view on each job's page, opened one job at a time

    Incorrect

    The matrix covers a single job, so a workspace-wide picture would need every job opened in turn.

  • BThe system.billing.usage table

    Incorrect

    The billing table records usage and cost, not run outcomes or error types.

  • CThe Runs tab under Jobs & Pipelines, filtered to Jobs

    Correct

    It shows a Finished runs count graph and a Top 5 error types table for the selected time range across jobs.

  • DFailure notifications configured on each job

    Incorrect

    Notifications report individual failures as they happen; they do not summarize error types.

The workspace-level Runs tab gives a cross-job view of recent outcomes and the most common errors, with filters for status, run as user and error code.

Practise domain 6 →Practise all domains →