Skip to content

Data Engineer Associate · Domain 4: Working with Lakeflow Jobs · 16% of the exam

Task 4.3: Implement job schedules using Lakeflow Jobs with an understanding of trigger types (scheduled, file arrival, and table update).

Simple and cron schedules with time zones, file arrival triggers on volumes and external locations, table update triggers, continuous jobs, and pausing a trigger.

Study it

  • Schedules, file arrival triggers, table update triggers and continuous jobs

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 2

A job uses a file arrival trigger on /Volumes/raw/sales/landing/. Which TWO statements about this trigger are true? (Choose TWO.)

  1. AIt needs a separate trigger for each subdirectory under landing/
  2. BIt can watch the root or a subpath of a Unity Catalog volume or external location
  3. CThe path can include a wildcard such as landing/*.csv to select files
  4. DIt checks for new files about once a minute on a best-effort basis
  5. EIt requires the job to also have a continuous trigger
Show the answer and why
  • AIt needs a separate trigger for each subdirectory under landing/

    Incorrect

    A file arrival trigger checks for new files recursively in all subdirectories of the configured location, so one trigger covers them.

  • BIt can watch the root or a subpath of a Unity Catalog volume or external location

    Correct

    The storage location must be a volume or external location in Unity Catalog, at its root or a subpath.

  • CThe path can include a wildcard such as landing/*.csv to select files

    Incorrect

    The trigger path cannot contain wildcards such as * or ?.

  • DIt checks for new files about once a minute on a best-effort basis

    Correct

    File arrival triggers make a best effort to check for new files every minute, depending on the storage performance.

  • EIt requires the job to also have a continuous trigger

    Incorrect

    A continuous trigger cannot be combined with other triggers; a file arrival trigger works on its own.

File arrival triggers watch a Unity Catalog storage location recursively and start a run when new files appear.

Question 2 · choose 1

A job must start at the top of every hour all year, with no skipped or doubled hourly runs when clocks change for daylight saving time. How should its schedule be set?

  1. AAn advanced schedule at minute 0 of each hour, with time zone UTC
  2. BAn advanced schedule at minute 0 of each hour, with time zone America/New_York
  3. CA simple schedule with an interval of 1 hour
  4. DA continuous trigger, with the notebook sleeping one hour before it ends
Show the answer and why
  • AAn advanced schedule at minute 0 of each hour, with time zone UTC

    Correct

    UTC has no daylight saving changes, and the documentation says to choose UTC when a job must run at every hour in absolute time.

  • BAn advanced schedule at minute 0 of each hour, with time zone America/New_York

    Incorrect

    In a time zone that observes daylight saving time, an hourly job can be skipped or appear delayed when the change happens.

  • CA simple schedule with an interval of 1 hour

    Incorrect

    A simple schedule runs at an interval from a start the scheduler chooses; you cannot pick the first run time, so runs are not on the hour.

  • DA continuous trigger, with the notebook sleeping one hour before it ends

    Incorrect

    A continuous job starts a new run whenever one ends; a sleep adds cost and drift instead of a clock-based schedule.

Advanced schedules set the period, start time and time zone. Choose UTC when absolute timing matters across daylight saving transitions.

Question 3 · choose 1

A job must run at 06:00 Istanbul time on weekdays only. Which trigger setup does this?

  1. AA simple schedule with an interval of 24 hours
  2. BA file arrival trigger on the folder where the job's input files land
  3. CA continuous trigger with a check for the weekday at the start of the notebook
  4. DAn advanced schedule with cron for 06:00 Mon-Fri, time zone Europe/Istanbul
Show the answer and why
  • AA simple schedule with an interval of 24 hours

    Incorrect

    A simple schedule repeats from a start the scheduler chooses and also runs on weekends.

  • BA file arrival trigger on the folder where the job's input files land

    Incorrect

    File arrival starts runs when files appear, not at a fixed time on chosen days.

  • CA continuous trigger with a check for the weekday at the start of the notebook

    Incorrect

    A continuous trigger starts a new run whenever the previous one ends, so the job would run all day.

  • DAn advanced schedule with cron for 06:00 Mon-Fri, time zone Europe/Istanbul

    Correct

    Advanced schedules set the period, start time and time zone, and can be edited as Quartz cron expressions.

Simple schedules give an interval; advanced schedules give full calendar control through Quartz cron syntax with a time zone.

Question 4 · choose 1

During a source system's maintenance week, a scheduled job must not start new runs, but its schedule and settings must be kept, and a run already in progress may finish. What should the data engineer do?

  1. ADelete the trigger and recreate it after the maintenance
  2. BPause the trigger, then resume it after the maintenance
  3. CSet the job's maximum concurrent runs to 0
  4. DCancel each new run as soon as it starts
Show the answer and why
  • ADelete the trigger and recreate it after the maintenance

    Incorrect

    Deleting removes the configured schedule, which then has to be rebuilt.

  • BPause the trigger, then resume it after the maintenance

    Correct

    A paused trigger starts no new runs while active runs continue, and resuming restores the same schedule.

  • CSet the job's maximum concurrent runs to 0

    Incorrect

    This changes a job setting that must be kept, and limiting concurrency is not how a schedule is suspended.

  • DCancel each new run as soon as it starts

    Incorrect

    Manual cancellation is error-prone and still starts compute for each run.

Pausing a trigger in the Schedules & Triggers section stops new runs without changing its configuration, and resuming restores the schedule.

Question 5 · choose 1

A job joins main.sales.orders and main.sales.customers, which different processes update. It should start only after both tables have been updated since its last run. How should the trigger be set?

  1. AA table update trigger on main.sales.orders, with Wait after last change set to one hour
  2. BOne trigger on both tables, with Trigger when set to Any table is updated
  3. COne trigger on both tables, with Trigger when set to All tables are updated
  4. DA file arrival trigger on the cloud storage paths that hold the two tables
Show the answer and why
  • AA table update trigger on main.sales.orders, with Wait after last change set to one hour

    Incorrect

    This watches orders alone; waiting after its last change does not make the run wait for customers to be updated.

  • BOne trigger on both tables, with Trigger when set to Any table is updated

    Incorrect

    Any starts a run as soon as one of the tables changes.

  • COne trigger on both tables, with Trigger when set to All tables are updated

    Correct

    With several tables in one trigger, All waits until each of them has been updated before starting a run.

  • DA file arrival trigger on the cloud storage paths that hold the two tables

    Incorrect

    File arrival triggers cannot watch paths that contain tables; table changes are what table update triggers detect.

A table update trigger can watch up to 10 tables and start a run when any or all of them change.

Question 6 · choose 1

Files arrive in bursts: about 50 files within one minute, a few times a day. A job with a file arrival trigger should start once per burst, after the burst has fully landed. Which advanced option should be set?

  1. AWait after last change in seconds, set to about 60
  2. BMinimum time between triggers in seconds, set to about 60
  3. CMaximum concurrent runs set to 1
  4. DA scheduled trigger each minute in addition
Show the answer and why
  • AWait after last change in seconds, set to about 60

    Correct

    The trigger waits this long after the most recent file arrives, and each new file resets the timer, so a run starts once the burst is complete.

  • BMinimum time between triggers in seconds, set to about 60

    Incorrect

    This is a cooldown after a run completes; the first file of a burst can still start a run while the rest are landing.

  • CMaximum concurrent runs set to 1

    Incorrect

    Concurrency caps how many runs are active at once; it does not make the trigger wait for the burst to finish.

  • DA scheduled trigger each minute in addition

    Incorrect

    A schedule starts runs by time, whether or not a burst has finished.

The two advanced options of file arrival and table update triggers are a cooldown (minimum time between triggers) and debouncing (wait after last change).

Practise domain 4 →Practise all domains →