Data Engineer Professional · Domain 1: Developing Code for Data Processing using Python and SQL · 23% of the exam
Task 1.5: Create and automate ETL workloads using Lakeflow Jobs via the UI, API, and CLI.
Creating and changing jobs with the Jobs API, the CLI and the SDK, run-now and one-time runs, job parameters, run identities and queueing.
Study it
Jobs from the UI, the Jobs API, the CLI and the SDK
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
An external scheduler starts a Lakeflow job through the Jobs API run-now endpoint. When a call times out after 10 minutes, the scheduler sends the same request again, and by then the first run may already have finished. Each scheduled trigger must start exactly one run of the job. What should the scheduler send?
AThe same idempotency_token on every retry of a trigger
BA job setting of max_concurrent_runs 1 with queueing turned off
CA runs/submit request with the job's tasks instead of run-now
DA job_parameters entry with the trigger's timestamp for each call
Show the answer and why
AThe same idempotency_token on every retry of a trigger
Correct
If a run with that token already exists, run-now returns the existing run's ID instead of creating a new run, whatever its state.
BA job setting of max_concurrent_runs 1 with queueing turned off
Incorrect
This only blocks overlapping runs; a retry that arrives after the first run finished still starts a second run.
CA runs/submit request with the job's tasks instead of run-now
Incorrect
One-time submitted runs are not saved as a job, and a retried submit creates another run as well.
DA job_parameters entry with the trigger's timestamp for each call
Incorrect
A parameter value is passed to the run; the API does not use it to detect that a run was already started.
run-now accepts an optional idempotency_token. When a run with the token exists, the request returns that run's ID and creates nothing new, which makes client retries safe.
A script must change only the schedule of an existing job through the Jobs API and leave every other setting untouched, without first reading the whole job definition. Which endpoint should it call?
APOST /api/2.2/jobs/reset with new_settings holding the schedule
BPOST /api/2.2/jobs/update with new_settings holding the schedule
CPOST /api/2.2/jobs/create with the same job name and new schedule
DPOST /api/2.2/jobs/run-now with the schedule as a parameter
Show the answer and why
APOST /api/2.2/jobs/reset with new_settings holding the schedule
Incorrect
reset overwrites all settings, so the other settings would be lost.
BPOST /api/2.2/jobs/update with new_settings holding the schedule
Correct
update adds, updates or removes specific top-level settings and keeps the rest.
CPOST /api/2.2/jobs/create with the same job name and new schedule
Incorrect
create makes a new job rather than changing the existing one.
DPOST /api/2.2/jobs/run-now with the schedule as a parameter
Incorrect
run-now starts a run; it does not change the job's settings.
jobs/reset replaces all of a job's settings, while jobs/update changes only the top-level fields given in new_settings and keeps the rest.
User A created a job and set its Run as identity to the service principal etl-sp. User C, who can trigger the job but cannot read the source tables, starts a run from the CLI. Whose privileges does the run use?
AThose of user C, who started the run from the CLI
BThose of user A, who created the job
CThose of etl-sp, the job's Run as identity
DThose of the workspace admins group, as a fallback
Show the answer and why
AThose of user C, who started the run from the CLI
Incorrect
The caller's identity triggers the run, but data access uses the Run as identity.
BThose of user A, who created the job
Incorrect
The creator's privileges apply only when the creator is the Run as identity.
CThose of etl-sp, the job's Run as identity
Correct
A job uses its Run as principal's privileges no matter who starts the run.
DThose of the workspace admins group, as a fallback
Incorrect
Jobs do not fall back to admin privileges.
Jobs have job permissions that decide who can view, run and manage them, and a Run as identity whose privileges are used when the job runs.
A long run of a job is in progress when an automation script updates the job through the Jobs API, changing its timeout, its tasks' parameters and its schedule. Which change takes effect for the run that is already active?
AThe new timeout_seconds value
BThe new task parameters
CThe new schedule
DAll of the changes, at the next task boundary
Show the answer and why
AThe new timeout_seconds value
Correct
Changes to timeout_seconds are applied to active runs; other changes apply to future runs only.
BThe new task parameters
Incorrect
Parameter changes apply to future runs, not the active one.
CThe new schedule
Incorrect
A schedule change affects when future runs start.
DAll of the changes, at the next task boundary
Incorrect
Only the timeout change reaches an active run.
For the Jobs API update and reset endpoints, changes to timeout_seconds are applied to active runs, and changes to other fields are applied to future runs only.