Skip to content

Data Engineer Professional · Domain 1: Developing Code for Data Processing using Python and SQL · 23% of the exam

Task 1.5: Create and automate ETL workloads using Lakeflow Jobs via the UI, API, and CLI.

Creating and changing jobs with the Jobs API, the CLI and the SDK, run-now and one-time runs, job parameters, run identities and queueing.

Study it

  • Jobs from the UI, the Jobs API, the CLI and the SDK

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

An external scheduler starts a Lakeflow job through the Jobs API run-now endpoint. When a call times out after 10 minutes, the scheduler sends the same request again, and by then the first run may already have finished. Each scheduled trigger must start exactly one run of the job. What should the scheduler send?

  1. AThe same idempotency_token on every retry of a trigger
  2. BA job setting of max_concurrent_runs 1 with queueing turned off
  3. CA runs/submit request with the job's tasks instead of run-now
  4. DA job_parameters entry with the trigger's timestamp for each call
Show the answer and why
  • AThe same idempotency_token on every retry of a trigger

    Correct

    If a run with that token already exists, run-now returns the existing run's ID instead of creating a new run, whatever its state.

  • BA job setting of max_concurrent_runs 1 with queueing turned off

    Incorrect

    This only blocks overlapping runs; a retry that arrives after the first run finished still starts a second run.

  • CA runs/submit request with the job's tasks instead of run-now

    Incorrect

    One-time submitted runs are not saved as a job, and a retried submit creates another run as well.

  • DA job_parameters entry with the trigger's timestamp for each call

    Incorrect

    A parameter value is passed to the run; the API does not use it to detect that a run was already started.

run-now accepts an optional idempotency_token. When a run with the token exists, the request returns that run's ID and creates nothing new, which makes client retries safe.

Question 2 · choose 1

A script must change only the schedule of an existing job through the Jobs API and leave every other setting untouched, without first reading the whole job definition. Which endpoint should it call?

  1. APOST /api/2.2/jobs/reset with new_settings holding the schedule
  2. BPOST /api/2.2/jobs/update with new_settings holding the schedule
  3. CPOST /api/2.2/jobs/create with the same job name and new schedule
  4. DPOST /api/2.2/jobs/run-now with the schedule as a parameter
Show the answer and why
  • APOST /api/2.2/jobs/reset with new_settings holding the schedule

    Incorrect

    reset overwrites all settings, so the other settings would be lost.

  • BPOST /api/2.2/jobs/update with new_settings holding the schedule

    Correct

    update adds, updates or removes specific top-level settings and keeps the rest.

  • CPOST /api/2.2/jobs/create with the same job name and new schedule

    Incorrect

    create makes a new job rather than changing the existing one.

  • DPOST /api/2.2/jobs/run-now with the schedule as a parameter

    Incorrect

    run-now starts a run; it does not change the job's settings.

jobs/reset replaces all of a job's settings, while jobs/update changes only the top-level fields given in new_settings and keeps the rest.

Question 3 · choose 1

A job definition is stored as job.json in a Git repository. Which Databricks CLI command creates the job from that file?

  1. Adatabricks jobs submit job.json
  2. Bdatabricks workspace import job.json
  3. Cdatabricks jobs run-now --json @job.json
  4. Ddatabricks jobs create --json @job.json
Show the answer and why
  • Adatabricks jobs submit job.json

    Incorrect

    submit starts a one-time run; it does not create a saved job.

  • Bdatabricks workspace import job.json

    Incorrect

    workspace import uploads files and notebooks, not job definitions.

  • Cdatabricks jobs run-now --json @job.json

    Incorrect

    run-now triggers an existing job by its ID.

  • Ddatabricks jobs create --json @job.json

    Correct

    jobs create takes the request body inline or from a file with @path.

The jobs command group mirrors the Jobs API. Commands that take a request body accept --json with an inline string or @ followed by a file path.

Question 4 · choose 1

User A created a job and set its Run as identity to the service principal etl-sp. User C, who can trigger the job but cannot read the source tables, starts a run from the CLI. Whose privileges does the run use?

  1. AThose of user C, who started the run from the CLI
  2. BThose of user A, who created the job
  3. CThose of etl-sp, the job's Run as identity
  4. DThose of the workspace admins group, as a fallback
Show the answer and why
  • AThose of user C, who started the run from the CLI

    Incorrect

    The caller's identity triggers the run, but data access uses the Run as identity.

  • BThose of user A, who created the job

    Incorrect

    The creator's privileges apply only when the creator is the Run as identity.

  • CThose of etl-sp, the job's Run as identity

    Correct

    A job uses its Run as principal's privileges no matter who starts the run.

  • DThose of the workspace admins group, as a fallback

    Incorrect

    Jobs do not fall back to admin privileges.

Jobs have job permissions that decide who can view, run and manage them, and a Run as identity whose privileges are used when the job runs.

Question 5 · choose 1

A long run of a job is in progress when an automation script updates the job through the Jobs API, changing its timeout, its tasks' parameters and its schedule. Which change takes effect for the run that is already active?

  1. AThe new timeout_seconds value
  2. BThe new task parameters
  3. CThe new schedule
  4. DAll of the changes, at the next task boundary
Show the answer and why
  • AThe new timeout_seconds value

    Correct

    Changes to timeout_seconds are applied to active runs; other changes apply to future runs only.

  • BThe new task parameters

    Incorrect

    Parameter changes apply to future runs, not the active one.

  • CThe new schedule

    Incorrect

    A schedule change affects when future runs start.

  • DAll of the changes, at the next task boundary

    Incorrect

    Only the timeout change reaches an active run.

For the Jobs API update and reset endpoints, changes to timeout_seconds are applied to active runs, and changes to other fields are applied to future runs only.

Practice domain 1 →Practice all domains →