Skip to content

Associate Developer for Apache Spark · Domain 3: Developing Apache Spark DataFrame/DataSet API Applications · 30% of the exam

Task 3.3: Manipulate and utilize Date data type, such as Unix epoch to date string, and extract date component

from_unixtime, to_date, date_format patterns, the session time zone, and extracting year, month and day of week.

Study it

  • Dates, timestamps and Unix epoch values

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

The epoch value 1700000000 is 2023-11-14 22:13:20 UTC. A developer runs: from pyspark.sql import functions as F spark.conf.set("spark.sql.session.timeZone", "Asia/Tokyo") df = spark.createDataFrame([(1700000000,)], "ts BIGINT") df.select(F.from_unixtime("ts", "yyyy-MM-dd").alias("day")).show() What does the column day hold?

  1. A2023-11-14, the UTC date
  2. B2023-11-15 07:13:20, the Tokyo time to the second
  3. C2023-11-14 22:13:20, the UTC time to the second
  4. D2023-11-15, the Tokyo date
Show the answer and why
  • A2023-11-14, the UTC date

    Incorrect

    from_unixtime renders the moment in the session's time zone, here Asia/Tokyo, nine hours ahead of UTC.

  • B2023-11-15 07:13:20, the Tokyo time to the second

    Incorrect

    The format argument yyyy-MM-dd replaces the default pattern, so no time of day is printed.

  • C2023-11-14 22:13:20, the UTC time to the second

    Incorrect

    This is the UTC moment in the default pattern, but the session time zone and the given format both apply.

  • D2023-11-15, the Tokyo date

    Correct

    The moment is rendered in the session time zone, Asia/Tokyo (07:13:20 on 15 November), with the yyyy-MM-dd pattern.

from_unixtime converts seconds since the Unix epoch to a string representing that moment in the current time zone, using the given format (default yyyy-MM-dd HH:mm:ss). The function's own example shows which zone that is: it sets spark.sql.session.timeZone (the session local time zone) to America/Los_Angeles and renders 1428476400, which is 07:00 UTC on 8 April 2015, as 2015-04-08 00:00:00, the Los Angeles time. Likewise 22:13:20 UTC on 14 November is 07:13:20 on 15 November in Tokyo, and the pattern yyyy-MM-dd keeps only the date.

Question 2 · choose 1

A developer runs: from pyspark.sql import functions as F df = spark.createDataFrame([("2026-02-01", "2026-03-01")], "start STRING, finish STRING") df.select(F.datediff(F.to_date("finish"), F.to_date("start")).alias("d")).show() What does d hold?

  1. A-28
  2. B1
  3. C28
  4. D29
Show the answer and why
  • A-28

    Incorrect

    datediff(end, start) counts the days from start to end, and finish is the later date here.

  • B1

    Incorrect

    datediff returns a number of days, not of months.

  • C28

    Correct

    datediff(end, start) returns the number of days from start to end: February 2026 has 28 days.

  • D29

    Incorrect

    2026 is not a leap year, so February has 28 days.

pyspark.sql.functions.datediff(end, start) returns the number of days from start to end. With end = 2026-03-01 and start = 2026-02-01, the result is 28, the number of days in February 2026. Swapping the arguments would give -28.

Question 3 · choose 1

A timestamp column ts holds 2026-11-15 14:35:20, and pyspark.sql.functions is imported as F. What does F.date_trunc("month", "ts") return for it?

  1. A2026-11
  2. B2026-11-15 00:00:00
  3. C2026-11-30 23:59:59
  4. D2026-11-01 00:00:00
Show the answer and why
  • A2026-11

    Incorrect

    date_trunc returns a timestamp, not a year-month string.

  • B2026-11-15 00:00:00

    Incorrect

    That is truncation to the day; month truncation goes back to the first day of the month.

  • C2026-11-30 23:59:59

    Incorrect

    Truncation moves down to the start of the unit, not to its end.

  • D2026-11-01 00:00:00

    Correct

    date_trunc with 'month' truncates the timestamp to the start of its month.

date_trunc(format, timestamp) returns a timestamp truncated to the unit given by format, for example 'month'. The API reference shows 1997-02-28 05:02:11 truncated with 'mon' to 1997-02-01 00:00:00, so 2026-11-15 14:35:20 becomes 2026-11-01 00:00:00.

Question 4 · choose 1

A date column due holds 2026-02-25, and pyspark.sql.functions is imported as F. What does F.date_add("due", 7) return for it?

  1. A2026-03-02 (February taken as 30 days)
  2. B2026-02-18 (seven days earlier)
  3. C2026-03-04 (seven days later)
  4. D2026-02-32
Show the answer and why
  • A2026-03-02 (February taken as 30 days)

    Incorrect

    February 2026 has 28 days, so seven days after the 25th is March 4.

  • B2026-02-18 (seven days earlier)

    Incorrect

    A positive number of days moves forward; negative values are deducted.

  • C2026-03-04 (seven days later)

    Correct

    date_add returns the date that is the given number of days after start.

  • D2026-02-32

    Incorrect

    date_add returns a valid calendar date, rolling into the next month.

pyspark.sql.functions.date_add(start, days) returns the date that is days days after start; a negative value deducts days instead. Seven days after 2026-02-25, in a February with 28 days, is 2026-03-04.

Practice domain 3 →Practice all domains →