Associate Developer for Apache Spark · Domain 3: Developing Apache Spark DataFrame/DataSet API Applications · 30% of the exam
Task 3.3: Manipulate and utilize Date data type, such as Unix epoch to date string, and extract date component
from_unixtime, to_date, date_format patterns, the session time zone, and extracting year, month and day of week.
Study it
Dates, timestamps and Unix epoch values
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
The epoch value 1700000000 is 2023-11-14 22:13:20 UTC. A developer runs:
from pyspark.sql import functions as F
spark.conf.set("spark.sql.session.timeZone", "Asia/Tokyo")
df = spark.createDataFrame([(1700000000,)], "ts BIGINT")
df.select(F.from_unixtime("ts", "yyyy-MM-dd").alias("day")).show()
What does the column day hold?
A2023-11-14, the UTC date
B2023-11-15 07:13:20, the Tokyo time to the second
C2023-11-14 22:13:20, the UTC time to the second
D2023-11-15, the Tokyo date
Show the answer and why
A2023-11-14, the UTC date
Incorrect
from_unixtime renders the moment in the session's time zone, here Asia/Tokyo, nine hours ahead of UTC.
B2023-11-15 07:13:20, the Tokyo time to the second
Incorrect
The format argument yyyy-MM-dd replaces the default pattern, so no time of day is printed.
C2023-11-14 22:13:20, the UTC time to the second
Incorrect
This is the UTC moment in the default pattern, but the session time zone and the given format both apply.
D2023-11-15, the Tokyo date
Correct
The moment is rendered in the session time zone, Asia/Tokyo (07:13:20 on 15 November), with the yyyy-MM-dd pattern.
from_unixtime converts seconds since the Unix epoch to a string representing that moment in the current time zone, using the given format (default yyyy-MM-dd HH:mm:ss). The function's own example shows which zone that is: it sets spark.sql.session.timeZone (the session local time zone) to America/Los_Angeles and renders 1428476400, which is 07:00 UTC on 8 April 2015, as 2015-04-08 00:00:00, the Los Angeles time. Likewise 22:13:20 UTC on 14 November is 07:13:20 on 15 November in Tokyo, and the pattern yyyy-MM-dd keeps only the date.
A developer runs:
from pyspark.sql import functions as F
df = spark.createDataFrame([("2026-02-01", "2026-03-01")], "start STRING, finish STRING")
df.select(F.datediff(F.to_date("finish"), F.to_date("start")).alias("d")).show()
What does d hold?
A-28
B1
C28
D29
Show the answer and why
A-28
Incorrect
datediff(end, start) counts the days from start to end, and finish is the later date here.
B1
Incorrect
datediff returns a number of days, not of months.
C28
Correct
datediff(end, start) returns the number of days from start to end: February 2026 has 28 days.
D29
Incorrect
2026 is not a leap year, so February has 28 days.
pyspark.sql.functions.datediff(end, start) returns the number of days from start to end. With end = 2026-03-01 and start = 2026-02-01, the result is 28, the number of days in February 2026. Swapping the arguments would give -28.
A timestamp column ts holds 2026-11-15 14:35:20, and pyspark.sql.functions is imported as F. What does F.date_trunc("month", "ts") return for it?
A2026-11
B2026-11-15 00:00:00
C2026-11-30 23:59:59
D2026-11-01 00:00:00
Show the answer and why
A2026-11
Incorrect
date_trunc returns a timestamp, not a year-month string.
B2026-11-15 00:00:00
Incorrect
That is truncation to the day; month truncation goes back to the first day of the month.
C2026-11-30 23:59:59
Incorrect
Truncation moves down to the start of the unit, not to its end.
D2026-11-01 00:00:00
Correct
date_trunc with 'month' truncates the timestamp to the start of its month.
date_trunc(format, timestamp) returns a timestamp truncated to the unit given by format, for example 'month'. The API reference shows 1997-02-28 05:02:11 truncated with 'mon' to 1997-02-01 00:00:00, so 2026-11-15 14:35:20 becomes 2026-11-01 00:00:00.
A date column due holds 2026-02-25, and pyspark.sql.functions is imported as F. What does F.date_add("due", 7) return for it?
A2026-03-02 (February taken as 30 days)
B2026-02-18 (seven days earlier)
C2026-03-04 (seven days later)
D2026-02-32
Show the answer and why
A2026-03-02 (February taken as 30 days)
Incorrect
February 2026 has 28 days, so seven days after the 25th is March 4.
B2026-02-18 (seven days earlier)
Incorrect
A positive number of days moves forward; negative values are deducted.
C2026-03-04 (seven days later)
Correct
date_add returns the date that is the given number of days after start.
D2026-02-32
Incorrect
date_add returns a valid calendar date, rolling into the next month.
pyspark.sql.functions.date_add(start, days) returns the date that is days days after start; a negative value deducts days instead. Seven days after 2026-02-25, in a February with 28 days, is 2026-03-04.