Professional Data Engineer · Domain 3: Storing the data · 20% of the exam
Task 3.1: Selecting storage systems.
Matching access patterns to BigQuery, Lakehouse, AlloyDB, Bigtable, Spanner, Cloud SQL, Cloud Storage, Firestore and Memorystore, planning storage cost and performance, and managing the data lifecycle.
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 1
A fleet company receives readings from 2 million vehicles, about 500,000 writes per second. The fleet app must fetch one vehicle's readings for a time range in under 10 ms. The workload needs no joins or multi-row transactions, analysts run their reports elsewhere, and the company wants the lowest cost at this write rate. Which store should serve the app?
AA BigQuery table partitioned by day and clustered by vehicle ID
BSpanner, with a table keyed on vehicle ID and reading timestamp
CBigtable, with row keys that start with the vehicle ID and then the time
DFirestore, with one document for each reading in a vehicle collection
Show the answer and why
AA BigQuery table partitioned by day and clustered by vehicle ID
Incorrect
BigQuery is built for analytical scans, not for single-digit-millisecond lookups behind an application at this rate.
BSpanner, with a table keyed on vehicle ID and reading timestamp
Incorrect
Spanner could hold the data, but it pays for relational features and transactions the workload does not need, which makes it the costlier fit.
CBigtable, with row keys that start with the vehicle ID and then the time
Correct
Bigtable is designed for very high write throughput and low-latency key and range reads; a row key of vehicle ID then timestamp keeps one vehicle's readings together and spreads writes across vehicles.
DFirestore, with one document for each reading in a vehicle collection
Incorrect
Firestore is a document database for app data with real-time sync; it is not the fit for half a million time-series writes per second.
Time-series data at very high write rates with key-range reads is the classic Bigtable workload. The row key design matters as much as the choice of service: leading with the device identifier and following with the time groups each device's data for range scans while avoiding a hotspot on the latest timestamp.
A log analytics dataset in BigQuery holds 900 TiB of logical bytes that take only 120 TiB of physical storage, because the data compresses well. Its tables are append-only, so little data is kept for time travel. Finance wants a smaller storage bill without changing queries or how long data is kept. What should the data engineer do?
AShorten the dataset's time travel window from seven days to two days
BSwitch the dataset to the physical storage billing model
CExport the older partitions to Coldline and query them as external tables
DSet a partition expiration so that old partitions are removed sooner
Show the answer and why
AShorten the dataset's time travel window from seven days to two days
Incorrect
Under the default logical billing model, time travel and fail-safe bytes are not charged, so shortening the window saves nothing here.
BSwitch the dataset to the physical storage billing model
Correct
Physical billing charges compressed bytes. The physical rate is under twice the logical rate, so with a 7.5-to-1 compression ratio the bill drops sharply; time travel bytes, which are few here, are added.
CExport the older partitions to Coldline and query them as external tables
Incorrect
Moving data out of BigQuery storage changes how it is queried and adds external-table limits, which the requirement rules out.
DSet a partition expiration so that old partitions are removed sooner
Incorrect
Partition expiration deletes data, which changes how long it is kept.
BigQuery bills storage on logical (uncompressed) bytes by default, or on physical (compressed) bytes if a dataset opts in. The physical rate per GiB is higher, but well-compressing data more than makes up for it. Under physical billing, time travel and fail-safe bytes are charged separately, so append-only data with little churn benefits most. The model can be changed again after 14 days.
A bank must keep scanned loan documents for 10 years. A file is read only when an auditor asks for it, on average less than once a year per file, and an auditor may need a file within minutes of asking. Which Cloud Storage class keeps the cost lowest?
ANearline storage class
BColdline storage class
CStandard storage class with Autoclass on
DArchive storage class
Show the answer and why
ANearline storage class
Incorrect
Nearline suits data read about once a month; for data read less than once a year its storage price is higher than needed.
BColdline storage class
Incorrect
Coldline suits data read at most once a quarter; Archive is cheaper to store for data read less than once a year.
CStandard storage class with Autoclass on
Incorrect
Starting files in Standard costs more while they wait to move to colder classes; for this known, rare pattern Archive can be chosen at once.
DArchive storage class
Correct
Archive is for data accessed less than once a year, and unlike tape-style archives its data is available within milliseconds, so the minutes requirement is met.
Cloud Storage classes trade storage price against access cost and minimum storage duration: Nearline 30 days, Coldline 90 days and Archive 365 days. All of them return data within milliseconds, so a need to read a file within minutes does not rule out Archive.
A trading platform stores account balances in a relational schema and accepts orders from users in North America, Europe and Asia. Reads and writes must be strongly consistent everywhere, the database must keep serving if a whole region is lost, with no committed data lost, and the target is 99.999% availability. Which database fits?
ACloud SQL with cross-region read replicas in the other continents
BBigtable with replication across clusters in three continents
CSpanner with a multi-region instance configuration
DAlloyDB with a secondary cluster in another region
Show the answer and why
ACloud SQL with cross-region read replicas in the other continents
Incorrect
Cloud SQL replicas are read-only and asynchronous; writes still go to one primary region, and a regional loss needs a promotion that can lose recent writes.
BBigtable with replication across clusters in three continents
Incorrect
Bigtable replication is eventually consistent across clusters and it has no relational schema or multi-row transactions.
CSpanner with a multi-region instance configuration
Correct
Multi-region Spanner replicates synchronously across regions with external consistency, keeps serving through a region loss and is designed for 99.999% availability.
DAlloyDB with a secondary cluster in another region
Incorrect
AlloyDB's cross-region replication is asynchronous to a secondary cluster that must be promoted, so it is not strongly consistent across regions.
Spanner combines relational semantics with horizontal scale and synchronous replication. Its multi-region configurations place replicas in several regions, serve strongly consistent reads and writes, survive the loss of a region and carry the highest availability target, which no single-primary database can match.
A field-service mobile app stores each technician's job lists and notes. The app must keep working without a connection, sync changes when the device is back online, and show updates made by the office in real time. Which database should back the app?
AFirestore, with its client libraries' offline support and listeners
BMemorystore for Redis, with each job list stored as a hash
CCloud SQL for PostgreSQL, behind an API on Cloud Run
DBigtable, with a row key that starts with the technician ID
Show the answer and why
AFirestore, with its client libraries' offline support and listeners
Correct
Firestore is a document database for mobile and web apps with offline persistence and real-time listeners that sync changes when devices reconnect.
BMemorystore for Redis, with each job list stored as a hash
Incorrect
Memorystore is an in-memory cache reached from the network; it has no offline client sync.
CCloud SQL for PostgreSQL, behind an API on Cloud Run
Incorrect
A relational database behind an API would need custom offline caching and change notifications built into the app.
DBigtable, with a row key that starts with the technician ID
Incorrect
Bigtable serves high-throughput key-value workloads from servers; it has no mobile offline sync or listeners.
Choosing storage starts from the access pattern. Mobile clients that write offline and need live updates are Firestore's core case: its SDKs cache data on the device, queue writes while offline and push real-time changes to listeners.
A media bucket holds 800 TB of clips. Some clips are untouched for months and then suddenly used heavily again; nobody can predict which. The team wants storage costs to follow actual access per object, without retrieval fees when an old clip becomes popular and without maintaining rules. What should the data engineer configure?
AA lifecycle rule that moves objects to Nearline after 30 days
BColdline as the bucket's default storage class
CStandard storage, with Object Versioning to protect the clips
DAutoclass on the bucket, managing the class of each object
Show the answer and why
AA lifecycle rule that moves objects to Nearline after 30 days
Incorrect
A fixed age rule moves clips that are still popular and charges retrieval fees each time a moved clip is read again.
BColdline as the bucket's default storage class
Incorrect
Coldline adds retrieval fees and a 90-day minimum to clips that are heavily used.
CStandard storage, with Object Versioning to protect the clips
Incorrect
Versioning protects against overwrites; it does nothing to match cost to access.
DAutoclass on the bucket, managing the class of each object
Correct
Autoclass moves each object between classes based on its own access and returns it to Standard when it is read, without retrieval fees or early-deletion charges.
Lifecycle rules suit predictable aging. When access is unpredictable per object, Autoclass transitions objects automatically: data that is not accessed moves to colder classes, and data that is accessed moves back to Standard, with no retrieval or early-deletion fees.
A booking application uses MySQL 8.0 with a 500 GB database, serves users in one country, and is being moved to Google Cloud. The team wants managed backups, patching and high availability, and it must not change the application's SQL or drivers. Which service fits?
ASpanner with the GoogleSQL dialect
BCloud SQL for MySQL with high availability
CAlloyDB for PostgreSQL
DBigtable with an HBase-compatible client
Show the answer and why
ASpanner with the GoogleSQL dialect
Incorrect
Spanner is built for horizontal scale and global consistency, and moving a MySQL application to it means schema and query changes.
BCloud SQL for MySQL with high availability
Correct
Cloud SQL runs MySQL as a managed service with backups, maintenance and high availability, so the application keeps its SQL and drivers.
CAlloyDB for PostgreSQL
Incorrect
AlloyDB is PostgreSQL-compatible, so a MySQL application would need its SQL and drivers changed.
DBigtable with an HBase-compatible client
Incorrect
Bigtable is a NoSQL wide-column store with no SQL engine for the application's queries.
A lift-and-shift of a regional MySQL database with no application changes is Cloud SQL's core case. Spanner and AlloyDB serve other needs (global scale, PostgreSQL with analytics), and both would require changes.