Task 1.3: Designing for flexibility and portability.

• AApache Iceberg managed tables in BigQuery, stored in the company's bucket

  Correct

  Iceberg managed tables keep data in a bucket the customer owns in the open Iceberg format, support GoogleSQL DML and Storage Write API streaming, and get automatic file sizing and clustering, while other engines read the same copy.

• BBigQuery standard tables, exported to Parquet files for the Spark jobs

  Incorrect

  Standard tables keep data in BigQuery storage, so Spark would read an exported second copy that is stale between exports.

• CBigLake tables over the Parquet files that the Spark jobs write to Cloud Storage

  Incorrect

  BigLake tables over Cloud Storage files are read through a connection for querying and governance; they do not give BigQuery DML and streaming writes on that data.

• DObject tables over the event files in the company's bucket

  Incorrect

  Object tables are read-only tables over unstructured objects, giving a metadata index of the files; they accept no DML and no streaming writes.

Open formats avoid locking the data to one engine. Apache Iceberg managed tables give a BigQuery-managed experience (DML, streaming, automatic storage optimization, time travel) while the data files live in the customer's own Cloud Storage bucket in Iceberg format, so Spark and other engines can work on the same single copy.

• AA BigQuery Data Transfer Service transfer from Amazon S3 into a dataset

  Incorrect

  The Amazon S3 transfer loads the files into BigQuery storage in Google Cloud, so the raw data leaves AWS.

• BStorage Transfer Service jobs that copy the files to a Cloud Storage bucket

  Incorrect

  Storage Transfer Service copies the objects into Cloud Storage, which moves the raw data out of AWS and adds transfer jobs.

• CA Dataflow pipeline that reads the S3 objects and writes them to BigQuery

  Incorrect

  A pipeline that writes to BigQuery moves the data into Google Cloud and is a transfer job to build and run.

• DBigQuery Omni, with a connection to AWS and BigLake tables over the S3 data

  Correct

  BigQuery Omni runs the query engine in the AWS region where the data sits, through a connection and BigLake tables, and can write results directly back to S3.

When data has to stay in another cloud, BigQuery Omni brings the query to the data: BigQuery compute runs in the same AWS or Azure region, the data is reached through a connection and BigLake tables, and results can be written back to the other cloud's storage so that nothing crosses clouds.

• AWrite the pipelines in BigQuery SQL scheduled with Dataform

  Incorrect

  GoogleSQL and Dataform workflows run in BigQuery; they cannot run on Flink or Spark without a rewrite.

• BWrite the pipelines with Apache Beam and run them on Dataflow

  Correct

  Beam is open source and supports several runners, including Dataflow, Flink and Spark, so the same code can move without a rewrite.

• CBuild the pipelines in Cloud Data Fusion's visual designer

  Incorrect

  Data Fusion pipelines are CDAP pipelines run by Data Fusion; they are not code that runs on Flink.

• DWrite the pipelines as Workflows definitions in YAML

  Incorrect

  Workflows orchestrates service calls in its own syntax; it is not a data processing model that Flink or Spark can run.

Dataflow runs Apache Beam pipelines. Beam has one programming model for batch and streaming and several runners, so pipeline code written for Dataflow can be moved to another runner, such as Flink or Spark, without rewriting it, which keeps the processing logic portable.

• AA Sensitive Data Protection discovery scan of the table's project

  Incorrect

  Discovery profiles data for sensitive information types and risk, not general column statistics.

• BAn auto data quality scan with a not-null rule on each column

  Incorrect

  Data quality scans check rules you define and report pass or fail; they do not describe the distribution of every column.

• CQueries against INFORMATION_SCHEMA.COLUMNS for the table

  Incorrect

  INFORMATION_SCHEMA.COLUMNS describes column metadata such as names and types, not the values in the data.

• DA Knowledge Catalog data profiling scan published to the table

  Correct

  A data profiling scan computes null percentage, approximate distinct percentage, top values and statistics per column, and can publish the results to the table's Data profile tab.

Data profiling in Knowledge Catalog describes what is in a table: per column it reports nulls, approximate distinct values, the top ten values and, for numbers and strings, statistics such as minimum, maximum and average. Its results can also seed recommended rules for auto data quality.

• AIn a column of the JSON data type, queried with field access

  Correct

  The JSON type needs no schema up front, and BigQuery encodes and processes each JSON field individually, so fields can come and go while queries use dot notation.

• BIn a STRUCT column whose fields mirror the current payload

  Incorrect

  A STRUCT has a fixed schema enforced for all values, so every weekly change would need a schema update.

• CIn a STRING column, parsed with JSON functions in each query

  Incorrect

  A string works, but every query must parse the text first, which gives up the per-field processing of the JSON type.

• DIn a separate table for each event version, joined by a view

  Incorrect

  A table per version multiplies tables and views with every change, the opposite of avoiding schema work.

For semi-structured data whose shape keeps changing, the JSON data type gives flexibility without giving up queryability: no schema is needed at load, and BigQuery stores each field so that queries can reach it directly. STRUCT is better when the structure is stable and should be enforced.

• ASchedule a weekly query that drops tables older than three days

  Incorrect

  A cleanup job works but is extra code to maintain and leaves tables in place for up to a week.

• BUse a lifecycle rule on the dataset to delete tables after three days

  Incorrect

  Lifecycle rules are a Cloud Storage feature for objects in buckets; BigQuery datasets use expiration settings.

• CSet a default table expiration of three days on the staging dataset

  Correct

  A dataset's default table expiration applies to each table created in it without its own expiration, and an expired table is deleted with its data.

• DSet a three-day time travel window on the staging dataset

  Incorrect

  The time travel window controls how far back past states can be read; it does not delete current tables.

Staging data is a classic use of expiration. A default table expiration on the dataset is applied to new tables that do not set their own, and BigQuery deletes each table once it expires, so the dataset cleans itself.