Professional Data Engineer · Domain 3: Storing the data · 20% of the exam
Task 3.4: Designing for a data platform.
Building a data platform from Knowledge Catalog, BigQuery and Cloud Storage, and a federated governance model for distributed data owned by domain teams.
Study it
A data platform on Knowledge Catalog: data mesh, data products and federated governance
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Google Cloud documentation page that proves it.
Question 1 · choose 1
A conglomerate has five business units, each with its own data team and Google Cloud projects. Central IT wants each unit to keep owning and publishing its own data, consumers anywhere in the company to find the published data in one place, and governance policies to be defined centrally. Which architecture fits?
AConsolidate the units' data into one central project that IT runs
BGive each unit an isolated environment and share data as exported files
CCopy the units' tables nightly into a shared enterprise warehouse
DA data mesh in which units publish data products into Knowledge Catalog
Show the answer and why
AConsolidate the units' data into one central project that IT runs
Incorrect
Centralizing moves ownership away from the teams that understand the data and makes the central team a bottleneck.
BGive each unit an isolated environment and share data as exported files
Incorrect
Isolated units with file exports give no shared catalog for discovery and no common governance.
CCopy the units' tables nightly into a shared enterprise warehouse
Incorrect
Nightly copies take ownership out of the units and leave consumers on stale duplicates.
DA data mesh in which units publish data products into Knowledge Catalog
Correct
In a data mesh, domain teams publish data products, central services use Knowledge Catalog as the catalog and discovery mechanism, and a governance team defines policies.
A data mesh keeps data with the domains that produce it and adds shared services on top: a self-service platform, a central catalog for discovery, and centrally defined governance that domain stewards apply. On Google Cloud, Knowledge Catalog provides the catalog and data products.
A governance council wants every table in the catalog, from BigQuery and from Cloud Storage, to carry the same structured metadata: a data owner, a sensitivity level chosen from four values, and a retention period. The sensitivity field must be mandatory, domain stewards will fill the values in, and the metadata must be searchable. What should the platform team create in Knowledge Catalog?
AAn aspect type with the three fields, making sensitivity a required enum
BA business glossary with one term for each of the four sensitivity levels
CA policy tag taxonomy with one policy tag for each of the four levels
DTable descriptions written to a template that lists the three fields
Show the answer and why
AAn aspect type with the three fields, making sensitivity a required enum
Correct
Aspect types are reusable templates of typed fields, including enums, whose fields can be marked required; aspects of that type then attach structured, searchable metadata to entries.
BA business glossary with one term for each of the four sensitivity levels
Incorrect
Glossary terms build a shared business vocabulary; they do not define a structured record with an owner, a mandatory field and a retention value.
CA policy tag taxonomy with one policy tag for each of the four levels
Incorrect
Policy tags control who can read columns; they are an access mechanism, not a metadata template for owner and retention.
DTable descriptions written to a template that lists the three fields
Incorrect
Descriptions are free text, so nothing makes a field mandatory or keeps the values consistent and structured.
In Knowledge Catalog, aspect types define the structure of metadata (field names, types such as enums, and whether a field is required) and aspects are the filled-in instances attached to entries or columns. They are the tool for consistent, governed, searchable metadata; glossaries and descriptions serve other purposes.
A company's catalog in Knowledge Catalog already covers its BigQuery and Cloud Storage data automatically. The platform team must add the tables of an on-premises Oracle database and a Snowflake account, so that users can search them in the same catalog alongside the Google Cloud assets. What should the team set up?
AA nightly export of the sources' schemas into a BigQuery table
BA managed connectivity pipeline that imports their metadata as entries
CBigQuery dataset descriptions that list the Oracle and Snowflake tables
DDatastream streams that replicate both sources into BigQuery
Show the answer and why
AA nightly export of the sources' schemas into a BigQuery table
Incorrect
A table of schemas is data in BigQuery, not catalog entries that users can search and enrich alongside the other assets.
BA managed connectivity pipeline that imports their metadata as entries
Correct
Managed connectivity pipelines run a connector for third-party sources such as Oracle and Snowflake and import the metadata into Knowledge Catalog as entries and aspects.
CBigQuery dataset descriptions that list the Oracle and Snowflake tables
Incorrect
A description is free text on a BigQuery dataset; it creates no catalog entries for the outside tables and no metadata to search or enrich.
DDatastream streams that replicate both sources into BigQuery
Incorrect
Replicating the data catalogs copies of it; the request is to catalog the sources where they are.
Knowledge Catalog harvests Google Cloud metadata on its own. For other systems, a managed connectivity pipeline runs a connector image on Managed Service for Apache Spark, orchestrated by Workflows, and imports the result as entries and aspects, so third-party assets are searchable in the same catalog.
A domain team plans to change the meaning of a column in a widely used BigQuery table. Before it does, it wants to see which tables and views, owned by any team, are built from that table by BigQuery jobs, and from which columns. What should it use?
AA search of INFORMATION_SCHEMA.JOBS for queries naming the table
BData Access audit logs exported to BigQuery and grepped
CA data profiling scan on the table to see its columns
DData lineage in Knowledge Catalog for the table
Show the answer and why
AA search of INFORMATION_SCHEMA.JOBS for queries naming the table
Incorrect
Searching job text is manual and incomplete, and it does not map the columns that flow into each table.
BData Access audit logs exported to BigQuery and grepped
Incorrect
Audit logs record who accessed what, not a graph of how data flows between tables and columns.
CA data profiling scan on the table to see its columns
Incorrect
Profiling describes the values in the table itself; it says nothing about what is built from it.
DData lineage in Knowledge Catalog for the table
Correct
Lineage records how data moves between assets, including column-level lineage for BigQuery jobs, so downstream impact can be traced.
Data lineage in Knowledge Catalog tracks the processes that read and write assets, such as BigQuery jobs, and shows upstream and downstream links down to columns. Impact analysis before a change is one of its main uses in a data platform with many owners.
A data platform spans 400 BigQuery datasets in 60 projects. Analysts must read every dataset that domain teams tag as non-sensitive, including datasets created later, and none of the others, even when both kinds sit in the same project. What should the platform team do?
AGrant Data Viewer on each non-sensitive dataset, one by one
BGrant Data Viewer on the folder that contains the 60 projects
CAttach a tag to the datasets and grant Data Viewer with a tag condition
DAuthorize the non-sensitive datasets on the analysts' own dataset
Show the answer and why
AGrant Data Viewer on each non-sensitive dataset, one by one
Incorrect
Per-dataset grants work but must be repeated for each new dataset, which is the gap the team wants to close.
BGrant Data Viewer on the folder that contains the 60 projects
Incorrect
A folder grant covers every dataset beneath it, sensitive ones included.
CAttach a tag to the datasets and grant Data Viewer with a tag condition
Correct
IAM Conditions on tags grant access to every resource carrying the tag value, including resources that do not exist yet.
DAuthorize the non-sensitive datasets on the analysts' own dataset
Incorrect
Authorized datasets let views in one dataset read another; they are not a way to grant users read access across many datasets.
Tags from Resource Manager can be attached to BigQuery datasets, tables and views, and IAM role grants can carry conditions on those tags. One conditional grant then covers every tagged resource, now and in the future, which scales governance better than resource-by-resource grants.
A company is building a data mesh. Domain teams wait weeks for the central team to create their projects, datasets and access groups, and each domain has wired up its own monitoring, so nobody can see across domains whether data products are healthy. Which TWO changes fit the platform team's role in a data mesh? (Choose TWO.)
AKeep the ticket queue and add more central engineers to process it
BGive each domain team the Owner role on the organization
CPublish reusable infrastructure code that domains run themselves
DLet each domain choose and run its own monitoring stack
EProvide shared tooling for data product observability
Show the answer and why
AKeep the ticket queue and add more central engineers to process it
Incorrect
A larger queue is still a central bottleneck, which a data mesh is meant to remove.
BGive each domain team the Owner role on the organization
Incorrect
Organization-wide owner access gives up the shared guardrails and governance that the mesh depends on.
CPublish reusable infrastructure code that domains run themselves
Correct
A self-service data platform gives producers templates and artifacts so they can create standard resources without waiting for a central team.
DLet each domain choose and run its own monitoring stack
Incorrect
Separate stacks keep the view split by domain; the platform is meant to integrate with operations tooling for global observability.
EProvide shared tooling for data product observability
Correct
The platform team provides the tooling for data product observability that producers and consumers use, integrated with operations tooling for a global view.
In Google's data mesh architecture, the data platform team provides a self-service platform: reusable infrastructure such as Terraform and pipeline templates that producers run on their own, and shared tooling for data product observability, integrated with operations tooling for global observability and compliance automation. Domains own their data products; the platform supplies what they all build on.