Task 2.6: Prioritize between Auto Loader, Lakeflow Connect (standard and managed connectors), partner connectors, and other ingestion methods based on technical requirements such as data volume, ingestion frequency, data types, and governance needs with Unity Catalog.

• AThe Lakeflow Connect MySQL database connector with an ingestion gateway

  Incorrect

  The managed MySQL database connector ingests with change data capture from the binary log, which the database team will not enable.

• BA Lakeflow Connect query-based connector using updated_at as the cursor column

  Correct

  Query-based connectors query the source directly and use a monotonically increasing cursor column to fetch new and changed rows, with no CDC setup.

• CAn Auto Loader stream pointed at the MySQL host

  Incorrect

  Auto Loader is a cloud object storage source; it discovers files, not database rows.

• DA COPY INTO statement that uses the MySQL JDBC URL as its source

  Incorrect

  COPY INTO loads files from a cloud storage location or volume, not tables from a database connection.

• EA nightly notebook job that reads rows by updated_at over JDBC and merges them

  Correct

  A JDBC read filtered on the cursor column, merged into the silver table and scheduled with Lakeflow Jobs, also works without CDC, with more code to maintain.

Without binary logs, CDC-based connectors are out. A query-based connector, or a custom JDBC read on the same kind of cursor column, can still pick up new and changed rows on a schedule.

• AA Lakehouse Federation connection to the HR system's API

  Incorrect

  Query federation reaches external relational databases over JDBC for read-only queries; it does not ingest from a REST API.

• BAn OpenSharing recipient that receives the HR data

  Incorrect

  OpenSharing shares data that a provider manages with other organizations; it does not pull data out of an internal API.

• CA Lakeflow Connect query-based connector on the HR API

  Incorrect

  Query-based connectors ingest from databases by querying the source database directly; an HTTP API is not a database they can query.

• DA custom connector for Lakeflow Connect, built in the workspace

  Correct

  Custom connectors let a team build its own connector for a source with no managed connector, and build, test, deploy and run it in its own workspace without registering it with the community.

Lakeflow Connect covers sources at several layers: managed connectors for popular applications and databases, connectors shared by the community, and custom connectors that a team builds and runs privately in its own workspace for a source nothing else covers.

• AUpload the .zip on the Create or modify a table using file upload page

  Incorrect

  The file upload page does not support compressed files such as zip and tar files.

• BExpand it with unzip in a %sh cell, then load the CSVs from the volume

  Correct

  The unzip Bash command, run with the %sh magic command, expands .zip files; the CSV files can then be read from the volume with the standard CSV options and written to the table.

• CExtract it on a laptop and upload the 40 CSVs on the file upload page

  Incorrect

  The file upload page accepts up to 10 files at a time, and each upload creates a new table or overwrites one rather than appending to it.

• DLoad the .zip from the volume with COPY INTO and FILEFORMAT = BINARYFILE

  Incorrect

  The binary file source turns each file into a single record of raw content and metadata, so the archive becomes one row, not CSV rows.

The file upload page rejects compressed files and caps an upload at 10 files. Databricks documents expanding .zip files with the unzip command from a %sh cell; once the extracted CSV files sit in a volume, they load into the table with the usual CSV reader options.

• AA Lakeflow Connect managed PostgreSQL connector

  Incorrect

  Managed connectors copy data into Databricks tables, which the requirement rules out.

• BA nightly notebook that reads the tables over JDBC into Delta

  Incorrect

  This copies data and needs code and a schedule.

• CLakehouse Federation with a foreign catalog for the database

  Correct

  Federation queries external sources in place, without moving data, and the managed connector FAQ points to it for ad hoc reporting and proofs of concept.

• DOpenSharing from the PostgreSQL database

  Incorrect

  OpenSharing shares data that a provider publishes; it does not query an operational PostgreSQL database.

Ingestion copies data for scale and performance; Lakehouse Federation leaves it in place, which suits ad hoc and proof-of-concept work.

• APartner Connect, starting from the partner's tile

  Correct

  Partner Connect provisions the SQL warehouse, a service principal and a personal access token for the partner, then hands off to the partner's trial sign-up.

• BA community connector registered in Lakeflow Connect

  Incorrect

  A community connector is connector code for a source; it does not create a warehouse, service principal and token for a partner's trial.

• CA custom connector built for Lakeflow Connect

  Incorrect

  A custom connector is a Lakeflow Connect connector for a source; it does not connect a third-party product or set up its warehouse and token.

• DAn OpenSharing recipient created for the partner

  Incorrect

  OpenSharing shares data from Databricks with other platforms; it does not set up a partner's ingestion tool.

Partner Connect is the shortcut to connect validated partner tools with new trial accounts. For an existing partner account, Databricks recommends connecting manually.