A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Events are recorded once per minute per device.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

• A. to_interval("event_time", "5 minutes").alias("time")
• B. window("event_time", "10 minutes").alias("time")
• C. "event_time"
• D. lag("event_time", "10 minutes").alias("time")
• E. window("event_time", "5 minutes").alias("time")

A data ingestion task requires a one-TB JSON dataset to be written out to Parquet with a target part-file size of 512 MB. Because Parquet is being used instead of Delta Lake, built-in file-sizing features such as Auto-Optimize & Auto-Compaction cannot be used.
Which strategy will yield the best performance without shuffling data?

• A. Set spark.sql.adaptive.advisoryPartitionSizeInBytes to 512 MB bytes, ingest the data, execute the narrow transformations, coalesce to 2,048 partitions (1TB*1024*1024/512), and then write to parquet.
• B. Set spark.sql.shuffle.partitions to 2,048 partitions (1TB*1024*1024/512), ingest the data, execute the narrow transformations, optimize the data by sorting it (which automatically repartitions the data), and then write to parquet.
• C. Set spark.sql.files.maxPartitionBytes to 512 MB, ingest the data, execute the narrow transformations, and then write to parquet.
• D. Set spark.sql.shuffle.partitions to 512, ingest the data, execute the narrow transformations, and then write to parquet.
• E. Ingest the data, execute the narrow transformations, repartition to 2,048 partitions (1TB*
  1024*1024/512), and then write to parquet.

A data engineer is implementing Unity Catalog governance for a multi-team environment. Data scientists need interactive clusters for basic data exploration tasks, while automated ETL jobs require dedicated processing. How should the data engineer configure cluster isolation policies to enforce least privilege and ensure Unity Catalog compliance?

• A. Create compute policies with STANDARD access mode for interactive workloads and DEDICATED access mode for automated jobs.
• B. Allow all users to create any cluster type and rely on manual configuration to enable Unity Catalog access modes.
• C. Configure all clusters with NO ISOLATION_SHARED access mode since Unity Catalog works with any cluster configuration.
• D. Use only DEDICATED access mode for both interactive workloads and automated jobs to maximize security isolation.

The data engineer team has been tasked with configured connections to an external database that does not have a supported native connector with Databricks. The external database already has data security configured by group membership. These groups map directly to user group already created in Databricks that represent various teams within the company. A new login credential has been created for each group in the external database. The Databricks Utilities Secrets module will be used to make these credentials available to Databricks users. Assuming that all the credentials are configured correctly on the external database and group membership is properly configured on Databricks, which statement describes how teams can be granted the minimum necessary access to using these credentials?

• A. "Read" permissions should be set on a secret scope containing only those credentials that will be used by a given team.
• B. "Read'' permissions should be set on a secret key mapped to those credentials that will be used by a given team.
• C. No additional configuration is necessary as long as all users are configured as administrators in the workspace where secrets have been added.
• D. "Manage" permission should be set on a secret scope containing only those credentials that will be used by a given team.

A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.

Which approach would allow using DLT expectations to validate all expected records are present in this table?

• A. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table
• B. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
• C. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null
• D. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table