DP-750 Exam Dumps - Microsoft Certified: Azure Databricks Data Engineer Questions and Answers

Production ingestion: Job compute

BI: Serverless SQL warehouse

Job compute is designed for automated production workloads executed through Lakeflow Jobs. Its lifecycle can be tied to the job run, providing workload isolation and avoiding the cost of maintaining an interactive all-purpose cluster continuously. It is therefore appropriate for scheduled ingestion and transformation processing. A serverless SQL warehouse is designed for BI and Databricks SQL workloads. It provides rapid startup, automatic infrastructure management, scaling, and optimized SQL-query execution for dashboards and reporting tools. All-purpose compute is intended primarily for interactive notebook development and exploration, while shared compute does not provide the same job-specific isolation or SQL-serving experience. Consequently, job compute should support production ingestion, and a serverless SQL warehouse should serve the BI workload.

The correct answer is D. Breaking the pipeline into separate tasks for ingestion, cleansing, and curation is the foundation of well-designed Lakeflow Jobs pipelines. Each task should own one responsibility — when a task does too much, debugging a failure becomes a hunt through unrelated code, and retry logic becomes expensive because you re-execute work that already succeeded.

Contoso ' s planned changes explicitly call for ' a clear execution order and dependencies ' and ' orchestrate multi-step ingestion and transformation workflows. ' Separate tasks map directly to those goals: Lakeflow Jobs tracks each task ' s status independently, so if cleansing fails, ingestion doesn ' t re-run.

Option A bundles everything into one notebook, which means a curation bug forces a full re-ingestion. Option B copies logic three times — any future change must be applied in triplicate, which is a maintenance hazard. Option C forces everything through SQL MERGE, which is the wrong tool for raw-event ingestion and doesn ' t address cleansing or schema drift.

The solution requires spark.readStream with format( ' cloudFiles ' ) for Auto Loader, paired with .writeStream using mergeSchema=true and a checkpointLocation.

Auto Loader ' s cloudFiles source incrementally processes new JSON files without rescanning the entire directory. The mergeSchema option handles schema drift — when sensors add new fields, the target Delta table schema expands automatically instead of throwing a parse error. This directly addresses Contoso ' s requirement to ' support schema drift. '

The checkpointLocation is what gives the pipeline its resilience. Databricks writes the stream ' s committed offset and schema state to that path. If the cluster restarts, the engine reads the checkpoint and picks up exactly where it left off — no events are reprocessed, satisfying ' exactly-once semantics ' and ' resume processing after failures without reprocessing the data. '

Without a checkpoint, the stream would restart from the beginning on every cluster bounce, which is precisely the problem Contoso is trying to eliminate.

The correct mapping is SCD Type 1 for equipment metadata and SCD Type 2 for IoT sensor ownership history.

SCD Type 1 overwrites the existing record whenever an attribute changes — no history is kept. Contoso ' s requirement for equipment metadata (name, manufacturer, model, commissioning date) states ' historical values are NOT required, ' which is the textbook definition of Type 1. A MERGE INTO with WHEN MATCHED THEN UPDATE handles this cleanly in Delta Lake.

SCD Type 2 creates a new row for each change, preserving the full history through effective-date or version columns. Contoso requires that ' analysts must track the full history of ownership ' as sensors change hands over time — that full audit trail is only possible with Type 2. Type 3 (keeping just the previous value in an extra column) would lose earlier ownership records, so it doesn ' t satisfy the ' full history ' requirement.

The correct answer is A. Photon is Azure Databricks ' native vectorized query engine, written in C++, designed to accelerate data ingestion and SQL-heavy workloads significantly over the standard Spark JVM path. Enabling it on a job compute cluster directly addresses Contoso ' s requirement for ' fast and consistent performance for BI workloads ' and ' production ingestion workloads that can scale automatically during telemetry spikes. '

Photon integrates transparently — no code changes are needed — and pairs well with autoscaling job clusters to handle the bursty 40,000-sensor telemetry load.

Option B contradicts the isolation requirement: Contoso explicitly needs production and development separated, not merged onto shared compute. Option C with a fixed large node gives peak capacity at all times, driving up costs even during quiet periods. Option D disabling autoscaling is the opposite of what ' s needed — telemetry spikes require elastic scaling, not a locked node count.

Two configurations are needed:

Task dependency with ' All succeeded ' run condition: Set Task2 and Task3 to depend on Task1. Change the run condition on Task2 and Task3 to ' All succeeded ' — this means they only execute when all their upstream dependencies (Task1) have succeeded. If Task1 fails, both downstream tasks are skipped automatically, not run with failed inputs.

Repair run for recovery: Lakeflow Jobs ' Repair Run feature lets you re-execute only the tasks that failed (Task1 in this case) and their dependents (Task2 and Task3 if they were skipped), while skipping Task1 and any other tasks that already completed successfully. Successfully completed tasks are never re-executed during repair — their results are reused as-is.

Together these provide both controlled failure propagation (nothing runs downstream of a failure) and efficient recovery.

The correct answer is B — No.

Lakeflow Connect is an ingestion service that physically copies data from external databases into Delta tables managed by Databricks. It ' s designed for scenarios where you want a replicated, writable Delta copy of external data — essentially a CDC-based ingestion pipeline.

That ' s the opposite of what ' s required here. The requirement states ' the data is NOT copied into Databricks-managed storage. ' Lakeflow Connect would create Delta tables in Databricks and copy DB1 ' s data into them — a direct violation.

Additionally, Lakeflow Connect creates Databricks-native Delta tables rather than exposing DB1 ' s original schemas and tables as virtual objects. Analysts querying through a Lakeflow Connect pipeline are querying a replicated copy, not the live source.

For zero-copy, live query federation of an external SQL Server into Unity Catalog, Lakehouse Federation (foreign catalog) is the correct tool.

The Finance catalog: USE CATALOG

The Purchases schema: USE SCHEMA

The Transactions table: SELECT

Two things are needed here:

For the rule enforcement: use @dlt.expect_or_drop. This drops any record that violates rule1 before it reaches Table1, while the pipeline continues processing all valid records. The table only ever receives clean data.

For reviewing metrics: the Lakeflow SDP Pipeline UI is the right tool — zero development effort required. The pipeline graph shows expectation pass/fail counts directly on each table node, and the event log provides a detailed per-batch breakdown of how many records were dropped and why. Data engineers can inspect this at any time without writing additional monitoring queries or connecting external dashboards.

This combination is one of the strongest arguments for SDP over hand-coded Structured Streaming: expectation observability is built in, not bolted on.

The correct answer is D — configure automatic termination.

The problem is specific: clusters sit idle after users finish working but nobody manually shuts them down. Automatic termination solves this directly — once a cluster has been idle for the configured period (no running commands, no attached notebooks with active execution), it shuts itself down. You eliminate the idle cost without any manual intervention and without affecting any workload that is actually running.

Option A (convert to job clusters) would force users off interactive all-purpose clusters, disrupting their development workflow. Option B (spot instances) reduces the hourly rate while a cluster is running but does nothing about the idle-time problem — a cheaper idle cluster is still waste. Option C (enable autoscaling) reduces the number of workers during light load but keeps the cluster alive at the minimum node count. It saves some cost but doesn ' t fully eliminate idle spend the way auto-termination does.