The Databricks Certified Data Engineer Associate Exam (Databricks-Certified-Data-Engineer-Associate)

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as commit, pull, push, branch management, and visual comparison of diffs when committing1. However, merge is not supported in the Git dialog2. You need to use the Repos UI or your Git provider to merge branches3. Merge is a way to combine the commit history from one branch into another branch1. During a merge, a merge conflict is encountered when Git cannot automatically combine code from one branch into another. Merge conflicts require manual resolution before a merge can be completed1. References: 4: Run Git operations on Databricks Repos4, 1: CI/CD techniques with Git and Databricks Repos1, 3: Collaborate in Repos3, 2: Databricks Repos - What it is and how we can use it2.

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as commit, pull, push, merge, and branch management. However, to clone a remote Git repository to a Databricks repo, you must use the Databricks UI or API. You cannot clone a Git repo using the CLI through a cluster’s web terminal, as the files won’t display in the Databricks UI1. References: 1: Run Git operations on Databricks Repos | Databricks on AWS2

Unity Catalog organizes data objects through a three-level namespace consisting of catalog, schema, and object. To query sales_catalog.market_data.summary, the analyst requires USE CATALOG on sales_catalog, USE SCHEMA on market_data, and SELECT on the summary view. Granting SELECT on the view alone is insufficient when either parent usage privilege is missing. The analyst does not normally require access to the underlying tables queried by the view because the view owner’s privileges are used to resolve those objects. A workspace or cluster is not part of the Unity Catalog securable hierarchy for resolving the view name. Storage credentials and external locations govern cloud-storage access rather than ordinary view-query permissions. Therefore, the engineer should verify privileges in the order catalog, schema, and view, making option B correct.

================

The DELETE command in Delta Lake allows you to remove data that matches a predicate from a Delta table. This command will delete all the rows where the value in the column age is greater than 25 from the existing Delta table my_table and save the updated table. The other options are either incorrect or do not achieve the desired result. Option A will only select the rows that match the predicate, but not delete them. Option B will update the rows that match the predicate, but not delete them. Option D will update the rows that do not match the predicate, but not delete them. Option E will delete the rows that do not match the predicate, which is the opposite of what we want. References: Table deletes, updates, and merges — Delta Lake Documentation

Unity Catalog attribute-based access control policies provide centralized, tag-driven governance. Administrators define policies at a catalog or schema scope and use governed tags to identify the tables or columns to which the policies apply. When matching tags are assigned to current or future objects, the applicable policy is evaluated automatically, avoiding repetitive manual attachment of filters and masks to individual tables. This approach improves scalability and consistency when many objects require the same protection rules. It is important that the policy applies to objects matching its tag conditions, rather than indiscriminately to every object. The other choices incorrectly describe supported object types, multiple-filter behavior, or implementation languages. Databricks specifically positions attribute-based access control as the centralized approach for consistently applying row filtering and column masking across many governed tables. Therefore, option A is correct.

================

The < = > operator performs null-safe equality in Databricks SQL. It returns TRUE when both operands contain the same non-NULL value and also when both operands are NULL. It returns FALSE when only one operand is NULL. By contrast, the standard equality operator returns UNKNOWN when either side is NULL, so it cannot match two NULL values in a WHERE condition. Option C implements inequality logic and does not provide null-safe equality. Option D can incorrectly treat a genuine empty string as equivalent to NULL, changing the meaning of the data. Therefore, option A is the only predicate that preserves SQL values while implementing the required NULL comparison behavior. This question was also checked against the existing Word question bank.

================

Option C extracts each JSON field into a separate Silver-table column and casts age to the appropriate integer type. get_json_object accepts a JSON string and a JSONPath expression rooted at $; paths such as $.user_id and $.age therefore identify the required top-level fields correctly. This produces analytics-ready columns and prevents downstream queries from repeatedly parsing the original JSON string. Option A returns one nested struct and incorrectly defines age as STRING, so further projection or casting would still be required. Option B attempts field notation directly on a STRING column, which is not valid until the JSON has been parsed into a structured type. Option D omits the required $ JSONPath root and leaves age uncast. Therefore, option C most completely standardizes the source data for the Silver layer.

================

The File Arrival trigger and Auto Loader perform separate, complementary functions. The trigger monitors the configured storage location and starts the Lakeflow Job after detecting new files. After the notebook starts, Auto Loader examines its own checkpoint and file-tracking state to determine which files have not yet been committed successfully. This separation provides incremental processing and exactly-once ingestion behavior. With availableNow=True, Auto Loader processes all data available when the query starts and then stops, allowing the job compute to terminate and control costs. File Notification mode is not required solely because a File Arrival trigger is used. Disabling checkpoints would remove progress tracking and could cause duplicate processing. The trigger also does not eliminate the need to configure the Auto Loader source path. Therefore, option A is correct.

================

The OPTIMIZE command in Delta Lake is designed to compact small files into larger files to improve query performance and reduce file system overhead. Importantly, OPTIMIZE is considered an idempotent operation when run on unchanged data. This means that once files have been compacted optimally, running OPTIMIZE again does not produce additional changes unless new small files have been added to the table. Databricks ensures that file compaction is efficient and avoids unnecessary rewrites, which helps control compute costs and maintain stable performance. While features like ZORDER can be applied during OPTIMIZE to improve data skipping, re-running the same command without new data will not further reorganize files. Options A, B, and C incorrectly imply additional restructuring or changes on repeated execution. According to Delta Lake best practices, OPTIMIZE should be run periodically after data ingestion events rather than repeatedly on unchanged datasets. This behavior reinforces predictable performance and efficient resource utilization in Databricks environments.

=========

DataFrame.dropna() removes rows containing NULL or NaN values. Its default how= " any " behavior removes a row when any column included in the specified subset is NULL. By passing [ " customer_email " , " customer_phone " ] as the subset, the operation removes a record when either required field is missing, while ignoring NULL values in unrelated columns. Option B can produce the same filtering result, but it uses two chained where() calls and therefore does not meet the single-method-call requirement. Option C uses how= " all " and would remove a row only when all evaluated columns are NULL. Option D contains one filter() operation but requires multiple column-function calls. Consequently, option A is the direct PySpark method designed for this requirement.

================