Directed acyclic graph: nodes (tasks or datasets) joined by dependency edges with no loops. Lakeflow Jobs and Lakeflow pipelines execute the nodes in order of their dependencies, running as many in parallel as they can.
Also called directed acyclic graph.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains DAG in context, with comparison tables and the common traps.
Terms in this definition
- EXECUTE
Allows calling a function or loading a registered model in Unity Catalog, and seeing its definition.
USE CATALOGandUSE SCHEMAare needed too.
Related terms
- Git-sync
With this option an Apache Airflow job keeps its DAG code in a Git repo rather than in storage that Fabric manages. Just two folders, dags and plugins, are kept in step, and editing in the built-in editor is no longer possible.