Managed data integration service for ETL and ELT, built from pipelines, copy activities, triggers, mapping data flows and integration runtimes. It works in batches rather than routing individual transactions.
Also called Data Factory.
Read more: Microsoft Learn
In the Ultra Transcenders books
AZ-305AI-103DP-900DP-750DP-700
Each book explains Azure Data Factory in context, with comparison tables and the common traps.
Terms in this definition
- ETL
Extract, transform, load: data is reshaped before it reaches the target. Data Factory offers ETL and ELT as a managed service.
- ELT
Extract, load, transform: raw data lands in the target system first and is transformed there. See ETL for the opposite order.
- Mapping data flows
Data transformations designed visually and executed on Spark, available inside Synapse and Data Factory pipelines.
- Geographic
A Traffic Manager routing method that picks the endpoint according to where the user is located geographically.
Related terms
- Apache Airflow job
A code-first Fabric Data Factory item for orchestrating work as Python DAGs on managed Apache Airflow, offering starter or custom pools, autoscaling and syncing DAGs from Git. It succeeds Workflow Orchestration Manager in Azure Data Factory and sits alongside pipelines as their code-based counterpart.
- Azure IR
Integration runtime in Data Factory that can connect only to data stores in the cloud.
- Azure-SSIS IR
When Data Factory needs to execute SSIS packages, it uses this integration runtime, hosting the SSISDB catalogue in either SQL Managed Instance or SQL Database.
- Copy activity
Activity in a Data Factory pipeline that moves data from one store to another.
- Fabric Data Factory
Presented as the next step on from Azure Data Factory, this Fabric workload moves and reshapes data using copy jobs, mirroring, Dataflow Gen2 and pipelines.
- Fabric workload
Microsoft Fabric groups its features into workloads, each serving a role or job, for instance Power BI, Real-Time Intelligence, Data Warehouse, Data Factory or Data Engineering; they all read and write the same OneLake data.
- Integration runtime
The compute used by Data Factory, which comes as Azure IR, Azure-SSIS IR or self-hosted IR.
- Invoke pipeline activity
Lets one pipeline start another as a step, whether that child sits in a different workspace or in Azure Data Factory or Synapse, so loading logic stays modular. An older legacy version is restricted to calling pipelines that share its workspace.