ETL

In the Ultra Transcenders books

AZ-305DP-900DP-750DP-600DP-700PL-300MD-102DP-800

Each book explains ETL in context, with comparison tables and the common traps.

Terms in this definition

• ELT

  Extract, load, transform: raw data lands in the target system first and is transformed there. See ETL for the opposite order.

• Azure Data Factory

  Managed data integration service for ETL and ELT, built from pipelines, copy activities, triggers, mapping data flows and integration runtimes. It works in batches rather than routing individual transactions.

Related terms

• Audit attributes

  Columns you can choose to add to fact or dimension tables to log the history of each row, such as the date it was inserted, the date it was last updated, or which ETL job made the change.

• Azure Synapse Link for Cosmos DB

  Feature for querying Cosmos DB's analytical column store without ETL and without using request units from the transactional store. New projects can no longer adopt it and should use Cosmos DB mirroring in Microsoft Fabric.

• Fabric mirroring

  Keeps a live copy of a source system such as Azure Cosmos DB or Azure SQL Database in OneLake, stored as Delta Parquet, so it can be analysed straight away with no ETL to write.

• Non-SELECT pool

  Under workload management, warehouses and SQL analytics endpoints in a workspace share two separate compute pools, split evenly by default. This one handles loading and ETL work; read queries go to the SELECT pool.

• SSIS

  Integration Services for SQL Server; Azure runs its ETL packages on the Azure-SSIS integration runtime.

• TRUNCATE TABLE

  Deletes every row from a table, and is available in Fabric warehouses. It requires ALTER permission on the table or its schema, the smallest permission you can give an ETL identity for this job.

• Warehouse snapshot

  Read-only, user-created item beneath a warehouse showing its data as it stood at one moment inside the retention window. Rolling its timestamp forward keeps reports stable while ETL runs.

• writeHeavy

  Newly created Fabric workspaces default to this Spark resource profile, controlled by spark.fabric.resourceProfile. Built for ETL and ingestion, it leaves V-Order switched off; reading suits readHeavyForSpark or readHeavyForPBI better.