The Delta Lake log layered over Parquet files that tracks every change, giving a data lake ACID transactions, versioned data and dependable updates.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Transaction log in context, with comparison tables and the common traps.
Terms in this definition
- Delta Lake
Table format adding transactions to files in the data lake. In Synapse, Spark pools can write these tables, while serverless SQL pools can only query them.
- log
Text written by a component to record something that happened. VCF Operations for logs takes these in, and after integration with VCF Operations they can be searched from its Analyze page and Log Analysis widget.
- OVER
Gives a T-SQL window function its window: PARTITION BY, ORDER BY and, if wanted, a ROWS or RANGE frame. Rankings and running totals can then be worked out while every row is kept.
- Parquet
Columnar file format, open source and aimed at analytics. Event Hubs Capture cannot produce it natively.
- Data lake
Holds files of every kind, structured, semi-structured or unstructured, usually spread over a distributed file system, and lets engines like Spark apply a schema when reading. In Azure this is Data Lake Storage: Blob Storage with a hierarchical namespace enabled.
- ACID
Four promises a database transaction makes: atomicity, consistency, isolation and durability. Azure Cosmos DB honours them, with snapshot isolation, only inside one logical partition, using stored procedures, triggers or transactional batch.
Related terms
- LSN
Log sequence number: a unique, ordered identifier given to every record in the transaction log. Change data capture sorts changes by their commit LSN (stored in __\$start_lsn) and uses a pair of LSNs to choose which changes to return.
- Veeam Explorer for Microsoft SQL Server
The Veeam Explorer for recovering SQL Server databases. It mounts the backup to the destination SQL Server and, if transaction log backups are available, can roll a database to a specific point in time.
- VSM
SQL Server transaction log data sits in this file type, kept in the repository together with VLB and VLM files.