Enabled with delta.autoOptimize.optimizeWrite, this Delta option shuffles data ahead of the write so that fewer but larger files result. Partitioned tables benefit the most, and MERGE, UPDATE and DELETE use it by default.
Also called optimize write.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Optimized writes in context, with comparison tables and the common traps.
Terms in this definition
- MERGE
A SQL command that compares a source with a target table and, depending on whether rows match, inserts, updates or deletes them. Fabric Warehouse supports it as generally available, and Spark SQL offers MERGE INTO for Delta tables.
- CRUD
Shorthand for create, read, update and delete, the four basic things you do with data. Data-plane roles in Azure Cosmos DB, for instance, authorise those operations on items.
Related terms
- readHeavyForPBI
A Spark resource profile in Fabric meant for tables that Direct Lake reports will read heavily; it enables V-Order and 1 GB optimize write bins. Workspaces created today use writeHeavy until changed.