Avoiding reads of files that cannot hold matching rows, using column statistics such as min, max and null counts gathered per file at write time. Liquid clustering and Z-ordering improve how much gets skipped.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Data skipping in context, with comparison tables and the common traps.
Terms in this definition
- Statistics
Information about how values are distributed in a column, which the warehouse optimiser uses to estimate plan costs. Fabric builds and updates it for you; manual CREATE, UPDATE and DROP STATISTICS are limited to single-column histograms.
- MIN
Gives back the lowest value found in a column holding numbers, dates or text, or compares two expressions and returns the lower. For TRUE/FALSE values, which this DAX aggregation doesn't accept, turn to MINA.
- MAX
Gives back the biggest value found in a column holding numbers, dates or text, or compares two expressions and returns the bigger. For TRUE/FALSE values, which this DAX aggregation doesn't accept, turn to MAXA.
- NULL
Marks that a row has nothing in a particular column, as when a person has no middle name. Columns defined NOT NULL can never be left empty.
- Liquid clustering
Organises a table's data around up to four clustering keys set with
CLUSTER BY, as an alternative to both partitioning andZORDER. The keys can be changed without rewriting existing files, andCLUSTER BY AUTOhands key selection to predictive optimization. - Z-ordering
Older technique:
OPTIMIZE ... ZORDER BY (cols)clusters related values into the same files so more data can be skipped. Incompatible with liquid clustering, which is preferred for new tables.