Decides which columns count towards SCD Type 2 versioning in AUTO CDC. Changes to any other column just overwrite the current row.
Also called track_history_column_list.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains TRACK HISTORY in context, with comparison tables and the common traps.
Terms in this definition
- COUNT
A DAX function returning how many non-blank numbers, dates or text values a column contains. Boolean columns need COUNTA instead, and for counting the rows of a table COUNTROWS is the better choice.
- History tracking
Retains previous versions of a row instead of replacing them. Lakeflow Connect tables leave it off by default, and in AUTO CDC the
TRACK HISTORYoption restricts which columns create new versions. - Versioning
When enabled on Blob Storage, earlier versions of each blob are retained. Point-in-time restore depends on it, though it never purges data itself.
- AUTO CDC
The Lakeflow pipelines API for change data capture: it merges incoming changes into a streaming table, keeping either just the latest values (SCD 1) or full history (SCD 2), and uses an ordering column to cope with records that arrive late. A snapshot variant also exists.
Related terms
- SCD
Slowly changing dimension. Type 1 keeps only current attribute values; type 2 retains every past version, bounded by
__START_ATand__END_AT. Pipelines get both viaAUTO CDC(STORED AS SCD TYPE 1orSTORED AS SCD TYPE 2), whileTRACK HISTORY ONnames which columns produce history.