Rather than reloading everything, a table is split into date-based partitions and only new or changed periods are reloaded. Semantic models rely on RangeStart and RangeEnd parameters here; Dataflow Gen2 uses buckets instead, and needs a fixed-schema destination.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Incremental refresh in context, with comparison tables and the common traps.
Terms in this definition
- Event
Table in Log Analytics where entries from Windows event logs are kept.
- RangeStart
Along with RangeEnd, a reserved Date/Time parameter in Power Query (the names are case-sensitive). Filtering a table's date column with the pair lets Power BI apply incremental refresh and build partitions by date.
- Dataflow Gen2
A visual, low-code Microsoft Fabric item for preparing data, based on Power Query. It links to data sources, transforms the data and writes the output to targets like a lakehouse or a warehouse.
Related terms
- ALM Toolkit
An open-source, no-cost utility for comparing Power BI semantic model schemas. Because it can release just the metadata changes you pick through the XMLA endpoint, a model that's already published keeps both its data and its incremental refresh partitions.
- Detect data changes
An extra incremental refresh option: nominate a timestamp column, for instance one recording when rows were last modified, and Power BI skips any partition whose latest timestamp hasn't moved since the last run.
- Lakeflow Spark Declarative Pipelines
Also known as Lakeflow pipelines: in Azure Databricks you say which tables you want, and the framework deals with orchestration, incremental refresh and data quality checks. Databricks recommends it for production ingestion.
- Optimal refresh
Turned on by default for materialized lake views, this setting chooses on every run between skipping, an incremental refresh and a full one. Source tables need Delta change data feed for incremental refresh to happen, and disabling the setting makes each run a full rebuild.
- Real-time DirectQuery partition
Turning this incremental refresh option on makes a table hybrid: rows newer than the refresh period are read through an extra DirectQuery partition. Premium, PPU or Embedded is needed, and Dual storage mode is advisable for related tables.