Optional data quality rules declared with CONSTRAINT ... EXPECT on the tables and views a pipeline defines. Each incoming record is checked; on failure the pipeline can just record a warning (the default), discard the record or halt the update, and the results are logged.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Expectations in context, with comparison tables and the common traps.
Terms in this definition
- Pipeline
Groups activities logically so that, together, they move and transform data, either one after another or side by side; found in both Azure Data Factory and Microsoft Fabric.
- CRUD
Shorthand for create, read, update and delete, the four basic things you do with data. Data-plane roles in Azure Cosmos DB, for instance, authorise those operations on items.
Related terms
- Pipeline product edition
The
editionsetting for a Lakeflow pipeline:COREcovers streaming ingestion,PROadds CDC andADVANCED, the default, adds expectations. If the pipeline uses a feature its edition lacks, the update fails. - SDP
Spark Declarative Pipelines: open-source Apache Spark tooling to declare batch or streaming data flows in Python or SQL. Databricks extends it as Lakeflow Spark Declarative Pipelines, adding
AUTO CDC, expectations and a queryable event log.