Events that keep arriving without end, for example log entries, sensor readings or social media posts; stream processing is how they are handled.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Streaming data in context, with comparison tables and the common traps.
Terms in this definition
- Stream processing
Handling each new item of data in real time as soon as it arrives, rather than gathering it into batches. It fits time-sensitive work, with latency measured in seconds or milliseconds against the hours batch processing usually needs.
Related terms
- Azure Stream Analytics
Serverless service that processes streaming data with a SQL-like language, offering windowing functions and anomaly detection out of the box.
- Real-Time Dashboard
An item in Fabric Real-Time Intelligence that displays streaming data as live visuals with low latency. Each tile combines a KQL query and a visual, and Copilot can help create tiles from natural-language requests.
- Real-Time hub
Microsoft Fabric's automatically provisioned, tenant-wide hub for finding, bringing in and managing streaming data, listing data streams, KQL tables, Microsoft sources and Fabric events in one place.
- SDP
Spark Declarative Pipelines: open-source Apache Spark tooling to declare batch or streaming data flows in Python or SQL. Databricks extends it as Lakeflow Spark Declarative Pipelines, adding
AUTO CDC, expectations and a queryable event log. - Streaming dataflows
Once offered no-code preparation and ingestion of streaming data in Power BI but has now been retired. Those capabilities live on in Azure Stream Analytics no-code stream processing.