The messaging service on Google Cloud. Databricks consumes it as a streaming source, both in pipelines and in Databricks SQL as read_pubsub.
Also called Google Pub/Sub.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains Pub/Sub in context, with comparison tables and the common traps.
Terms in this definition
- Databricks
Analytics platform built on Apache Spark where data is processed in notebooks; Unity Catalog is the governance model it recommends.
- Streaming
Sending model output piece by piece as server-sent events. Only the delivery changes; the answer's content, completeness and cost stay the same.
- Databricks SQL
Brings warehousing to an Azure Databricks lakehouse. ANSI SQL runs against Delta tables using SQL warehouses, compute tuned for heavy query concurrency and BI tools.
- CRUD
Shorthand for create, read, update and delete, the four basic things you do with data. Data-plane roles in Azure Cosmos DB, for instance, authorise those operations on items.
Related terms
- Azure Event Grid
Microsoft's managed pub/sub broker for discrete events. Publishers hand events over, filters decide who receives them, and handlers get them over HTTP by push or by pulling, with failed deliveries retried and, if necessary, dead-lettered. It also works as a broker for MQTT.
- Dapr
Distributed Application Runtime, offered in Container Apps and as an AKS extension, which gives apps ready-made building blocks like pub/sub and state management.
- Dapr components
Connect Dapr APIs in Azure Container Apps to the services behind them, such as a state store on Cosmos DB or pub/sub on Service Bus. They are defined at environment level, and each one's scopes list names the Dapr app IDs allowed to load it.
- Standard connectors
Part of Lakeflow Connect: Auto Loader for object storage, plus SFTP and message buses (Kafka, Pulsar, Pub/Sub). Choose them over managed connectors when you need more control, working in Databricks SQL, Structured Streaming or pipelines.