A compute setting under Advanced > Logging that copies logs from the driver, the workers and the event stream to a chosen destination every five minutes for as long as the compute runs. A Unity Catalog volume is the preferred destination; DBFS also works.
Also called cluster log delivery.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Compute log delivery in context, with comparison tables and the common traps.
Terms in this definition
- Event
Table in Log Analytics where entries from Windows event logs are kept.
- STREAM
Placed before a source in a query, as in
STREAM read_files(...), this keyword tells SQL to process it as a stream, which is how streaming tables get loaded bit by bit. - Agents (classic) API
First-generation Foundry Agent Service API, based on threads, messages and runs. It is deprecated, replaced by conversations and responses, and retires on 31 March 2027.
- Unity Catalog
Azure Databricks' governance solution covering both data and AI in one place, with centralised permissions, auditing, data discovery and lineage.
- Volume
Holds non-tabular files sitting in cloud storage under Unity Catalog governance, addressed as
/Volumes/<catalog>/<schema>/<volume>. A volume can be external or managed, andWRITE VOLUMEorREAD VOLUMEdecide who may use it. - DBFS
Abstraction in Databricks that presents underlying storage, ADLS Gen2 for example, as a file system.