Lakeflow Jobs alerts and notifications

Notifications tell people or systems that a run started, succeeded, failed, ran too long or fell behind. They can be set on the whole job and on individual tasks; editing job notifications needs CAN MANAGE or IS OWNER on the job.

Destinations are email addresses or system destinations: Slack, Microsoft Teams, PagerDuty and HTTP webhooks. A workspace admin creates system destinations in the admin settings (Edit system notifications); each job or task can use at most three system destinations per event type. Webhooks receive a JSON payload with an event_type such as jobs.on_failure or jobs.on_duration_warning_threshold_exceeded. Because the content of Slack and Teams messages may change, build automation on a user-defined webhook instead.

Duration thresholds

Under Duration and streaming backlog thresholds (job) or Metric thresholds (task), the Run duration metric has two fields:

• Warning: the expected completion time. Exceeding it raises an event that a Duration warning notification can report; the run continues.
• Timeout: the maximum completion time. Exceeding it ends the run with the status Timed Out.

Reducing noise

• Mute notifications for skipped runs and Mute notifications for canceled runs filter those events; muting at job level does not mute task-level notifications.
• On a task notification, Mute notifications until the last retry suppresses alerts until the final retry.
• Streaming backlog thresholds and notifications are not supported for pipeline tasks, and they need the Jobs service to track the streaming query, so do not call awaitTermination() in jobs.

Common trap: Relying on a job-level failure notification to hear about every failed attempt - job-level notifications are not sent when failed tasks are retried. Add task-level notifications for per-attempt alerts, and use Mute notifications until the last retry if you want only the final outcome.

Common trap: Treating the Warning duration as a hard limit - it only raises a duration-warning event. Only Timeout stops the run and marks it Timed Out.

Get the whole book

This note is one section of Ultra Transcenders DP-750: Implementing Data Engineering Solutions Using Azure Databricks, an independent study guide that explains every topic the exam covers by technology, with comparison tables, diagrams and the common traps, plus a glossary linked to Microsoft Learn.

About the book · DP-750 terms in the glossary · All DP-750 study notes