Lakeflow Jobs

In the Ultra Transcenders books

DBX-DEP

Each book explains Lakeflow Jobs in context, with comparison tables and the common traps.

Terms in this definition

• Databricks

  Analytics platform built on Apache Spark where data is processed in notebooks; Unity Catalog is the governance model it recommends.

• Job

  A sequence of steps executed together on one agent or runner, or on the server for agentless work. While running, each occupies one of your parallel jobs.

• CONTAINS

  Searches columns with a full-text index for words, phrases, prefixes, inflected forms or synonyms; you use it as a predicate in WHERE.

• LEAST

  Pass up to 254 expressions to this T-SQL function and it hands back whichever is smallest. Both Fabric Warehouse and SQL analytics endpoints support it.

• DAG

  Directed acyclic graph: nodes (tasks or datasets) joined by dependency edges with no loops. Lakeflow Jobs and Lakeflow pipelines execute the nodes in order of their dependencies, running as many in parallel as they can.

Related terms

• Agent Builder

  A Microsoft Copilot tool where people describe a declarative agent in plain language, or fill in a Configure tab, and ground it in sources like SharePoint or Copilot connectors. If the agent later needs actions or workflows, a copy moves into Copilot Studio.

• Automatic remediation

  When switched on in a VCF Operations policy, an alert firing immediately launches its linked orchestrator workflow through the Management Pack for Orchestrator. This works only with workflows needing a single input.

• Azure Deployment Environments

  Retiring on 22 February 2027 (Microsoft points people to Azure Verified Modules, Bicep or ARM deployments and Azure DevOps or GitHub workflows). Until then, it gives developers self-service app infrastructure built from curated IaC definitions kept in dev center catalogs, governed by platform engineers.

• azure/login

  OpenID Connect is preferred over a service principal secret when this Action signs GitHub workflows in to Azure for later PowerShell or CLI steps.

• azure/sql-action

  Used in GitHub workflows to publish a SQL project or .dacpac to SQL Server or an Azure SQL database; it is a wrapper round SqlPackage.

• azure/webapps-deploy

  GitHub workflows use this Action to publish container images or packages to App Service, with an optional slot.

• Catalog

  Where project members in the VCF Automation portal go to order what their organization administrator has published (blueprints, workflows, VM images and so on) and deploy it into a namespace belonging to their selected project.

• CMDB

  Configuration management database: the authoritative store of IT assets and how they are configured, ServiceNow being one example. Workflows triggered by VCF Automation event subscriptions can record new deployments in it.