Job compute

In the Ultra Transcenders books

DP-750

Each book explains Job compute in context, with comparison tables and the common traps.

Terms in this definition

• Job

  A sequence of steps executed together on one agent or runner, or on the server for agentless work. While running, each occupies one of your parallel jobs.

• OVER

  Gives a T-SQL window function its window: PARTITION BY, ORDER BY and, if wanted, a ROWS or RANGE frame. Rankings and running totals can then be worked out while every row is kept.

• Classic compute

  All-purpose, jobs and pipeline compute running in your own Azure subscription, which you set up and manage yourself. Its counterpart is serverless compute, which Azure Databricks runs for you.

• All-purpose compute

  Interactive classic compute for notebook work, shareable between several people and restarted by hand. Its pricing differs from jobs compute, and production jobs are better run elsewhere.

• Share

  In OpenSharing, the container you fill with tables, views, volumes, models and notebooks for recipients. You need CREATE SHARE on the metastore to make one; add a schema and all its assets, including later ones, go with history.

Related terms

• Databricks Runtime

  The software image, Apache Spark among its core parts, that Azure Databricks installs on compute. Long-term support (LTS) releases are the advised choice when job compute runs production work.

• LTS

  Applies to some Databricks Runtime versions, which Learn recommends for job compute in production. Feature work on such a version stops after about six months, and for three years afterwards it gets security and stability fixes backported.

• Policy family

  Templates supplied by Azure Databricks that match the default compute policies, such as Personal Compute and Job Compute. Custom policies inherit rules from a family, and families cannot be changed or removed.