A Fabric Spark mode where notebooks, or notebook activities in pipelines, run by one user with matching default lakehouse, Spark configuration and libraries reuse a single Spark session. They start more quickly and only the session that started it is charged.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains High concurrency mode in context, with comparison tables and the common traps.
Terms in this definition
- WHERE
Limits a SELECT, UPDATE or DELETE to just the rows meeting a condition. Omit it, and the statement hits every row.
- Notebook
Combines runnable code cells, in Python, SQL, Scala or R, with their results in one interactive Azure Databricks document; it runs on attached compute, and a job can run it as a task.
- Chat message roles
Labels on chat messages: instructions go under system, the person's input under user, the model's previous answers under assistant, and results returned by a called tool under tool (or function).
- Default lakehouse
Pinned to a notebook, it determines where relative
Files/andTables/paths and Spark SQL table names without a qualifier point. If you switch to another lakehouse or rename it, the session must be restarted.
Related terms
- Autotune
A preview option in Fabric Spark that studies previous runs to pick better values for shuffle partitions, broadcast join threshold and max partition bytes query by query. It's disabled by default (
spark.ms.autotune.enabled), limited to Runtime 1.2, and incompatible with private endpoints and high concurrency mode. - FAIR scheduling
A Spark policy that shares out cluster capacity evenly among jobs that run concurrently. When notebooks share one session in Fabric's high concurrency mode, this is what splits executors between their REPL cores.
- Session tag
Notebook activities in a pipeline can be given matching text in their advanced settings; notebooks with that text then share one high concurrency Spark session, cutting start-up time. This needs high concurrency mode for pipelines enabled on the workspace, and by default a tag opens a further session once five notebooks are using it.