A SQL hint that balances output partitions, so that none ends up undersized or oversized; this is useful for table writes. With AQE switched off it has no effect.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains REBALANCE in context, with comparison tables and the common traps.
Terms in this definition
- Serverless
Compute tier for single Azure SQL databases that scales automatically, pauses when idle and charges by the second. It is offered in General Purpose and Hyperscale, not Business Critical, and reserved capacity does not apply.
- Event
Table in Log Analytics where entries from Windows event logs are kept.
- AQE
Adaptive query execution: Spark re-plans a query while it runs, for instance turning a sort-merge join into a broadcast join, merging shuffle partitions, dealing with skew and propagating empty relations. Databricks advises leaving it enabled.
Related terms
- Cluster Management
Found under Administration > Control Panel in VCF Operations, this page reports on nodes and the cluster and has actions like Rebalance. More node operations are available from the administration interface.
- Partitioning hints
The SQL hints
COALESCE,REPARTITION,REPARTITION_BY_RANGEandREBALANCE, which steer how many output partitions (and hence files) are produced and how evenly; with AQE turned off,REBALANCEhas no effect.