Short for least recently used. When space runs out, both the Apache Spark cache and the disk cache evict the data that has gone longest without being used.
Also called least recently used.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains LRU in context, with comparison tables and the common traps.
Terms in this definition
- Agents (classic) API
First-generation Foundry Agent Service API, based on threads, messages and runs. It is deprecated, replaced by conversations and responses, and retires on 31 March 2027.
- Apache Spark cache
A cache you request explicitly, in memory or on disk, for a table or DataFrame using
df.cache(),df.persist()orCACHE TABLE, and release withunpersist. It differs from the automatic disk cache and cannot be used on serverless compute. - Disk cache
Keeps copies of remote Parquet files, Delta tables among them, on the local SSDs of worker nodes so the same data is read faster next time. Azure Databricks handles it automatically: no code is required and it invalidates itself when files change, unlike the Apache Spark cache.