A set of Photon-only optimisations that make reads and updates faster. Updates rely on deletion vectors automatically, and no setting is needed for each query.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains Predictive I/O in context, with comparison tables and the common traps.
Terms in this definition
- Set
Secret permission in Key Vault for writing secrets; some older material refers to it as Create.
- RELY
Tells the optimiser it can depend on a primary key or unique constraint when rewriting queries, perhaps dropping a join or a
DISTINCT. The keys remain unenforced, and this works only on serverless compute or on pro or serverless SQL warehouses. - Deletion vectors
Lets Delta Lake and Iceberg tables flag removed or changed rows through metadata rather than rewriting complete Parquet files, a soft delete that speeds up deletes, updates and merges. The physical rewrite happens later, when
OPTIMIZEor aREORG TABLEpurge runs.
Related terms
- Bloom filter indexes
An older way of skipping data in Delta tables, now deprecated, that Databricks advises dropping and replacing with liquid clustering and predictive I/O. The filters built into Parquet files are unrelated.
- IO
Reading from and writing to storage. Databricks' Predictive IO, written Predictive I/O in its documentation, is a group of optimisations for faster data reads and updates.
- Pro SQL warehouse
A middle option between the classic and serverless warehouse types, hosted in your cloud account with Photon and Predictive I/O. Apart from serverless, nothing else can run SQL or dbt tasks in a job.