The Databricks service for similarity retrieval in RAG: it turns a source table into an index of embeddings and metadata, then answers each query with the closest matches.
Also called formerly Databricks Vector Search, Databricks Vector Search, Mosaic AI Vector Search.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains AI Search in context, with comparison tables and the common traps.
Terms in this definition
- Databricks
Analytics platform built on Apache Spark where data is processed in notebooks; Unity Catalog is the governance model it recommends.
- RAG
Technique for answering from private or recent information: relevant content is fetched from your own data, typically a search index, and inserted into the prompt as grounding so the model can respond with citations.
- Event
Table in Log Analytics where entries from Windows event logs are kept.
- Index
Speeds up queries that filter on certain columns by keeping those columns sorted, with pointers back to each row; the price is more storage and slower writes.
- Embeddings
Vectors of numbers encoding what text means, placing related content close together as judged by cosine similarity. Switching model or vector size requires embedding all content again.
- Metadata
Describes data rather than being the data itself, for instance how a file is laid out or what rows each chunk contains, so tools can read things efficiently.
Related terms
- ABAC EXCEPT clause
The section of an ABAC policy listing who it doesn't apply to. Whoever runs a deep or shallow clone, OpenSharing or AI Search indexing against a protected table has to be excluded from all policies that apply to it.
- AI Bridge
databricks-ai-bridgelets agents written in Python reach Genie Agents and AI Search indexes with no MCP layer in between. It is the recommended choice when starting a new agent in Python. - AI Search endpoint
Hosting for AI Search indexes. Capacity, response times, how syncing works and which filters are allowed all depend on whether it was created as Standard or Storage-optimized.
- AI Search reranker
An optional extra step in AI Search that scores the retrieved documents again against the query. Results become more relevant, but queries take longer.
- Azure OpenAI Embedding skill
Skill in an AI Search skillset that sends each chunk to an embedding model deployment (text-embedding-3-small, text-embedding-3-large or ada-002) during indexing to produce vectors; charges come from Azure OpenAI.
- Continuous sync
Refreshes an AI Search index within seconds, using a provisioned streaming cluster that makes it the pricier Delta Sync mode. Only standard endpoints offer it.
- Dedicated full-text index
An AI Search index, in Beta, that has no embedding columns and uses BM25 keyword search alone to find exact terms and identifiers. It runs only on storage-optimized endpoints with triggered sync.
- Delta Sync Index
Follows changes in a source table automatically, in either Continuous or Triggered mode. This type of AI Search index can take your own embeddings or have Databricks generate them.