The piece that looks up the chunks best matching a query, and can be exposed to an agent as a tool. Retrieval judges and the Review App locate its context through the RETRIEVER span it gets in MLflow tracing.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains Retriever in context, with comparison tables and the common traps.
Terms in this definition
- Agent
A specialised form of Microsoft Copilot set up for one particular job, pairing instructions with knowledge and skills. You can create one in Copilot Studio, SharePoint or Agent Builder, and administrators control them from the Microsoft 365 admin center.
- Chat message roles
Labels on chat messages: instructions go under system, the person's input under user, the model's previous answers under assistant, and results returned by a called tool under tool (or function).
- Review App
Domain experts use this MLflow website to label traces or chat with an agent. Agents hosted on Databricks Apps lack the Chat UI and use labeling sessions for review instead.
- Span
One operation inside a trace, such as a single LLM call or prompt flow node, carrying attributes plus start and end times. Nested spans reveal the order of calls.
- MLflow Tracing
Captures what goes into and comes out of every intermediate step of a request, plus metadata, which helps pinpoint where an agent did something unexpected.
Related terms
- Component metrics
Scores for a single stage of an LLM application, worked out from what entered and left that stage; a retriever's precision at K is one example.
- Compound metrics
Evaluation measures of how the steps of a process work together, for example whether a generated answer stays faithful to the retriever's output.
- Query rewriting
Restating what someone asked, possibly as several queries, to improve the odds of finding suitable documents. Whatever is produced has to fit the search method the retriever uses.