mlflow.genai.evaluate

In the Ultra Transcenders books

DBX-GENAI

Each book explains mlflow.genai.evaluate in context, with comparison tables and the common traps.

Terms in this definition

• MLflow

  An open-source framework for experiment tracking and model management. Azure Machine Learning adopts it as its tracking API, so each workspace acts as a compatible MLflow tracking server.

• FUNCTION

  Registered models count as functions in Unity Catalog, so this securable type governs them too: access is given through GRANT ... ON FUNCTION, and REST calls pass FUNCTION as securable_type.

• Agent

  A specialised form of Microsoft Copilot set up for one particular job, pairing instructions with knowledge and skills. You can create one in Copilot Studio, SharePoint or Agent Builder, and administrators control them from the Microsoft 365 admin center.

• Agents (classic) API

  First-generation Foundry Agent Service API, based on threads, messages and runs. It is deprecated, replaced by conversations and responses, and retires on 31 March 2027.

• OVER

  Gives a T-SQL window function its window: PARTITION BY, ORDER BY and, if wanted, a ROWS or RANGE frame. Rankings and running totals can then be worked out while every row is kept.

• Evaluation dataset

  A constant collection of test inputs, a query plus optional context and ground truth, fed to every model or prompt variant so their results can be compared fairly.

• Chat message roles

  Labels on chat messages: instructions go under system, the person's input under user, the model's previous answers under assistant, and results returned by a called tool under tool (or function).

Related terms

• RAGAS

  An outside toolkit for judging LLM apps. mlflow.genai.evaluate() can use its metrics as scorers.

• Third-party scorers

  Wrappers that bring open-source tools, DeepEval and TruLens among them, into mlflow.genai.evaluate(), where they run next to MLflow's own judges.