Checks output against rules written in ordinary language, perhaps about compliance or tone, and reports whether they were met; it is one of MLflow's LLM judges.
Also called Guidelines.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains Guidelines judge in context, with comparison tables and the common traps.
Terms in this definition
- MLflow
An open-source framework for experiment tracking and model management. Azure Machine Learning adopts it as its tracking API, so each workspace acts as a compatible MLflow tracking server.
- LLM
Large language model, usually a transformer network with billions of parameters that learnt to predict the next token from vast amounts of text. Compared with a small language model it is more capable across tasks, but slower and costlier.
Related terms
- CSF
Short for the NIST Cybersecurity Framework: optional standards, guidelines and practices that help organisations handle cyber risk. Alongside ISO, it supplies most controls in the baseline assessment Compliance Manager starts with.
- Foundation Architecture
The least specific stage of the Architecture Continuum, offering components, principles and guidelines that more specialised architectures build upon. One example is the TRM.
- Reference Library
Holds guidelines, templates, patterns and libraries of viewpoints within the Architecture Repository, giving new architecture work a quicker start.
- Reserved keys
expected_facts,guidelinesand similar entries underexpectationsin evaluation data, which the stock judges and scorers recognise by name. - WCAG
Web accessibility guidelines published by the W3C and followed by Power BI. Version 2.1, success criterion 1.4.3, sets a minimum text-to-background contrast of 4.5:1.