Monitoring signal that checks how the production distribution of each input feature differs from a baseline, normally the training data, with measures like PSI or Jensen-Shannon distance; crossing a threshold can kick off retraining.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Data drift in context, with comparison tables and the common traps.
Terms in this definition
- Monitoring signal
Each model monitor in Azure Machine Learning is made up of these checks, each carrying separate metrics and thresholds: data quality, data drift, prediction drift, feature attribution drift, model performance, or custom.
- Baseline
A formally reviewed and approved specification that may afterwards be altered only under formal change control. The word also describes the architecture as it stands today.
- LIKE
Compares strings with a pattern that can contain the % and _ wildcards. Because it only understands character patterns, searching big volumes of text this way is much slower than using full-text search.
Related terms
- Reference data (model monitoring)
Baseline that a model monitoring signal compares against: either training or validation data, or recent production data. For data drift, training data is the recommended choice.