Capability in Azure Machine Learning that, on a schedule, checks production inference data against reference data for each signal. When a metric crosses its threshold it raises an alert, by email unless Event Grid events are configured.
Also called Azure Machine Learning model monitoring.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Model monitoring in context, with comparison tables and the common traps.
Terms in this definition
- Capability
Something that a person, organisation or system is able to do.
- Azure Machine Learning
Azure platform where you train, deploy, monitor and retrain models of your own, practising MLOps through tools such as pipelines, online endpoints, model monitoring and prompt flow.
- Inference
The act of producing a response with a trained model. Each request neither retrains the deployed model nor makes it look up stored training documents.
- metric
Measurement over time that VCF Operations gathers for an object, such as CPU usage. A policy turns collection of each one on or off, or lets it inherit the setting.
- Event Grid
Routes events, Key Vault SecretNearExpiry for example, by pushing them to handlers such as Azure Functions. It stores nothing and cannot receive Entra diagnostic settings.
Related terms
- Reference data (model monitoring)
Baseline that a model monitoring signal compares against: either training or validation data, or recent production data. For data drift, training data is the recommended choice.