HTTPS endpoint serving synchronous, low-latency predictions one request at a time, able to place multiple deployments behind a single scoring URI. It can be managed or Kubernetes-based.
Also called real-time endpoint.
Read more: Microsoft Learn
In the Ultra Transcenders books
Each book explains Online endpoint in context, with comparison tables and the common traps.
Terms in this definition
- HTTPS
Secure HTTP, wrapped in TLS. VCF products serve their web UIs and REST APIs this way on 443.
- Scoring URI
The one URL that clients use to call an endpoint; it does not change even as the deployments behind it are swapped.
Related terms
- AmlOnlineEndpointConsoleLog
Table in Log Analytics capturing console output from Azure Machine Learning online endpoint containers, useful when a container will not start or handles requests incorrectly.
- az ml online-endpoint update --traffic
Azure CLI command for dividing live requests across the deployments behind one online endpoint, given as name=percentage pairs such as
blue=90 green=10; the values have to add up to either 100 or 0. - Blue-green deployment
Safe rollout approach for an online endpoint: a new green deployment is added with no traffic, tested, then given a growing share; to roll back, send all traffic to blue again (blue=100).
- Connections Active
Online endpoint metric in Azure Monitor showing how many client TCP connections are open at once.
- Custom speech model expiration
What happens once a custom speech model expires: real-time endpoint calls drop back to the newest base model for that locale, whereas batch jobs naming the model fail with a 4xx error; the model itself remains.
- Managed online endpoint
Azure Machine Learning online endpoint where Azure provisions and runs the compute for you; it offers autoscale, traffic mirroring and splitting traffic between deployments.
- No-code deployment
Azure Machine Learning itself supplies the scoring script and environment, so an MLflow model can go to a batch or online endpoint with neither written by you.
- Online deployment
The resources behind an online endpoint, namely model, environment, scoring script, instance type and instance count. One endpoint may contain several, for example blue and green.