Unity Gateway

In the Ultra Transcenders books

DBX-DEPDBX-GENAI

Each book explains Unity Gateway in context, with comparison tables and the common traps.

Terms in this definition

• AMPRS

  An acronym for availability, manageability, performance, recoverability and security: the five design qualities used to sort non-functional requirements and to weigh up trade-offs between design decisions.

• Databricks

  Analytics platform built on Apache Spark where data is processed in notebooks; Unity Catalog is the governance model it recommends.

• Control plane

  Management layer used to create, configure and remove resources, as distinct from the data plane where they are used. Azure Resource Manager fills this role for Azure, and in AKS Azure operates the Kubernetes control plane on your behalf.

• MCP

  Model Context Protocol, an open standard for making tools and data available to models. To use one, a Foundry agent points at a remote MCP server, naming it with a server label and giving its URL.

• Budgets

  In Cost Management, spending limits you can scope and filter by tag, which raise alerts when actual or forecast spend nears or passes them.

• Usage tracking

  Unity Gateway services log to system.ai_gateway.usage and legacy endpoints to system.serving.endpoint_usage. Those system tables hold token figures and details about each call, never the payload itself.

Related terms

• AI Guardrails

  The legacy AI Gateway way of filtering unsafe content and personal data on a serving endpoint. Unity Gateway service policies are now preferred.

• API Management llm-content-safety policy

  Policy for the AI gateway in API Management: it passes prompts or completions to Azure AI Content Safety and rejects the call with 403 if a blocklist matches, Prompt Shields detect an attack, or a harm category passes its threshold, where 0 is strictest and 7 most lenient.

• API Management llm-token-limit policy

  Inbound AI gateway policy that caps token usage per counter key. Exceeding tokens-per-minute produces 429, an exhausted token quota produces 403, and estimate-prompt-tokens lets it reject before calling the backend.

• Budgets

  Account-level controls that follow monthly spending, in USD at list price, for the whole account or a filtered part of it, and raise alerts. Per-user thresholds and blocking of further usage are limited to budgets scoped to Unity Gateway.

• External MCP server

  An MCP server you host yourself or get from a third party, then register with Unity Gateway for governance. It has to support Streamable HTTP, and a Unity Catalog connection stores its credentials.

• External service policy

  A Beta policy type in Unity Gateway that asks a guardrail vendor to judge content, then acts on the decision. Redaction isn't possible with it, and the vendor's endpoint has to support the Databricks external policy API.

• Fallback

  When a request fails with an error like 429 or 5xx, this Unity Gateway option tries alternative destinations one after another in the order configured. Only the first choice of destination comes from traffic splitting; retries ignore the split.

• Foundry Control Plane

  Lets organisations govern, monitor and trace their agents from one central place in Microsoft Foundry. Agents built outside Foundry can be included after being registered via an AI gateway into a project that is wired to Application Insights.