A service that lets you call Databricks-hosted foundation models with no deployment of your own to look after. A serving endpoint provides the access, and you can use modes like pay-per-token or provisioned throughput.
Read more: Databricks docs
In the Ultra Transcenders books
Each book explains Foundation Model APIs in context, with comparison tables and the common traps.
Terms in this definition
- Foundation
Nutanix's own tool for building clusters, loading nodes first with AOS and a hypervisor that are supported together. Its three forms are the standalone VM, the app and the version in the CVM.
- Model deployment
Inside a resource, each deployment is a named copy of a Foundry model with its own TPM quota and deployment type. Requests identify the model by this deployment name.
- LIKE
Compares strings with a pattern that can contain the % and _ wildcards. Because it only understands character patterns, searching big volumes of text this way is much slower than using full-text search.
- Provisioned throughput
Deployment option that reserves model capacity for you in provisioned throughput units (PTUs) to give predictable latency and throughput. The capacity is charged by the hour regardless of usage.
Related terms
- GPT OSS
Reasoning models with open weights from OpenAI, which Foundation Model APIs can serve. For batch inference or real-time copilots, GPT OSS 20B is the lighter choice.
- ITPM
Caps how many input tokens a pay-per-token endpoint on Foundation Model APIs can process each 60 seconds, a limit set by the platform. A 429 comes back when it is passed, and where OTPM or QPH is tighter, that one governs instead.
- OTPM
Limits how many output tokens Foundation Model APIs will produce in 60 seconds. On admission, each request sets aside its
max_tokensfrom this allowance, and a 429 error turns away any that wouldn't fit. - Platform limits
Ceilings on ITPM, OTPM and QPH (input and output tokens a minute, and queries an hour) built into pay-per-token Foundation Model APIs, where the tightest one wins. Any rate limits of your own are a different thing.
- Reserved provisioned throughput
A cheaper unit rate on Foundation Model APIs in return for committing to dedicated model units for one or three months, billed in full even if unused. Only a curated list of models qualifies.