How many queries a system handles each second, a figure used to compare different kinds of vector index.
Also called queries per second.
In the Ultra Transcenders books
Each book explains QPS in context, with comparison tables and the common traps.
Terms in this definition
- Chat message roles
Labels on chat messages: instructions go under system, the person's input under user, the model's previous answers under assistant, and results returned by a called tool under tool (or function).
- Vector index
An entry in a Cosmos DB indexing policy, of type flat, quantizedFlat or diskANN, that makes VectorDistance searches faster. It can't be changed once created and needs the vector search feature turned on.
Related terms
- High QPS
A standard-endpoint option in AI Search that adds capacity when you need more than the default 20-200 QPS. You pay for that capacity however much traffic arrives, and scaling is best effort.