An AI agent goes further than a single prompt and response: it can reason about a request and take actions to fulfil it.
- An agent uses a generative model as its reasoning engine, plus instructions, knowledge (retrieval) and tools, to decide what actions to take and to compose a response. It is not a lookup table of canned answers. Figure 5.1 shows how the parts fit together.
- A prompt agent in Foundry Agent Service is defined by three things: a model, instructions and tools. It is declarative, so Foundry runs it with no application code or container for you to manage.
- By contrast, a hosted agent is a container you build in code; it can’t be edited in the portal UI.
Settings that shape an agent
Each setting controls one thing, so match the requirement to the right setting.
- Instructions set the agent’s role, goals, tone and limits.
- Temperature changes the randomness of responses.
- TPM is a deployment quota, not an agent behaviour.
- Max completion tokens caps the length of a response.
- A deployment slot, an embedding index or a fine-tuning job doesn’t define the agent’s role.
Controlling tool use
The tool_choice parameter decides whether the model calls tools.
tool_choice |
Effect |
|---|---|
auto (default) |
The model decides whether to call a tool; there’s no guarantee it will |
required |
The model must call one or more tools on every response |
none |
No tool calls |
| A specific tool’s specification | Forces that one tool |