Before choosing a model, it helps to know what one actually does when you send it a prompt. This section walks through the ideas from training to response.
What generative AI is
Generative AI creates new content (text, images, audio, code) from a prompt. It is not supervised learning, where a model is trained on labelled examples to predict a label or a number.
Foundation models are large models pretrained with self-supervised learning: they learn from huge unlabelled collections of text by repeatedly predicting the next token. Because the “label” is just the next piece of the text itself, nobody has to label the data by hand.
Inside a transformer LLM
Most large language models (LLMs) use the transformer architecture. A prompt passes through three stages:
- Tokenization: the text is split into tokens (words, parts of words or punctuation).
- Embedding: each token becomes a multi-dimensional vector (a list of numbers). Tokens with similar meanings sit close together in this vector space.
- Next-token prediction: the model produces a probability distribution over possible next tokens, picks one, appends it to the text and repeats until the response is complete.
Along the way, attention weighs how much each other token in the context should influence the prediction. Figure 2.1 shows the loop, and how training differs from inference.
Common trap: Attention is not the vector itself; the vector is the embedding. Object detection and anonymisation are not stages of an LLM either.
Training vs inference
A model is built once and then used many times, and these are two separate activities.
- Training learns the model’s weights (the numbers inside the network).
- Generating a response for a user is inference. A deployed model is not retrained on each request.
- The model doesn’t store training documents and look them up. Its output is new content that is statistically similar to what it learned.
LLMs and SLMs
Language models come in different sizes. An LLM and a small language model (SLM) use the same kind of architecture; the difference is size, measured in number of parameters. SLMs are cheaper and faster but less broadly capable.
Limits: accuracy and training cutoff
Because a model predicts likely text rather than checking facts, its answers need care.
- An LLM doesn’t fact-check, so it can produce content that sounds plausible but is wrong. Grounding data from trusted sources reduces this risk but never eliminates it.
- A general-purpose chatbot is not fit for medical diagnosis.
- A model only knows data up to its training cutoff. Questions about recent events need up-to-date information supplied at request time through grounding, for example with retrieval-augmented generation (RAG), covered with prompts and grounding later in this book.