Tokens: how AI reads text ▶ Watch
LLMs do not read letters or whole words. They read tokens - chunks of text that are often a word or part of a word. "Unbelievable" might become "un", "believ", "able". In English, one token is roughly three-quarters of a word, so 1,000 tokens is about 750 words. AI services usually charge and set limits by tokens.
The core trick: predicting the next token ▶ Watch
At its heart an LLM does one thing: given the text so far, it predicts the most suitable next token. It adds that token and predicts again, building answers word by word. Because it was trained on an enormous amount of human writing, doing this well requires learning grammar, facts, reasoning patterns, styles and even programming.
How an LLM is trained ▶ Watch
- Pre-training - the model reads a huge, diverse collection of text and code and learns to predict the next token. This produces a powerful but unfocused "base model".
- Supervised fine-tuning - it is trained on high-quality example conversations so it learns to follow instructions.
- Alignment - techniques such as RLHF (reinforcement learning from human feedback) and AI feedback guided by written principles make it more helpful, honest and safe.
- Deployment and improvement - it is released, monitored, and improved in later versions.
Important LLM settings and terms ▶ Watch
| Term | Meaning |
|---|---|
| Context window | How much text (in tokens) the model can consider at once: your prompt, files and the conversation so far. |
| Temperature | Controls randomness. Low = focused and consistent; high = more creative and varied. |
| System prompt | Background instructions that set the assistant's role and rules for the whole conversation. |
| Max tokens | The maximum length of the answer. |
| Knowledge cutoff | The date after which the model has no training data. |
| Multimodal | Able to work with images, audio or video as well as text. |
| Tool use | The model can call external functions such as web search or a calculator. |
How AI creates images: diffusion ▶ Watch
Image generators are trained by taking real pictures, adding noise to them, and teaching a network to remove the noise. To create a new image, the model starts from pure noise and removes it step by step, steered by your text description, until a clear picture appears. The same idea is used for video and audio generation.
Multimodal AI ▶ Watch
Modern models increasingly handle several types of data in one system. You can photograph a maths problem and ask for an explanation, upload a chart and ask for insights, or have a spoken conversation in real time. Multimodality makes AI far more natural to use.
Chapter summary
- LLMs read and write in tokens and work by predicting the next token, over and over.
- Training goes through pre-training, fine-tuning and alignment.
- Context window, temperature and system prompts control how a model behaves.
- Diffusion models create images by removing noise; multimodal models combine text, images and audio.