● The AI Primer, illustrated
How modern AI works, from the ground up.
What happens when you send a prompt, how an AI model is trained and judged, what it costs to run, how search over meaning works, and how to build an AI agent people can rely on: 53 lessons in six parts, in order. Every idea starts with an everyday picture, is worked through with numbers you can check by hand, drawn, and built in plain Python you can run.
Choose a path
Every lesson in reading order, one line per part. Pick who you are to see the lessons meant for you, in order.
Every lesson
Each one stands on the ones before it. Every part opens with what it answers and what you can do afterwards; the symbols reference at the end is for whenever a formula stops you.
Part 1: how a language model works
The machinery inside a language model: a neural network, the loss and the optimizer that train it, how text becomes tokens, and the attention and transformer blocks that turn them into a reply. If you read one lesson here besides the first, read Attention.
- 01What happens when you send a promptFrom your words to numbers, through the layers, to a reply one word at a timeFully illustrated
- 02Neural networksHow a neural network, a program made of simple units, learns from its mistakesFully illustrated
- 03Loss functionsThe one number training pushes down, and why choosing it is choosing the behaviourFully illustrated
- 04OptimizersHow a neural network turns the slope of its loss into a step, and how to set the learning rateFully illustrated
- 05Training deep neural networksWhy deep stacks stop learning, and the fixes inside every transformer blockFully illustrated
- 06AttentionHow each word looks at the others, and why the cost grows with the square of the lengthFully illustrated
- 07Positional informationWhy order must be added, sinusoids and RoPEFully illustrated
- 08The transformerThe block every language model stacks, and how to turn a config file into gigabytesFully illustrated
- 09TokenizationHow text becomes the pieces a language model reads, and why language models miscount lettersFully illustrated
Part 2: how it is trained and judged
How a language model gets its weights: pretraining on a large share of the web, then fine-tuning, reward training, reasoning and alignment. Along the way it covers how to judge the result: overfitting, metrics and benchmark scores. In a hurry, read Training stages and Reading benchmarks.
- 10Training stagesThe stages that turn a random neural network into a chat assistant, and when to fine-tune instead of promptFully illustrated
- 11Pretraining at scaleWhat it takes to train on the internet: the data belt, parallelism across GPUs, mixed precisionFully illustrated
- 12Fine-tuning in practiceWhat a fine-tune costs and risks, and how to prepare data so the language model keeps its old skillsFully illustrated
- 13Overfitting and regularizationWhy an AI model memorises instead of learning, and the checks that catch itFully illustrated
- 14Reinforcement learningHow an AI model learns from a score instead of an answer, and how it games the scoreFully illustrated
- 15Reasoning modelsWhy letting a language model think before it answers helps, and when the extra tokens are worth itFully illustrated
- 16Alignment and safetyHow a language model is taught to behave, and why a score that keeps rising can hide a worse oneFully illustrated
- 17MetricsThe numbers that decide whether an AI model ships, and the one that fools you on rare eventsFully illustrated
- 18Reading benchmarksWhat a benchmark score can and cannot tell you, and how wide its error bar isFully illustrated
Part 3: how it is run
What it takes, in time and memory, to turn a trained model into answers: the chip, the cache, the sampling step and the designs that make long inputs affordable. Latency and GPU bills follow from arithmetic you can do by hand. The part ends inside a trained model, with what it learned.
- 19The hardware underneathWhy a GPU is fast at some work and slow at the rest, and how to read a spec sheetFully illustrated
- 20InferenceWhy the first token is slow and the rest are cheap, and what memory a conversation costsFully illustrated
- 21Structured outputHow to guarantee valid JSON by controlling which tokens a language model may pickFully illustrated
- 22Long context and efficient architecturesHow a language model reads a million tokens without paying for a million squaredFully illustrated
- 23Looking inside the modelHow to read the numbers inside a trained AI model and find what a layer representsFully illustrated
Part 4: embeddings and search
Embeddings are lists of numbers that place similar meanings close together. This part builds them, then the search systems that use them to find, among millions of passages, the few that answer a question. The retrieval systems in Part 6 are built on it.
- 24Word embeddingsHow a word becomes a vector that captures its meaning, and what that vector cannot captureFully illustrated
- 25SimilarityHow to compare two embeddings, and why the wrong metric ranks silently wrongFully illustrated
- 26Training embedding modelsHow an embedding model is taught what similar means, from pairs of texts that belong togetherFully illustrated
- 27Making embeddings smallerHow to store millions of embeddings in a fraction of the memory without losing your neighboursFully illustrated
- 28Vector indexesHow an index finds the nearest vectors among millions without checking every oneFully illustrated
- 29RetrievalThe whole search pipeline: keywords, embeddings, hybrid ranking, rerankers and chunkingFully illustrated
- 30Clustering and matchingHow embeddings group a pile of unlabelled text, find duplicates and route requestsFully illustrated
- 31Embeddings in productionWhat breaks when an embedding index serves real traffic, and how to measure retrieval on its own
Part 5: other kinds of model, and generating media
Models that are not transformers, and models that make pictures and sound: trees for tables of data, convolutions for images, and the autoencoders, GANs and diffusion models behind generated images, audio and video. It ends with models that see and hear. Skip it if your work is text on a hosted model.
- 32Decision trees and boostingHow a tree learns a table of data, and why boosted trees still beat neural networks there
- 33CNNs and RNNsHow convolutions see and recurrent nets remember, and why transformers won
- 34Autoencoders and VAEsSqueezing data into a code and back, and sampling new data from it
- 35GANsHow two neural networks trained against each other learn to make realistic samples, and why the training is unstable
- 36Diffusion and flow matchingTurning noise into images one small step at a time
- 37Multimodal modelsHow pictures and sound become tokens a language model can read, and what each costs in context
Part 6: building systems people rely on
How to build on a language model: the message exchange, the agent loop, tools, retrieval, context and memory. Then how to know it works and keep it safe: evaluation, guardrails, cost, observability and deployment. The part ends with why hard tasks still fail.
- 38Talking to a language modelThe message format a language model reads, and how tool calling works
- 39OrchestrationWhen to script the steps and when to let the language model choose them, and the named patterns in between
- 40The agent loopThe loop that runs a language model's errands, with the limits that let it stop
- 41ToolsHow to design a tool so a wrong call is caught before it costs money
- 42Coding and computer-use agentsWhy agents became dependable on code first, and how one drives a repository or a screen
- 43Model Context ProtocolHow one tool can serve every application, what goes over the wire, and the risks of a stranger's tools
- 44Retrieval-augmented generationHow a language model answers from your documents, with citations and access controlFully illustrated
- 45Context engineeringWhat goes in the prompt when code assembles it, and why an agent gets worse as a session runs
- 46Giving an AI agent a memoryHow an agent carries what it learned across conversations, and keeps one user's memory from another's
- 47How an AI agent plans a long taskWhy a twenty-step task fails when a three-step demo worked, and how a plan survives it
- 48EvaluationHow to know a change helped before a customer finds out that it did not
- 49GuardrailsHow to keep text a language model reads from becoming commands it follows, and check what it sends out
- 50Cost and latencyThe one number that decides cost, and the levers that move it
- 51ObservabilityHow to find out what an agent did yesterday, and turn that record into the next improvement
- 52Safe deploymentHow an agent earns the right to act, one step of trust at a time
- 53Why AI agents fail in productionThe common failure modes, and the fix for each
Reference: reading the symbols
Every symbol an AI formula uses (Σ, ‖x‖, ∂, ᵀ), decoded as a short loop with numbers you can check by hand. It is not the first lesson: open it when a formula stops you, from the link beside any formula.