c01.s01
A recipe with a promise
What does a program promise you?
Reader
Read
Every section carries three layers at stable anchors: #core, #deepenand #research.
Chapter 1
1. Rules That Turn Inputs Into Outputs
What does a program promise you?
c01.s02
The same answer, two ways
When does repeating a step beat doing it once?
c01.s03
Functions that take functions
Why would a program accept another program?
c01.s04
Building with boxes
How do you use a part without reading its insides?
c01.s05
When code becomes data
What is the difference between a program and the thing it runs?
c01.s06
Testing the promise
How do you know a program kept its contract?
Chapter 2
2. Learning a Rule From Examples
What must a table of examples carry for prediction to be possible?
c02.s01
A table of examples
What must a table of examples carry for prediction to be possible?
c02.s02
The nearest neighbour
How can the closest example be the answer?
c02.s03
Measuring the mistake
What does one wrong answer cost?
c02.s04
The wall between train and test
Why must the test examples be kept away?
c02.s05
Choosing k
When is one neighbour too few?
c02.s06
What learning will mean
What is the learning problem, stated once and for all?
Chapter 3
3. How to Know a Claim Is True
What does "for all" commit you to?
c03.s01
Claims with quantifiers
What does "for all" commit you to?
c03.s02
Direct proof and contraposition
How does a proof move from assumptions to conclusion?
c03.s03
Contradiction and induction
When does assuming the opposite — or the previous case — help?
c03.s04
Sets, functions and relations
Why does a proof need sets and functions?
c03.s05
Counting and diagonalisation
How do you prove that something cannot be listed?
c03.s06
The five proof errors
Which mistakes look like proofs?
Chapter 4
4. Experiments That Do Not Lie
How much does one run tell you?
c04.s01
A result and its luck
How much does one run tell you?
c04.s02
Leakage
How does the answer sneak into the input?
c04.s03
Ablation
What happens when you remove one part?
c04.s04
Logging and provenance
How do you know what you ran?
c04.s05
A one-file experiment harness
What does a reproducible experiment look like?
c04.s06
Plotting for honesty
Which graph tells the truth?
Chapter 5
5. How Long Does It Take?
How do we measure the work a program does?
c05.s01
Counting the steps
How do we measure the work a program does?
c05.s02
The growth ladder
What does the growth notation hide?
c05.s03
Divide and conquer
How does a recurrence predict the cost?
c05.s04
The sorting wall
Why can comparison sorting not beat n log n?
c05.s05
Remembering answers
When does saving a result beat recomputing it?
c05.s06
When no fast algorithm is known
What does "hard" mean for a problem?
Chapter 6
6. The Real Machine
How is a decimal number stored in 32 bits?
c06.s01
Bits and floats
How is a decimal number stored in 32 bits?
c06.s02
Rounding and cancellation
Why does subtracting two correct numbers give a wrong answer?
c06.s03
Stable softmax
How do you compute a probability without overflowing?
c06.s04
The memory hierarchy
Why does the same arithmetic run at ten speeds?
c06.s05
Parallel machines
What does a GPU do differently?
c06.s06
Mixed precision
When can you use fewer bits on purpose?
Chapter 7
7. Lists of Numbers That Do Things
What makes a list of numbers a vector?
c07.s01
Vectors and the spaces they live in
What makes a list of numbers a vector?
c07.s02
Span, basis and dimension
How few vectors do you need to build all the others?
c07.s03
A matrix is a machine
What does a matrix do to a vector?
c07.s04
Kernel and image
Which inputs vanish, and which outputs are reachable?
c07.s05
Solving a system
How does elimination find every solution?
c07.s06
The four subspaces
Which four spaces does a matrix define?
Chapter 8
8. Projection and the Best Fit
How do you measure the angle between two lists?
c08.s01
Dot products and angles
How do you measure the angle between two lists?
c08.s02
The shadow of a vector
What is the closest point in a subspace?
c08.s03
Orthonormal bases and QR
Why is an orthogonal basis easier to use?
c08.s04
The normal equations
Where does the least-squares formula come from?
c08.s05
The pseudoinverse
What is the best answer when no exact solution exists?
c08.s06
Residuals and the first overfit
When does a good fit become a bad model?
Chapter 9
9. Directions That Stay Put
Which directions does a matrix leave alone?
c09.s01
Eigenvalues and eigenvectors
Which directions does a matrix leave alone?
c09.s02
Diagonalisation and dynamics
What happens when you apply the same matrix many times?
c09.s03
The power method
How do you find the dominant direction by repeating?
c09.s04
The spectral theorem
Why can a symmetric matrix be rotated into a diagonal one?
c09.s05
When is a quadratic form a bowl?
When is a quadratic form a bowl?
c09.s06
The SVD
What does every matrix do, in three steps?
Chapter 10
10. Change, Slopes and Approximation
What does it mean to get arbitrarily close?
c10.s01
Limits and continuity
What does it mean to get arbitrarily close?
c10.s02
The derivative as a best linear map
What is the slope of a curved thing?
c10.s03
Taylor approximation and curvature
How do you replace a curve with a polynomial?
c10.s04
Gradients in many dimensions
Which way is uphill?
c10.s05
The chain rule is backpropagation
How does a change in one input reach the output?
c10.s06
Checking a derivative
How do you know the formula is right?
Chapter 11
11. Probability as Reasoning
What rules must a degree of belief obey?
c11.s01
Uncertainty as extended logic
What rules must a degree of belief obey?
c11.s02
Sample spaces and counting
How many outcomes are there?
c11.s03
Conditional probability
How does new information change the count?
c11.s04
Bayes' rule
How do you turn a likelihood into a belief?
c11.s05
Independence and base rates
When does one event tell you nothing about another?
c11.s06
Two readings of probability
Are probabilities frequencies or beliefs?
Chapter 12
12. Averages, Spread and Surprises
How do you attach a number to an outcome?
c12.s01
Random variables
How do you attach a number to an outcome?
c12.s02
Expectation and variance
What is the centre, and how wide is the spread?
c12.s03
The law of large numbers
Why does an average settle down?
c12.s04
The central limit theorem
Why does the bell curve keep appearing?
c12.s05
Tails and concentration
How surprising can a large deviation be?
c12.s06
The multivariate Gaussian
What does a cloud of correlated numbers look like?
Chapter 13
13. Estimating From Data
What makes one guess better than another?
c13.s01
What an estimator is
What makes one guess better than another?
c13.s02
Bias, variance and MSE
Where does the error in an estimate come from?
c13.s03
Maximum likelihood
Which parameter makes the data most likely?
c13.s04
How much can one observation tell you?
How much does one observation tell you?
c13.s05
Consistency and asymptotics
What happens to an estimate as the sample grows?
c13.s06
The bias–variance trade
Why does a slightly wrong model sometimes predict better?
Chapter 14
14. Lines, Curves and Boundaries
What does the best straight line mean?
c14.s01
Fitting a line
What does the best straight line mean?
c14.s02
When the line is wrong
Which assumptions does linear regression make?
c14.s03
Ridge and shrinkage
Why shrink the coefficients?
c14.s04
Lasso and sparsity
How does a penalty delete features?
c14.s05
Logistic regression
How does a line become a probability?
c14.s06
A family of boundaries
What do all generalised linear models share?
Chapter 15
15. Judging a Model Honestly
How optimistic is a model about itself?
c15.s01
Why training error lies
How optimistic is a model about itself?
c15.s02
Validation and cross-validation
How do you reserve data without wasting it?
c15.s03
Leakage and contamination
How does a test set become a training set by accident?
c15.s04
The bootstrap
How do you get an error bar from one sample?
c15.s05
What a p-value means
What does a significance test actually test?
c15.s06
Model criticism
How do you look for the way a model is wrong?
Chapter 16
16. Information and Compression
How much information is in one symbol?
c16.s01
Surprise and entropy
How much information is in one symbol?
c16.s02
Joint, conditional and relative entropy
How do entropies combine?
c16.s03
Mutual information
What does one variable tell you about another?
c16.s04
Huffman coding
How do you build the shortest code?
c16.s05
The source coding theorem
Why can no code beat entropy?
c16.s06
Channels and capacity
How fast can you send a message through noise?
Chapter 17
17. Going Downhill on Purpose
How does a gradient point to a better answer?
c17.s01
Which way is downhill?
How does a gradient point to a better answer?
c17.s02
Choosing a step size
What happens when the step is too small or too large?
c17.s03
What does a bound on the curvature buy you?
What does a bound on the curvature buy you?
c17.s04
Convergence under convexity
How fast does descent reach the bottom?
c17.s05
Stochastic gradients
What if each step sees only one example?
c17.s06
Schedules and batch size
How do you set the step size over a long run?
Chapter 18
18. Getting Gradients From a Computer
When do you use symbols, numbers or a program?
c18.s01
Three ways to differentiate
When do you use symbols, numbers or a program?
c18.s02
Forward mode
How does one input direction propagate?
c18.s03
Reverse mode
How does one output direction propagate backwards?
c18.s04
The tape and the graph
What does the computer remember?
c18.s05
Memory and checkpointing
How do you train when the graph does not fit?
c18.s06
Where autodiff lies
Which functions break the derivative?
Chapter 19
19. Neural Networks: Simple Parts, Stacked
How does one unit draw a line?
c19.s01
The perceptron
How does one unit draw a line?
c19.s02
XOR and the limit of one layer
What can one unit never do?
c19.s03
One hidden layer
How does a second layer bend the boundary?
c19.s04
Activation functions
Why does a nonlinearity change everything?
c19.s05
Backprop by hand
How do the gradients reach every weight?
c19.s06
The training loop
What does a network actually learn?
Chapter 20
20. Training Deep Networks Without Breaking Them
Where does the gradient go?
c20.s01
Why deep networks stop learning
Where does the gradient go?
c20.s02
Initialisation
How do you start so that signals survive?
c20.s03
Normalisation and regularisation
Why rescale the activations, and when should you drop units?
c20.s04
Residual connections
How does a skip connection make depth trainable?
c20.s05
What do momentum and Adam add?
What do momentum and Adam add to plain descent?
c20.s06
Debugging a training run
Which loss curve means which bug?
Chapter 21
21. Attention, Images and Sequences
Why does looking at neighbours help?
c21.s01
Neighbours and weight sharing
Why does looking at neighbours help?
c21.s02
Recurrence and its gradient
What happens when a network reads one word at a time?
c21.s03
Attention
How does a model choose which words matter?
c21.s04
Queries, keys and values
What is the arithmetic of attention?
c21.s05
The transformer block
How do attention and a small network fit together?
c21.s06
Why convolution and attention differ
Does attention replace convolution?
Chapter 22
22. Making Pictures Out of Noise
What does it mean to learn a distribution?
c22.s01
What a generative model is
What does it mean to learn a distribution?
c22.s02
How do you learn a hidden cause?
How do you learn a hidden cause?
c22.s03
The variational autoencoder
How do you train a model that samples?
c22.s04
Diffusion
How does removing noise create an image?
c22.s05
Sampling from a generative model
How do you draw a new sample?
c22.s06
Flow matching and evaluation
How do you compare two generators?
Chapter 23
23. Learning by Acting
What is different about learning from reward?
c23.s01
The agent–environment loop
What is different about learning from reward?
c23.s02
Rewards and returns
How do you compare two futures?
c23.s03
What does the world need to remember?
What does the world need to remember?
c23.s04
How does a value summarize the future?
How does a value summarize the future?
c23.s05
Learning values from experience
How do you compute the best policy when you know the world — and when you do not?
c23.s06
Policy gradients
How do you improve a policy without knowing the world?
Chapter 24
24. Scale, Language Models and What They Learned
How does a model learn to predict the next token?
c24.s01
Tokens and language modelling
How does a model learn to predict the next token?
c24.s02
Pretraining
What does a model learn from, and what objective does it use?
c24.s03
Scaling laws
What happens when you multiply data, parameters and compute?
c24.s04
What the model learned
What is inside a hidden layer?
c24.s05
Interpretability
Can we explain a prediction?
c24.s06
Alignment
How could a powerful model be aimed at the wrong thing?
Fixture. Only chapter 1 is loaded in this build. The exporter writessrc/data/chapters/cNN.json from ../book/chapters/cNN.md for each merged chapter.