Attention gives each word one unit to divide across the sentence, and a row shows where that word spends it.
Question: How does a model choose which words matter?
Synthetic Synthetic: the seven relevance scores behind this heatmap are set by hand, not learned from data, and the sharpness knob is an illustration, not a parameter of a trained model. The weights are computed exactly from those scores by data/scripts/c21.s03.L1.py. The rule they follow is the textbook one: attention weights are positive and sum to one in every row [S:74:224-225].
Frame 1 of 1: it · sharpness 0.60At the default setting the row is already lopsided. The row for it divides one unit of attention over all seven words at sharpness 0.60. The hot cell is cat at 0.316; the cold cell is because at 0.066. An even split would give every word 0.143.
it · sharpness 0.60. At the default setting the row is already lopsided. The row for it divides one unit of attention over all seven words at sharpness 0.60. The hot cell is cat at 0.316; the cold cell is because at 0.066. An even split would give every word 0.143.
Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.
Text description of this lab
A seven-by-seven grid of attention weights for the sentence The cat sat because it was tired. Rows are the word doing the looking, columns the word being looked at: a darker cell holds a larger share, every cell prints its share, and every row splits one unit. The chosen row is outlined, its hottest column is boxed, and a bar chart repeats that row. The sharpness control moves the row from almost even to nearly one-hot.