Deeper

Lab · c17.s01.L1

Gradient descent 2D

The negative gradient is the steepest downhill direction at the current point, but on a stretched bowl that local arrow leads across the contours, not straight to the bottom.

Question: How does a gradient point to a better answer?

Gradient descent 2DA stretched bowl from above: five nested grey ellipses around the lowest point, labelled 0.5, 1, 2, 4 and 6.5. A navy path steps downhill from the upper right, lowering the loss at every step. A russet arrow shows the negative gradient there, crossing the contours at a right angle. The path zigzags across the long axis because the arrow is steepest only where it stands. A side panel prints the current point, loss and gradient.Gradient descent 2D0.51246.5-3.73.7-2.02.0f(x, y) = 0.5 * (x^2 + 4 y^2)grad f(x, y) = (x, 4 y)step size eta = 0.45; arrows show the negative gradient, length 0.45Current pointpoint (x, y)loss fgradientsteparrow = -grad farrow length 0.45(2.000, 1.500)6.5000(2.00, 6.00)0 of 20
Re-simulated Re-simulated: 20 fixed-size gradient steps of step size 0.45 on f(x, y) = 0.5 * (x^2 + 4 y^2), computed for every start on the control grid. The five ellipses are the exact level sets 0.5, 1, 2, 4 and 6.5; each frame adds one recorded point and the negative-gradient arrow at the current point.
Lab controls
2

Arrow keys move one stop (0.5); Page Up and Page Down move faster; Home sets -2.0, End sets 2.0

1.5

Arrow keys move one stop (0.5); Page Up and Page Down move faster; Home sets -1.5, End sets 1.5

Frame 1 of 6: StartStart at (2.0, 1.5) with loss 6.5000. The gradient is (2.00, 6.00), so the russet arrow points across the grey contours at a right angle: the steepest downhill direction at this point, and not at the centre of the bowl.

Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.

Text description of this lab

A stretched bowl from above: five nested grey ellipses around the lowest point, labelled 0.5, 1, 2, 4 and 6.5. A navy path steps downhill from the upper right, lowering the loss at every step. A russet arrow shows the negative gradient there, crossing the contours at a right angle. The path zigzags across the long axis because the arrow is steepest only where it stands. A side panel prints the current point, loss and gradient.

Return to the section that hosts this lab

Caption status: re-simulated. Every lab declares one of published, re-simulated, schematic, synthetic or interpolated.