The three routes to a gradient are priced by formula length, input count and operation count, so their costs cross over as the model grows.
Question: When do you use symbols, numbers or a program?
Schematic Schematic: the counts are exact arithmetic for the toy program L = w_1 * w_2 * ... * w_n under stated counting rules — symbolic n(n-2), numeric 2n(n-1) centred and n(n-1) one-sided, automatic 3(n-1) multiplications — but they model work, not measured time. No stopwatch was run and no library was profiled. The chart plots those counts on log–log axes; the size control moves the marker and the numeric-step control halves that route.
Frame 1 of 6: Three routes at n = 9At 9 weights the formula route needs 63 multiplications, the numeric route 144 centred (72 one-sided), and the program route 24. The vertical line marks this size; both axes are logarithmic.
Three routes at n = 9. At 9 weights the formula route needs 63 multiplications, the numeric route 144 centred (72 one-sided), and the program route 24. The vertical line marks this size; both axes are logarithmic.
Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.
Text description of this lab
Log–log chart of multiplications against model size, 3 weights to 1,000,000. Three lines: navy solid symbolic, russet dashed numeric, gold dotted automatic. The symbolic and numeric lines climb twice as steeply as the automatic line. A marker at 9 weights reads 63 multiplications for the formula route, 144 centred or 72 one-sided for the numeric route, and 24 for the program route. The formula route is cheapest up to 4 weights, the program route from 5.