Shrinking an estimate toward zero trades a small, growing bias for a larger, shrinking variance.
Question: Why does a slightly wrong model sometimes predict better?
Re-simulated Re-simulated: the three error curves are the exact bias², variance and MSE of the shrunk mean c × sample mean for n Gaussian observations with mean 1.00 and standard deviation 1.00, and the dot strip is a seeded simulation of 100 replications; data/scripts/c13.s06.L1.py reproduces both. At n = 10 the MSE curve bottoms out at c = 0.909 with MSE 0.091 against 0.100 at c = 1.00, a 9.1% gain; at n = 100 the best c is 0.990 and the gain is 1.0%. The slider covers the full 25-point c grid at n = 10, and the frames show the honest and best stored settings for n = 2, 5, 20, 50 and 100.
Frame 1 of 8: The honest setting, c = 1.00No shrinkage: the estimate is the sample mean, so the bias is 0.000 and the variance is 0.100. MSE is 0.100, and the dotted line marks the best c = 0.909, where MSE is 0.091.
The honest setting, c = 1.00. No shrinkage: the estimate is the sample mean, so the bias is 0.000 and the variance is 0.100. MSE is 0.100, and the dotted line marks the best c = 0.909, where MSE is 0.091.
Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.
Text description of this lab
Error against shrinkage c, 0.00 to 1.20, at ten observations. The navy MSE curve starts at 1.000, bottoms out at 0.091 at c = 0.909, then rises; the russet bias² curve grows as c falls, the gold variance curve shrinks. A strip of one hundred simulated estimates clusters near 0.90 rather than the true mean 1.00, and the readout gives bias −0.100, variance 0.081 and MSE 0.091.