Practise: Estimating From Data

The population mean is $(1+2+3+4+5)/5=3$. The squared deviations from 3 are 4, 1, 0, 1 and 4. Their average is $(4+1+0+1+4)/5=2$, so the population variance is 2. The average of four independent draws has variance $2/4=0.5$. Its bias is zero, because the average of the four draws has expected value 3. Section 13.2 gives $\mathrm{MSE}=\mathrm{Var}+(\text{bias}{)}^2$, so the mean squared error is $0.5+0=0.5$. You can also enumerate all $5^4=625$ ordered four-tuples; the average of their 625 squared errors is $1/2$. The rule beats reporting one reading, whose mean squared error is 2, by a factor of four.

For P, the mean squared error is $12+(-2{)}^2=12+4=16$. For Q, it is $14+1^2=14+1=15$. Q has the smaller mean squared error, so the better rule is Q, at 15. P is wrong in a larger systematic way: its average guess sits 2 units below the truth. Q is wrong by only 1 unit on average, and that smaller offset more than pays for its larger variance. The comparison uses the sampling distribution of each rule across repeated experiments, not one estimate from either rule.

The sum of squared deviations from the sample average has expected value $(n-1){\sigma }^2$. The rule divides that sum by $n$, so its expected value is $\frac{n-1}{n}{\sigma }^2$. At $n=8$ the factor is $7/8=0.875$. The rule averages 87.5 percent of the true variance, a shortfall of $1/8=12.5$ percent. Section 13.2 notes that this biased rule has smaller variance than the unbiased rule that divides by $n-1$, so its mean squared error can be lower even though its centre is wrong. The factor approaches 1 as $n$ grows, which is one reason the bias matters less in large samples.

The likelihood of any fixed sequence with 9 heads and 3 tails is $p^9(1-p{)}^3$, because the order does not change the product. At $p=0.75$ this is $(3/4{)}^9(1/4{)}^3$. At $p=0.5$ it is $(1/2{)}^{12}$. The ratio is $$\frac{(3/4{)}^9(1/4{)}^3}{(1/2{)}^{12}}=\frac{3^9/4^{12}}{1/2^{12}}=\frac{3^9}{2^{12}}=\frac{19683}{4096}=\mathrm{4.805.}$$ To two decimal places the ratio is 4.81, so the data favour 0.75 over 0.5 by about 4.8 to 1. The maximum likelihood estimate is $9/12=0.75$, as section 13.3 says for $x$ heads in $n$ tosses. The ratio compares two candidates; it is not the probability that either candidate is right.

Section 13.4 gives $\mathrm{se}(\hat{p})=\sqrt{p(1-p)/n}$. Put $p=0.4$ and $n=25$. The numerator is $0.4\times 0.6=0.24$. Divide by 25 to get $0.0096$. The square root is $0.09798$, which rounds to $0.098$. The same section reports the Fisher information of one toss as $1/(p(1-p))=1/0.24=4.167$ inverse-variance units, so 25 tosses carry $25\times 4.167=104.167$ units. The standard error is the inverse square root of that total, which is the same $0.098$. The interval is not the estimate's whole uncertainty; it assumes independent tosses.

The standard error is $\sqrt{p(1-p)/n}$. Set it equal to $0.01$ and square: $$0.0001=\frac{0.3\times 0.7}{n}=\frac{0.21}{n}.$$ Multiply by $n$ and divide by $0.0001$: $n=0.21/0.0001=2100$. The answer is a whole number, so no rounding is needed. Check it by putting $n=2100$ back into the formula: $0.21/2100=0.0001$, whose square root is $0.01$. The square-root rate is why this is larger than a reader might guess: to shrink the standard error from $0.1$ at 21 tosses to $0.01$, you need one hundred times the data, not ten times.

Model A's expected error is $2+1^2+6=9$. Model B's expected error is $2+3^2+1=12$. The smaller value is 9, so Model A predicts better even though Model B has the smaller variance. Model B pays 9 units for its bias, and that is more than the 5 units it saves in variance. The noise term, 2, is common to both and sets a floor that neither model can cross. The comparison holds at this input and for squared error; under a different loss the ranking can change.

The mean is $(1+2+30)/3=33/3=11$. Its squared error is $(11-2{)}^2=9^2=81$. The sorted readings are 1, 2 and 30, so the median is 2. Its squared error is $(2-2{)}^2=0$. The worse rule on this dataset is the mean, at 81, and the median is exact. One dataset does not settle which rule is better: section 13.1 shows that a single large reading pulls the mean, while a population with a different shape can favour the mean. You judge the rules by their sampling distributions, and this dataset is one draw from that distribution.