Derive: Averages, Spread and Surprises

Put ${\mu }_X=E(X)$ and ${\mu }_Y=E(Y)$. By the definition in section 12.2, $$\mathrm{Var}(X+Y)=E\left[{\left((X-{\mu }_X)+(Y-{\mu }_Y)\right)}^2\right].$$ Expand the square: $$E\left[(X-{\mu }_X{)}^2\right]+E\left[(Y-{\mu }_Y{)}^2\right]+2E\left[(X-{\mu }_X)(Y-{\mu }_Y)\right].$$ The first two terms are the two variances. For the cross term, independence gives $E[XY]=E(X)E(Y)$, so $$E\left[(X-{\mu }_X)(Y-{\mu }_Y)\right]=E(XY)-{\mu }_X{\mu }_Y=0.$$ Therefore $\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)$. Applying the result $n-1$ times to independent values, $$\mathrm{Var}(X_1+\cdots +X_n)=n{\sigma }^2.$$ The average is the sum multiplied by the constant $1/n$, and multiplying a variable by a constant multiplies its variance by the square of that constant: $$\mathrm{Var}(\bar{X})=\mathrm{Var}\left(\frac{X_1+\cdots +X_n}{n}\right)=\frac{1}{n^2}n{\sigma }^2=\frac{{\sigma }^2}{n}.$$ The standard error is the square root, $\sigma /\sqrt{n}$, which is exactly the formula section 12.3 quotes. At $\sigma =12$ and $n=36$ the variance is $144/36=4$, so the standard error is $\sqrt{4}=2$. The derivation shows why the rate is a square root: the sum's variance grows with $n$, and dividing by $n$ divides the variance by $n^2$, leaving one power of $n$ in the denominator.

The count $S$ of heads in four tosses has mean $4\times \frac{1}{2}=2$ and variance $4\times \frac{1}{2}\times \frac{1}{2}=1$. The event $S\ge 3$ is a gap of at least 1 from the mean, so Chebyshev gives $$P(S\ge 3)\le \frac{1}{1^2}=1.$$ The exact count is five sequences out of sixteen: the four sequences with exactly one tail, and the sequence HHHH. Hence $$P(S\ge 3)=\frac{4+1}{16}=\frac{5}{16}=\mathrm{0.3125.}$$ The bound of 1 is satisfied, and it is empty: every probability is at most 1, so the inequality tells you nothing here. The bound $4/n$ falls below 1 at $n=8$, where it equals $0.5$, and the exact value at $n=8$ is $37/256=0.145$. The edge case matters because a bound at or above 1 is not evidence of safety, and section 12.5 makes the same warning in general form. Chebyshev is never false; at small $n$ it is silent.

The diagonal entries are the variances, so $a={\sigma }_x^2=9$ and $b={\sigma }_z^2=16$. The off-diagonal entry is the covariance $c=\rho {\sigma }_x{\sigma }_z=12\rho$. The determinant is $$ab-c^2={\sigma }_x^2{\sigma }_z^2-{\rho }^2{\sigma }_x^2{\sigma }_z^2={\sigma }_x^2{\sigma }_z^2(1-{\rho }^2).$$ The two variances are positive, so the determinant has the sign of $1-{\rho }^2$. Positive definiteness asks for a positive determinant, which gives $$1-{\rho }^2>0⟺|\rho |<1.$$ That is the derivation behind the sentence in section 12.6: the correlation is bounded because a larger value would make the determinant negative, and a negative determinant describes a shape that cannot be a probability density. At $\rho =0.95$, $$9\times 16\times (1-0.95^2)=144\times 0.0975=14.04,$$ so the matrix is still positive definite and the ellipse is narrow. At $\rho =1$ the determinant is zero, the matrix is only positive semidefinite, and the cloud collapses onto a line: one measurement becomes an exact straight-line function of the other. At $\rho =-1$ the same collapse happens with the opposite slope.