Practise: Projection and the Best Fit

The dot product is $1\times 2+2\times 1=2+2=4$. The first norm is $\sqrt{1^2+2^2}=\sqrt{5}$, and the second is $\sqrt{2^2+1^2}=\sqrt{5}$, so the product of the norms is $\sqrt{5}\sqrt{5}=5$. The cosine is $4/5=0.8$. The angle is $\arccos (0.8)\approx 36.9^{\circ }$, while the shoppers' cosine of 0.96 gives about $16.3^{\circ }$, so this pair agrees less than the two baskets of section 8.1. The two lists hold the same two numbers in the other order, and the score below 1 measures that difference. A cosine of 0.8 is a strong positive score: the scale runs from $-1$ to $1$, so 0.8 sits close to the top and says the lists point mostly the same way.

Two lists are orthogonal when their dot product is zero, because the denominator of the cosine is a product of lengths and never zero for a nonzero list. The dot product here is $2c+12$. Setting it to zero gives $2c=-12$, so $c=-6$. The check is $2\times (-6)+3\times 4=-12+12=0$, and the cosine is 0, so the angle is $90^{\circ }$. Once the second entry is fixed at 4, the first entry is forced: every nonzero multiple of $(-6,4)$ is also perpendicular to $(2,3)$, but only $c=-6$ keeps the second entry at 4.

The direction is $\mathbf{s}=(1,2)$ and the point is $\mathbf{b}=(2,5)$. The dot product is $2\times 1+5\times 2=12$, and the direction with itself is $1^2+2^2=5$, so the multiplier is $12/5=2.4$. The projection is $2.4\times (1,2)=(2.4,4.8)$. The error is $(2-2.4,5-4.8)=(-0.4,0.2)$. Its dot product with the direction is $-0.4\times 1+0.2\times 2=0$, so the error is perpendicular to the line, as section 8.2 requires. The distance is the length of the error, $\sqrt{0.16+0.04}=\sqrt{0.2}=0.447$. Every other point of the line sits farther away, because the step from the shadow to that point is perpendicular to the error and Pythagoras adds a positive square.

The first coordinate is $x\cdot q_1=6/5+4/5=10/5=2$. The second is $x\cdot q_2=-8/5+3/5=-5/5=-1$. The rebuild is $2q_1-q_2=(6/5+4/5,8/5-3/5)=(2,1)$, which returns the list exactly. The check by Parseval is that the squared coordinates add to the squared length: $2^2+(-1{)}^2=5$, and $2^2+1^2=5$. The two basis lists have length 1 and dot product $3/5\times (-4/5)+4/5\times 3/5=0$, so they qualify. Reading the coordinates as dot products is valid only because of that perpendicularity; in the slanted basis of section 8.3 the same reading returns 6 where the true coordinate is 1.

The two column sums are 3, the sum of the squared times is $0+1+4=5$, the sum of the readings is $3+5+6=14$, and the sum of time times reading is $0\times 3+1\times 5+2\times 6=17$. The normal equations are $3c_0+3c_1=14$ and $3c_0+5c_1=17$. Subtracting the first from the second leaves $2c_1=3$, so the slope is $c_1=3/2$. Substituting back gives $3c_0=14-9/2=19/2$, so the intercept is $c_0=19/6$. The predictions are $19/6$, $14/3$ and $37/6$, and the residuals are $-1/6$, $1/3$ and $-1/6$. The residual sums to $-1/6+1/3-1/6=0$, and its dot product with the time column is $0\times (-1/6)+1\times (1/3)+2\times (-1/6)=1/3-1/3=0$, so the residual is perpendicular to both columns. The residual sum of squares is $1/36+1/9+1/36=1/6$.

The first column of the table marks the rows that contain object 1, which are the first and third, so its sum of squares is 2 and its product sum with the readings is $2+6=8$. The second column marks rows two and three, giving 2 and $3+6=9$. The cross product of the two columns is 1, because only the third row contains both objects. The normal equations are $2w_1+w_2=8$ and $w_1+2w_2=9$. Subtracting gives $w_1-w_2=-1$, so $w_1=w_2-1$; substituting into the first gives $2(w_2-1)+w_2=8$, so $3w_2=10$ and $w_2=10/3$. Then $w_1=7/3$. The predictions are $7/3$, $10/3$ and $17/3$, the residuals are $-1/3$, $-1/3$ and $1/3$, and the residual sum of squares is $1/3$. Each residual dot product with a column is zero, which is the check of section 8.5. The third reading, 6, is larger than the sum of the first two, so both weights are pulled above the values that would satisfy the single-object readings exactly.

The centered times are $-1.5$, $-0.5$, $0.5$ and $1.5$, whose squares sum to $2.25+0.25+0.25+2.25=5$. The centered readings are $-0.5$, $0.5$, $-0.5$ and $0.5$. The product sum is $(-1.5)(-0.5)+(-0.5)(0.5)+(0.5)(-0.5)+(1.5)(0.5)=0.75-0.25-0.25+0.75=1$, so the slope is $1/5=0.2$. The intercept is $0.5-0.2\times 1.5=0.2$. The predictions at times 0, 1, 2 and 3 are 0.2, 0.4, 0.6 and 0.8, and the residuals are $-0.2$, $0.6$, $-0.6$ and $0.2$. The residual sum of squares is $0.04+0.36+0.36+0.04=0.8$. The alternating signs are the pattern section 8.6 reads from a residual plot: the line misses the low and high readings in a repeating way, so a curve with more columns can lower this number. A cubic through the same four rows scores zero, which is why the number alone does not settle the choice of model.

No. The line is the set of multiples of $(1,1)$, and $(5,6)$ has unequal entries, so it does not lie on the line at all. The true multiplier is $(4\times 1+6\times 1)/(1^2+1^2)=10/2=5$, so the projection is $(5,5)$, and the error is $(4-5,6-5)=(-1,1)$, whose dot product with $(1,1)$ is $-1+1=0$. The claimed point has the shorter error $(-1,0)$, of length 1 against $\sqrt{2}$, which is the trap: a short error proves nothing unless the point lies in the subspace. Section 8.2's proof needs both pieces, the point on the line and the error perpendicular to it, and the claim fails the first. The same warning appears in section 8.2's picture: the shadow lies on the line you chose, and a nearer point from some other set is not a projection onto that line.