Practise: Lines, Curves and Boundaries

The mean sunshine is $\bar{x}=(1+2+3+4)/4=2.5$, and the mean count is $\bar{y}=(2+3+6+7)/4=4.5$. The numerator of the slope is $$(-1.5)(-2.5)+(-0.5)(-1.5)+(0.5)(1.5)+(1.5)(2.5)=3.75+0.75+0.75+3.75=9.$$ The denominator is $$(-1.5{)}^2+(-0.5{)}^2+(0.5{)}^2+(1.5{)}^2=2.25+0.25+0.25+2.25=5.$$ The slope is $9/5=1.8$. The intercept is $\bar{y}-{\beta }_1\bar{x}=4.5-1.8\times 2.5=0$, so the fitted line is $\hat{y}=1.8x$. The predictions are $1.8$, $3.6$, $5.4$ and $7.2$, and the residuals are $0.2$, $-0.6$, $0.6$ and $-0.2$, which sum to zero as a line with an intercept requires.

At the four inputs the line predicts $0.9$, $2.3$, $3.7$ and $5.1$. The observed counts are $1$, $2$, $4$ and $5$, so the residuals are $0.1$, $-0.3$, $0.3$ and $-0.1$. Their squares add to $0.01+0.09+0.09+0.01=0.2$. The mean count is $(1+2+4+5)/4=3$, and the squared deviations from it are $4$, $1$, $1$ and $4$, a total of $10$. Therefore $$R^2=1-\frac{0.2}{10}=1-0.02=\mathrm{0.98.}$$ The line accounts for 98 per cent of the variation in the counts. That summary says the line tracks these four rows closely. It says nothing about whether the errors are independent or equally spread, which is the warning of section 14.2.

The fitted value at input $x$ is $ax+ax=2ax$. Against $y=3x$ the residual is $3x-2ax=(3-2a)x$. The residual sum of squares is $$(3-2a{)}^2(1^2+2^2+3^2)=14(3-2a{)}^2.$$ The ridge penalty is $14(a^2+a^2)=28a^2$. The total is $$14(9-12a+4a^2)+28a^2=126-168a+84a^2=84(a-1{)}^2+42.$$ The smallest value sits at $a=1$. Each copy then carries weight $1$, the pair carries total weight $2$, and the predictions are $2$, $4$ and $6$ against $3$, $6$ and $9$. The residual sum of squares is $14$ and the penalty is $28$, so the two terms of the ridge total are $14$ and $28$ at the optimum. The charge pulls the total weight below the least-squares value of $3$ and splits what remains evenly between two columns that carry one fact.

The second input has $|b|=2$, so its coefficient is $2-\lambda /2$ while that number stays positive and zero afterwards. The crossing is at $2-\lambda /2=0$, which gives $\lambda =4$. At that charge the first coefficient is $5-2=3$, and the third input disappeared earlier, at $\lambda =1$. The first input survives until $\lambda =10$. The order of deletion is the order of support: the input with the smallest fit leaves first, and the strongest input leaves last. Ridge on the same three inputs would keep all three coefficients positive in magnitude at every finite $\lambda$, because division by $1+\lambda$ never produces an exact zero.

At $x=2$ the score is $z=-1+0.8\times 2=0.6$. The logistic function returns $$p=\frac{1}{1+e^{-0.6}}=\frac{1}{1+0.5488}=\frac{1}{1.5488}=0.6457,$$ which rounds to $0.646$. The score is positive, so the probability sits above one half, but only slightly: the curve is close to linear near zero. A one-unit rise in $x$ adds $0.8$ to the score and multiplies the odds by $e^{0.8}=2.23$, while the probability itself moves by a smaller amount.

Section 14.5 writes the model as $\log (p/(1-p))={\beta }_0+{\beta }_{1}x$. Raising $x$ by one adds ${\beta }_1=0.7$ to the log-odds, and the odds therefore multiply by $$e^{0.7}=2.0138,$$ which rounds to $2.01$. The probability does not multiply by that factor; its change depends on where on the curve you start. The multiplier is constant across the whole range of the other inputs, which is the strong assumption section 14.5 tells you to check before using the fit for decisions.

With a log link the mean is $$\mu =e^{0.5}=1.6487,$$ which rounds to $1.65$. A score of zero would give a mean of $1$, and a score of $-2$ would give $0.135$. The exponential cannot return a negative number, which is the property a straight line lacks when the answer is a count. The same score read through the identity link gives a mean of $0.5$ and through the log-odds link gives a probability of $0.622$.

The residuals start positive, become negative in the middle and return positive at the right. That U shape is the signature section 14.2 gives for a curved relationship: a straight line cannot follow a curve that bends once, so it sits too low at both ends and too high in the middle. Option (b) would show residuals that fan out or pinch as the input grows, not a consistent U. Option (c) is a property of two inputs and cannot be read from a sequence against one input. Option (d) would show one residual far from the rest at an extreme input, with the others behaving normally. The plot identifies the pattern; it does not by itself prove which promise failed, because the errors are unobserved and the residuals are only their stand-ins.