When the line is wrong

A second café stands on a beach where demand increases explosively on hot days. Its four pairs are (1, 1), (2, 4), (3, 9) and (4, 16): each additional hour of sunshine multiplies the crowd instead of adding to it. Least squares still returns a line, and the line has slope 5 with intercept −5, because that is what the arithmetic does to any four pairs you hand it. Nothing inside the procedure raises an objection, because the procedure never inspects the shape of the table it receives.

The line predicts 0, 5, 10 and 15, so the four residuals are 1, −1, −1 and 1. They add to zero, their squares add to 4, and the total variation is 129, which gives R² = 0.969. By that summary the fit appears excellent, and the summary is not lying. It answers a narrower question than the owner asked: R² compares the squared misses of this line with the squared misses of the flat line through the average, and it never asks whether those misses trace a curve. The line is fitted under a list of assumptions about the errors around it, meaning claims about their average size, their independence and their spread, and those assumptions are what the rest of this section names. !Both panels plot the same four points (1, 1), (2, 4), (3, 9) and (4, 16). The least-squares line has slope 5 and intercept −5, and R² = 0.969, yet its residuals 1, −1, −1, 1 trace a U: the line runs above the points in the middle and below them at both ends, so it cannot follow the curve that produced the table. Status: schematic.

The trap is the gap between the arithmetic and the claim. A linear regression is a bundle of assumptions about the errors, not a claim that the points sit near a line. Write the recorded answer as y, the linear rule as β0 + β1x, and the leftover for row i as εi, the error term for that row. The model then says the recorded answer equals the linear rule plus that leftover: !Each panel adds one afternoon to the café table of §14.1; the dashed line is the fit to the original four, the solid line the fit after the addition. In (a) the new afternoon sits at 2.5 hours, inside the measured range, and 8.8 above the original line; the refit lifts the line without tilting it, so the slope stays at 2.2 and the new residual is 7.0, the largest in the panel. In (b) the new afternoon sits at 6 hours, beyond the measured range, and only 2.1 above the extrapolated line; the refit tilts to a slope of 2.6 and the new residual falls to 0.6, smaller than two of the four afternoons inside the range, so the residual plot raises no alarm while the line has moved. Status: schematic.

The line is the systematic part and εi is the error left over for item i. The assumptions add that those errors are independent of the input, so knowing x tells you nothing about the likely size or sign of the error. They are also uncorrelated with one another, so one row's error says nothing about the next row's, and equally spread at every input value, with the same variance σ² throughout [S:49:81] [S:49:84]. Each clause describes the unseen population of future answers rather than the recorded table, so a residual plot offers evidence about them rather than proof [S:49:109]. When the true relationship departs far from a line, most conclusions drawn from the fit become suspect [S:49:109].

Six problems cover nearly every way a fit misleads. They are a curved relationship, correlated errors that move together, errors whose spread changes with the input, one extreme answer, one point that drags the line, and inputs that duplicate each other [S:49:109]. The first four leave fingerprints in the residuals, which is why plotting them remains the least expensive diagnostic available.

The assumptions are also what the standard errors, the intervals and the tests rest on. A standard error is the estimated spread of a coefficient across repeated tables, an interval is a range built from it, and a test compares the coefficient with that spread. A broken promise quietly corrupts every number printed beside the coefficients [S:49:110].

Worked example. Take the table of squares and read the residual plot, which graphs each residual against its input. The sequence 1, −1, −1, 1 dips in the middle and rises at both ends, and that U shape is the signature of a curve a straight line cannot follow. Extend the fitted line to five hours and it predicts 20, while the pattern in the table gives 25. The miss is 5 units, and no residual inside the measured range had announced a miss of that size, because the misses at the two ends stay small even when the curve is badly wrong.

The plot, not the R², carries the warning. These four points lie exactly on $y=x^2$, so adding a column holding the square of the sunshine value fits them with residuals 0, 0, 0, 0. That is saturation, not noise: the residual plot has nothing left to show, and with four points and three parameters the fit has one residual degree of freedom, so it cannot discriminate. A fifth afternoon is what would test the repair.

ISLP's reassurance about a little pattern comes from a table far larger than four rows, where the fit does not interpolate, and an empty plot there means something [S:49:110]. The change of form is the remedy and the admission at once: the fit stops being a straight line.

An extreme answer is the fourth problem, and a single careless recording is sufficient to bend the fit toward itself. The residual plot identifies the culprit, although deciding how large a residual is too large takes a stated rule rather than a glance [S:49:113]. The fifth problem is the more dangerous of the two, because an unusual answer is easy to spot in a plot while an unusual row is easy to miss. In simple regression a row whose input lies outside the measured range can move the fit, because the fitted line has little company there.

With several inputs the test is not the range of any single column. A row can hold an ordinary value for every input taken alone and still sit far outside the joint cloud, invisible to a look at the margins. It can also swing the fit a long way [S:49:114]. Such a row is said to have high sway.

A row that both has high sway and carries a large residual is called influential, and its damage grows with the distance of its input from the rest of the table. Sway alone is not influence: a row with high sway whose answer sits on the line moves almost nothing, and an outlier with ordinary sway leaves the line almost where it was [S:49:113].

Collinearity is the sixth problem and the least visible. When two inputs carry nearly identical information, the fit cannot determine which one deserves the credit, so the coefficients swing and their standard errors grow [S:49:116]. The damage runs the other way from the intuition: the inflated standard error deflates each coefficient's t-statistic, so the model can report no effect for an input that is doing the work while its twin absorbs the credit.

In the credit table the standard error of the limit coefficient rises twelvefold and its p-value climbs from below one in a thousand to about seven tenths once rating enters. The source says the importance of the limit variable has been masked [S:49:117]. Discarding one of the pair usually costs almost nothing in fit and restores the interpretation.

A further illusion hides inside the summary number, and the illusion is easy to state. On the training table R² never falls when a column is added, and it rises unless the new column is exactly orthogonal to the existing ones, in which case it stays put. The reason is arithmetic: the fit gains one more direction in which to reduce the residual sum of squares, and it uses that direction whenever doing so helps at all [S:49:97]. A high R² therefore certifies that the line moved closer to the training points, and says nothing about held-out rows.

The idea has a cost. Nothing in the residuals says which promise failed, and reading a plot is a judgement rather than a test. With twenty inputs there is no single picture to inspect, and the errors themselves stay unobserved, so every check works through their stand-ins, which are only as trustworthy as the fit that produced them.

The formal statement. Stack the rows into a matrix $X$ whose first column is all ones, and write the model as $y=X\beta +\epsilon$. Least squares returns

$$\hat{\beta }={\left(X^{\mathsf{T}}X\right)}^{-1}{X}^{\mathsf{T}}y.$$

The inverse requires $X^{\mathsf{T}}X$ to be invertible, which fails when two columns are identical or when the table holds more columns than rows [S:49:95]. The working assumptions are a mean-zero error given the inputs, a common error variance ${\sigma }^2$, and no correlation between errors [S:49:84]. Under them the estimate is unbiased, with covariance ${\sigma }^2(X^{\mathsf{T}}X{)}^{-1}$.

The derivation. Substitute the model into the estimator to get $\hat{\beta }=\beta +(X^{\mathsf{T}}X{)}^{-1}{X}^{\mathsf{T}}\epsilon$. The expectation is $\beta$ because the errors have mean zero. The covariance comes from sandwiching ${\sigma }^{2}I$ between the two matrices. With one input the slope entry is ${\sigma }^2$ divided by the sum of squared input deviations, so the standard error of the slope falls as the inputs spread out [S:49:84]. The confidence interval and the t-statistic then read that standard error [S:49:86]. Every number printed beside a coefficient inherits all three assumptions.

The edge case. The residuals are not the errors. The fitted values are $Hy$ with $H=X(X^{\mathsf{T}}X{)}^{-1}{X}^{\mathsf{T}}$, so the residuals are $(I-H)\epsilon$ and each has variance

$$\mathrm{Var}(e_i)={\sigma }^2\left(1-h_{ii}\right).$$

The diagonal entries $h_{ii}$ differ across rows, and a row with an unusual input carries a value that moves toward one [S:49:114]. Its residual is then forced toward zero even when its error is large. L1 sends you to the residual plot, and for the table of squares that advice works, because all four residuals have the same size and the pattern is visible. It misleads in the case L1 calls most dangerous: an influential row hides inside a small residual, which is why the textbook divides each residual by its own standard error before judging size [S:49:113].

What L1 hid. L1 gives six problems as one list. They are three kinds of failure. Curvature, correlated errors and changing spread break the mean or the covariance, so the estimates are biased or the standard errors are wrong [S:49:109]. An outlier is a large studentised residual and a high-leverage row is a large diagonal entry $h_{ii}$ of $H$. Influence requires both: a high-leverage row whose answer sits on the line barely moves it, and an outlier with ordinary leverage leaves the line almost where it was. When the two coincide they change the fit while no assumption about the errors is false, and the source calls that coincidence a particularly dangerous combination [S:49:115]. Collinearity is a statement about the columns of $X$, and it leaves predictions stable while the individual coefficients wander along a flat valley [S:49:116].

Where the analogy breaks. "Promises about the errors" suggests a contract you can audit. You cannot audit it from the same table. The errors are unobserved, and the statement that they are unrelated to the input concerns the population of future answers [S:49:109]. A residual plot can show that a promise failed. It cannot show that one held.

Original sources. ISLP section 3.3.3 lists the six problems and points to the residual plot as the first diagnostic to draw [S:49:109]. It states that a pattern in that plot may indicate a problem with the model [S:49:110], and it gives the studentised residual as the repair for rows of unequal influence, with a cut-off of three [S:49:113]. The influence statistic itself appears in the same section, together with the argument that a row far from the others can move the whole line [S:49:114]. Collinearity gets a variance inflation factor and two remedies, one of them dropping a column [S:49:118].

An open question. No rule fixes how large a residual must be before you call a row an outlier. The corpus says the choice is difficult in practice and offers the studentised rule of three as a convention rather than a test [S:49:113]. The field has not settled a decision procedure with a stated error rate. Evidence that would settle a case: a rule with a known false-positive rate under a named error distribution, checked on tables where the contaminated rows are known in advance.

A frontier result. Reading residuals and then changing the model is an adaptive use of one table, and adaptive use costs something measurable. In the adaptive setting the guarantee of the holdout method is exponentially worse in the number of queries than in the fixed setting [S:59:170], and once the number of queries reaches the table size the guarantee is vacuous [S:59:169]. That result turns L1's "reading a plot is a judgement rather than a test" into a quantitative warning about the analyst as well as the model.

What to read next. Read ISLP section 3.3.3 for the six problems, the residual plot and the influence statistic [S:49:109] [S:49:114]. Then read Hardt and Recht's chapter on the scientific basis of benchmarks for the account of adaptive reuse and what limits its damage [S:59:169].