Leakage

Fifty items, five thousand measured features, and not one feature related to the label. The honest score is 50 per cent, because there is nothing to learn. A standard recipe reported 3 per cent [S:64:264].

The recipe did nothing dishonest, which is the uncomfortable part of the story. Every step was an ordinary step that a careful person would defend in a code review, and the damage came from the order in which the steps were applied rather than from any one of them. It kept the features that correlated best with the labels, and it did that before it split the items into two piles. This section asks how the answer sneaks into the input, and the answer is that a step saw every item before the boundary was drawn. !Two orders of the same three steps. Above, the screen is fitted before the split and its bracket spans all fifty items, held-out labels included, so the kept columns average a correlation of 0.28 with the labels on ten held-out items instead of 0. Below, the split comes first, the bracket spans the forty training items only, and the same pipeline returns chance. The data, the model and the scoring rule are unchanged; only the bracket moves. Status: schematic.

The idea. Leakage is information about the label reaching the model by a path you did not intend. A feature is a measured property of an item, and the [[term:training-set]]training set and test set are the items used to fit and to score. Here the features were chosen using all fifty labels, so the chosen features had seen the held-out items too [S:64:264]. Their correlation with the labels, measured on a random ten of those items, averaged 0.28 instead of zero [S:64:264]. !Mean errors on the ten held-out items against the number of columns kept, over 200 splits of a fifty-item table with 5,000 label-free columns. With the screen fitted on all fifty rows the errors fall from 1.37 at ten columns to no more than 0.04 from a hundred columns on. With the screen fitted on the forty training rows they stay between 4.1 and 5.2, around the 5.065 of a rule that ignores every column. The gap between the lines is the leak, and it widens with the number of columns kept. Status: re-simulated.

The rule that follows is short. With a multistep procedure, the split must come before any selection or filtering step, so that every step is fitted on the training part alone [S:64:265]. One exception is safe from label leakage: a filter that never looks at the label cannot pass label information across the boundary, because it has none to pass [S:64:266]. In code the fault has a recognisable shape, namely a mean, a scale or a shortlist computed once, outside the step that divides the items. The repair is to move that computation inside, so that each part of the division derives its own version from its own items.

Worked example. Six items, labelled 1, 0, 1, 0, 1, 1. A feature is a column of six values, each a 0 or a 1. How many different columns exist?

$$2^6=64$$

Six binary entries make 64 patterns, so the 64 columns contain every pattern that the labels could take, including the label pattern itself. This is the extreme case: the candidate set is the entire column space.

Screen the columns by agreement with the labels and keep the best one. The screen enumerates all 64, so exactly one agrees on all six items and it wins. Split four items for training and two for testing: the winning column agrees with both test items, and the score is 2/2, or 100 per cent. The labels are coin flips, so the true score is 50 per cent. Nothing crossed the boundary by hand; the selection step carried the test answers across it.

The count grows quickly with the number of items, and twenty items admit 1,048,576 distinct binary columns that the toy can enumerate. A real screen cannot enumerate them. Five thousand measured columns over fifty items is a tiny sample of the possible columns, and the expected number of columns that match every label is the number of candidates times $2^{-n}$, which is far below one. The failure comes from selection instead: screen many columns on the same items you later score, and the best of them correlates with the labels by chance before any signal is present. The best of $m$ random columns over $n$ items reaches a correlation near

$$\sqrt{\frac{2\ln m}{n}}\approx 0.58$$

for five thousand columns and fifty items. That is the mechanism behind the 3 per cent: the selected columns still average 0.28 on the ten items whose labels the selection used [S:64:264].

The same fault in the data. Grouping is the second door, and it is the one that survives a clean procedure. A thousand photographs of ten thousand pixels are not ten million independent items, because the pixels inside one photograph travel together [S:47:66]. If a split puts pixels from one photograph on both sides, the model has seen part of its own test item, and the score rises for a reason that will not survive deployment.

The third door is the collection itself. A famous and probably apocryphal story describes a classifier that separated photographs of Russian tanks from American ones, and the separation turned out to track the weather on the day each set was photographed [S:109:259]. Leakage names this too: a spurious relationship between the features and the target that is an artefact of how the data were gathered [S:109:259]. In that story the photographs contained the answer to a different question, and no later split could repair the damage, because the fault lay in the gathering and not in the arithmetic.

Revising for an examination with the answer key open on the desk is the usual picture of this fault. The difference is intent: nobody in the leak story means to cheat, yet the code carries the answers across the boundary quietly, and the person who wrote the code sees a better number than the truth.

The limitation is that leakage is invisible in the score, which is exactly why a good score cannot rule it out. A leaked result usually looks better than it should. Leakage can also leave a score unchanged, which is why the reported number cannot rule it out and why the defences are the order of the steps and a planted leak [S:109:259].

The formal statement. Let $D$ be the finite family of items with their labels. A procedure is a composition of steps, each a map from data to data. Call a step label-aware when its output depends on the labels. The rule of this section is: the split is drawn first, and every label-aware step is fitted on the training part alone [S:64:265]. A filter that reads only the features is exempt, and the exemption has a precise scope.

The derivation. Count columns. With $n$ items and binary labels there are $2^n$ distinct columns, and one of them equals the label pattern. A screen that enumerates all $2^n$ columns and keeps the one agreeing best with all $n$ labels reaches zero training error by construction. With $m$ independently drawn candidate columns the count is different: the expected number of perfect columns is

$$m\cdot 2^{-n}$$

Read it as: perfection is bought by enumeration, not by luck. For the study's five thousand columns over fifty items the expected count is far below one, so the label pattern is essentially absent. What the screen does instead is select: it keeps the columns with the highest observed correlation on the same items, and those columns average 0.28 on a random ten rather than zero [S:64:264]. The 0.28 is the size of the advantage that selection bought, and it is not evidence that any column contained the labels.

The edge case L1 blurs. L1 says a filter that never reads the label is safe. That is a claim about bias, not about identity. An unsupervised step fitted on all of $D$ passes no label information, so the estimate stays unbiased. The number it produces is still not the number from a step fitted inside each fold, because the mean and the scale differ [S:64:266]. Safety and equality are different claims, and a report that conflates them overstates what a fold-local pipeline reproduced.

A second edge case. A correctly ordered cross-validation is still a random quantity. Its estimate carries a standard error, and on small folds that error can exceed the difference you want to report [S:64:268]. L1's extreme case runs from 100 per cent down to a true 50 per cent. The ordinary case is a few points of fold-to-fold noise on every number.

What L1 hid, and why it was safe. L1's repair, move the computation inside, is necessary and not sufficient for grouped data. When one photograph contributes many pixels, a fold-local mean still puts pixels of that photograph on both sides of the boundary [S:47:66]. The unit of the split must be the thing that was drawn independently, and choosing that unit is a modelling decision rather than a line of code. L1 placed this in its second half; L2 makes it a hypothesis of the rule.

Where the analogy breaks. The answer-key picture puts the fault in intent. A student who opens the key knows the score is not theirs. The selection step here is defensible on its own, and the composition is what fails, so nobody in the process sees a boundary being crossed.

Original sources. The demonstration that motivates the section is in The Elements of Statistical Learning, section 7.10.2. Fifty samples, five thousand predictors independent of the labels, a screen on all fifty, and an average cross-validation error of 3 per cent against a true 50 per cent [S:64:264]. The same section states the repair for a multistep procedure, and the exception for unsupervised screening [S:64:265], [S:64:266]. The definition of leakage as a spurious relationship produced by collection or sampling is in Fairness and Machine Learning, with the tank story as its illustration [S:109:259]. The grouped-data version, where pixels are not independent items, is in A Course in Machine Learning [S:47:66].

An open question. How to detect contamination of a benchmark by a training corpus is unsettled. A 2024 report calls it an open field, and notes that any of these methods can suffer from false positives and false negatives [S:431:33]. GPT-3's authors reached the same conclusion from the other side: they could measure the effect of partial decontamination, but could not prove that the remaining clean subset was drawn from the same distribution as the whole [S:232:33]. Evidence that would settle it: a detection rule with a stated false-positive rate, validated on a benchmark whose contamination is known by construction, together with the score difference reported with an interval.

A frontier method. Current practice is conservative overlap counting. Llama 3 flags an example when a chosen share of its tokens belongs to an 8-gram that occurs in the pretraining corpus, picks the threshold separately for each dataset, and reports the estimated performance gain [S:431:34]. That is the leak of this section measured after the fact, and its blindness is that it can see contamination only through a change in the score: Llama 3 reports fifty-two per cent estimated contamination on NaturalQuestions with virtually no effect on performance, and high contamination on SQuAD and MATH with no performance gain [S:431:34]. An unchanged score is therefore not evidence of a clean benchmark.

What to read next. Read section 7.10.2 of [S:64:264] first, because the histogram of correlations is the fault made visible. Then read [S:431:33] and [S:431:34] for what a modern contamination audit looks like, and [S:47:66] for the split unit, which is the part no code review catches.