Derive: Judging a Model Honestly

The training error is zero because each of the eight inputs is present in the table. On row $i$ the rule finds the stored label $y_i$ and returns it, so all eight predictions equal the recorded labels: $$\text{training error}=\frac{0}{8}=0.$$

For leave-one-out, remove row $i$ and predict it from the remaining seven. The nearest remaining input is determined by the gaps. Write $g_i=x_{i+1}-x_i$. At an interior row $i$, the distance to the left neighbour is $g_{i-1}$ and the distance to the right neighbour is $g_i$. The gaps are $$1,2,3,4,5,6,7,$$ so $g_{i-1}<g_i$ and the left neighbour is strictly closer. Row 1 has no left neighbour, so its nearest remaining input is row 2. Therefore the leave-one-out prediction is $y_2$ for row 1 and $y_{i-1}$ for every other row: $${\hat{y}}^{(-i)}=(0,0,0,1,1,0,1,1).$$

Compare this with the recorded labels:

| row | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |---|---|---|---|---|---|---|---|---| | $x$ | 0 | 1 | 3 | 6 | 10 | 15 | 21 | 28 | | $y$ | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | | held-out prediction | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | | correct? | yes | yes | no | yes | no | no | yes | no |

Rows 3, 5, 6 and 8 are wrong, so the leave-one-out error rate is $$\frac{4}{8}=\mathrm{0.5.}$$

The comparison is section 15.1's point in one table. One stored label per row drives the training error to zero, and the zero says only that the table can reproduce what it was given. Section 15.2's leave-one-out score asks a different question, because the held-out row is never in the table and the rule must fall back on a neighbour. The fallback recovers the label on the rows whose label agrees with the neighbour they copy, namely rows 1, 2, 4 and 7. The same fitted rule therefore scores 0 in training and 0.5 under leave-one-out. Leave-one-out is cross-validation with one row per fold; it costs eight fits and it estimates the procedure's error on unseen rows, not the memoriser's stored answers.

The observed maximum 9 occurs in exactly one of the five rows. A single draw with replacement misses that row with probability $$1-\frac{1}{5}=\frac{4}{5}.$$ The five draws are independent, so the probability that every draw misses the row is $${\left(\frac{4}{5}\right)}^5=\frac{1024}{3125}=\mathrm{0.32768.}$$ The complement is the probability that at least one draw selects the row, which is exactly the event that the resample maximum equals 9: $$1-{\left(\frac{4}{5}\right)}^5=1-\frac{1024}{3125}=\frac{2101}{3125}=\mathrm{0.67232.}$$

The bootstrap distribution of the maximum therefore places mass $0.67232$ at 9 and the remaining $0.32768$ below 9. It places no mass above 9, because no resample can contain a value that the sample does not contain. A genuine new sample of five from the population can exceed 9 whenever the population has values above the observed maximum, so the bootstrap distribution of the maximum is too narrow on the upper side. The same argument applies to any extreme statistic: the sample maximum is the largest value the resample can ever see, so the resample treats the observed extreme as a hard ceiling and overstates the chance that it recurs. Section 15.4 says the bootstrap cannot invent values that the sample never contained; the maximum is the case where that limit changes the answer rather than merely coarsening it. The mean and the median do not have this problem in the same form, because they move when the counts of ordinary values change.

For one test the probability of failing to reject is $$1-\alpha =\mathrm{0.95.}$$ The twenty tests are independent, so the probability that none of them rejects is the product $$0.95^{20}=0.3584859224\dots$$ Therefore the probability that at least one rejects is $$1-0.95^{20}=1-0.3584859224\dots =0.6415140776\dots ,$$ which rounds to $0.6415$.

The expected number of rejections is $$20\times 0.05=1.$$ The two numbers answer different questions. The expected count says that the twenty tests produce one false rejection on average, which sounds harmless. The probability says that in a single run of the twenty tests there is a 64 percent chance of seeing at least one false rejection. Section 15.5 uses the expected count to explain why searching produces small p-values; the probability shows how often the search actually lands on one. The calculation assumes independence, and dependent tests change the exact probability, but the direction survives: many tests raise the chance of a false alarm. This is why the section's repair divides the threshold by the number of tests or controls the false discovery rate.