Deeper

Lab · c15.s02.L1

Validation curves

A held-out score belongs to the reservation, not to the model alone: rotating which rows are hidden and averaging scores every row once, at one fit per fold.

Question: How do you reserve data without wasting it?

Validation curvesTwenty-four messages lie on a link-count axis, spam above and ordinary below, cut by a threshold. The rows of the held-out fold are ringed and the ones it misclassifies are crossed. Below, bars give each fold's errors and a curve plots the cross-validated score against folds, with a band over thirty deals. One fold scores 0/6 and another 4/6; rotating the held-out rows averages to 4/24, 16.7%, counts every row once, and costs one fit per fold.Validation curvespredict spam when links > t · 24 messages · spam above, ordinary belowring = held out · cross = missspamordinary01links per message00%45%scorefolds kScore against folds241224all rows 4/24 = 16.7%Every row is scored once; each fit sees the other rows.t = 0.56250/60/60/64/6Fold 4 · 4/64 fits · 4/24 = 16.7%18.5%The score belongs to the six rows this fold happened to hide.
Synthetic Synthetic: the 24 messages, their link counts and their labels were written for this lab; no measurement or published table stands behind them. Every number in the frames is read from data/c15.s02.L1.json, recomputed from seed 20261108 by data/scripts/c15.s02.L1.py. The walkthrough holds k = 4 at deal 5: three folds score 0/6 and fold 4 scores 4/6, while the curve carries the mean and band over all 30 deals.
Lab controls
4

Arrow keys move through the seven values; Home selects 2, End selects 24

5

Arrow keys step one deal, Page Up/Down five deals, Home selects 1, End selects 30

4

Arrow keys step one fold, Page Up/Down two folds, Home selects 1, End selects 24

Frame 1 of 9: Fold 4 hides six rowsFold 4 holds out six rows and scores 4/6. Its four misses are the spam at 0.45 and 0.53 links and the ordinary mail at 0.61 and 0.67 links; the other two rows in the fold are scored correctly.

Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.

Text description of this lab

Twenty-four messages lie on a link-count axis, spam above and ordinary below, cut by a threshold. The rows of the held-out fold are ringed and the ones it misclassifies are crossed. Below, bars give each fold's errors and a curve plots the cross-validated score against folds, with a band over thirty deals. One fold scores 0/6 and another 4/6; rotating the held-out rows averages to 4/24, 16.7%, counts every row once, and costs one fit per fold.

Return to the section that hosts this lab

Caption status: synthetic. Every lab declares one of published, re-simulated, schematic, synthetic or interpolated.