Deeper

Lab · c15.s05.L1

Permutation test

A permutation p-value is the share of label reshuffles whose statistic is at least as extreme as the observed one: it counts how often a broken label link does as well, and says nothing about the probability that the hypothesis is true.

Question: What does a significance test actually test?

Permutation testFourteen tomato plants stand in a row, each at its yield: seven marked F for fertilized and seven marked C for control. A russet line marks the observed gap of 0.39 kg between group averages. As reshuffles run, the F and C marks re-deal across the same plants and a gold histogram of shuffled gaps grows around zero. Bars right of the line are shaded; a counter reports 24 of 500, so p = 0.048.Permutation test: re-deal the labelsF mean 3.21 kg − C mean 2.83 kg = 0.39 kg observed gap14 plants, 7 F and 7 C, on the same 14 yields: only the chips re-dealyield kg2.43.03.8Shuffled gaps: where the re-dealt labels landtail shaded920shuffled gap in mean yield (kg)-0.70.00.7FCFCFCFCFCFCFCobserved 0.39 kgreshuffles drawn 0tail: none yetp = —
Synthetic Synthetic: the fourteen yields and their labels were written for this lab, not measured on a real trial. The 500 shuffles come from data/scripts/c15.s05.L1.py with the fixed seed 20261108, and all 3432 label splits are also enumerated (158 in the tail, p = 0.046) as a check. Each still is one checkpoint of the dial: the yields never move, only the F and C chips re-deal.
Lab controls
0 shuffles

Left and Right move one checkpoint (25 shuffles); Page Up and Page Down move four (100); Home selects 0; End selects 500. Each position redraws the strip and the histogram.

Frame 1 of 6: No reshuffles yetThe fourteen plants keep their yields, seven F and seven C where the trial put them. No reshuffles have been drawn, so the histogram is empty and the readout is a dash. The russet line marks the observed gap of 0.39 kg: p will be the share of reshuffles that reach or pass it.

Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.

Text description of this lab

Fourteen tomato plants stand in a row, each at its yield: seven marked F for fertilized and seven marked C for control. A russet line marks the observed gap of 0.39 kg between group averages. As reshuffles run, the F and C marks re-deal across the same plants and a gold histogram of shuffled gaps grows around zero. Bars right of the line are shaded; a counter reports 24 of 500, so p = 0.048.

Return to the section that hosts this lab

Caption status: synthetic. Every lab declares one of published, re-simulated, schematic, synthetic or interpolated.