Deeper

Lab · c06.s02.L1

Cancellation demo

Cancellation does not create error: it exposes the rounding the two operands already carried, in proportion to the cancellation ratio.

Question: Why does subtracting two correct numbers give a wrong answer?

Cancellation demoTwo numbers straddling 1 share 0 to 18 leading digits. A log-scale chart plots the relative error of their difference: a russet curve with round markers climbs, a gold dashed line shows one rounding times the cancellation ratio, and a dotted rule marks 1, where no digits are correct. At 16 shared digits both store as 1.0, the difference is 0.0, and the relative error is 1.0: the subtraction exposed rounding the operands carried.Cancellation demoa - b, two correct numbers near 1, sharing leading digitsmeasured relative errorceiling: one rounding x ratio1 = no correct digits1e-181e-151e-121e-91e-61e-310369121518gold ceiling capped at 1leading digits the two numbers sharerelative error (log scale)readoutformatdigits sharedabtrue diffcomputed diffrel. errorcancel ratioceilingRe-simulated: IEEE-754 binary32 and binary64 subtraction re-run by data/scripts/c06.s02.L1.py.64-bit (binary64)01.50.51.01.00.02.02.220446049250313e-1664-bit (binary64): 0 shared digits
Re-simulated Re-simulated: IEEE-754 binary32 and binary64 subtraction re-run by data/scripts/c06.s02.L1.py at 73 grid points from 0 to 18 shared digits. The frames follow binary64 from 0 digits to the cliff at 16 and on to 18, then show the same collapse in binary32 between 6 and 8 digits. Each frame draws the measured relative error (russet, solid, round markers), one rounding times the cancellation ratio (gold, dashed), and the rule at 1 (ink, dotted); the one exact-zero error is drawn at the 1e-18 floor, with a readout for the marked grid point.
Lab controls
8 digits

Arrow keys move one step; Page Up/Down move one digit; Home/End go to 0 and 18.

64-bit (binary64)

Space or Arrow keys switch format; Home/End pick the first and last option.

1100 ms per frame

Arrow keys on the slider; Home and End to the ends

Frame 1 of 15: 0 digits, binary64At 0 shared digits the operands are a = 1.5 and b = 0.5. The true difference is 1.0 and the computed difference is 1.0, so the relative error is exactly 0.0; it is drawn at the 1e-18 floor. Nothing cancels, and the cancellation ratio is 2.0.

Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.

Text description of this lab

Two numbers straddling 1 share 0 to 18 leading digits. A log-scale chart plots the relative error of their difference: a russet curve with round markers climbs, a gold dashed line shows one rounding times the cancellation ratio, and a dotted rule marks 1, where no digits are correct. At 16 shared digits both store as 1.0, the difference is 0.0, and the relative error is 1.0: the subtraction exposed rounding the operands carried.

Return to the section that hosts this lab

Caption status: re-simulated. Every lab declares one of published, re-simulated, schematic, synthetic or interpolated.