Two networks have the same architecture and both fit a table of two hundred rows with zero training error. Their hidden units report different values on every row. You must keep one network and say what its hidden layer computes. The chapter supplies the pieces of the decision but no procedure that settles it, and the field has no procedure that settles it for every use. A lower training loss cannot choose, because both losses are zero. A hidden unit is a coordinate the training pressure selected, not a column anyone recorded, and two runs can arrive at different coordinates with the same fit.
Take a position on this question: what evidence should decide which network you keep, and what may you claim about what its hidden units compute?
In your answer: say what you measure and on which rows; say how that measurement chooses between the two networks; say what the evidence lets you claim about a hidden unit and what it does not; and name one way your procedure misleads you even when every step is followed.
Rubric. Tick each criterion your answer meets, then save the self-assessment. The platform does not grade it.
Worked solution
There is no single correct answer, so this is a model of the shape a complete answer takes, not a key. A level-3 answer begins by fixing the comparison before it looks at either network. It sets aside a held-out block of rows, or defines a perturbation of the inputs, and states that this block plays no part in fitting either network. Then it measures the loss on that block for both networks and chooses the smaller. It reports the two values and their gap, not only the winner, because a gap of one row on a block of twenty is not the same evidence as a gap of one hundred rows on a block of a thousand.
The answer then turns to the hidden units and lowers the claim. The evidence supports a statement about the whole network: on rows like the held-out ones, this network's output unit separates the classes using the coordinates its hidden layer supplies. It does not support the statement that a particular unit detects a particular feature. To get closer to a feature claim, the answer intervenes. It zeroes one hidden unit on every row and measures how much the output changes, or clamps the unit at its average and repeats the measurement. A unit whose removal changes many predictions is one the output uses; a unit whose removal changes nothing may be redundant, and a second unit may carry the same information in another coordinate. The answer repeats the whole procedure under a second random seed, because two runs that fit equally well can settle in different coordinates, and a claim that appears in one seed and not the next is a claim about that run.
The answer names a failure. The held-out block can be small enough that the difference between the two networks is smaller than the variation between two seeds of the same network, in which case the choice is noise dressed as evidence. Choosing the winner on the same rows used to report its score inflates the score, because the winner was picked for doing well there. An ablation can show nothing when two hidden units share the same work, and it can show a large effect when a unit is the only path to the output, which is a fact about the wiring rather than about the meaning of the unit. Finally, the answer says what would reverse its choice: a second held-out block on which the loser wins, a seed change that erases the gap, or an ablation that shows the unit it wanted to name is not used by the output.