Section 22.6 lists four ways to compare generators: held-out likelihood, the Inception score, the Fréchet inception distance, and the precision-recall pair. Each answers a different question, and none separates realism from coverage on its own. Likelihood rewards coverage but can prefer a blurry average to a sharp picture. The Inception score does not reward variety inside a class. The Fréchet distance reacts to both realism and variety but cannot say which one failed. Precision and recall separate the two, but only relative to a chosen notion of the data manifold. The field has no procedure that settles which comparison to run, or what a reported number licenses, for every use.
Take a position: design a protocol for comparing two image generators, and state what the resulting numbers are allowed to claim and what they are not.
In your answer: name the purpose the comparison serves and the population it is about; say which quantities you measure, on how many samples, with which fixed preprocessing and which pretrained network; say how the protocol separates fidelity from coverage, and what it does when the two disagree; state one conclusion the protocol supports and one it does not; and name a way the protocol misleads a careful reader even when every step is followed.
Rubric. Tick each criterion your answer meets, then save the self-assessment. The platform does not grade it.
Worked solution
There is no single correct answer, so this is a model of the shape a complete answer takes, not a key. A level-3 answer starts from the purpose. For a controlled comparison between two released generators on the same data, the purpose is to say which distribution is closer to the reference and in what respect, not to crown a universal winner. The answer fixes the instruments before seeing results: a stated number of samples, say ten thousand per generator, the same resizing and colour handling for both, and one named feature network with its weights frozen. It reports a distance and a coverage measure side by side. The distance can be the Fréchet distance in the frozen features; the coverage measure can be precision and recall against a reference manifold, with the manifold construction and its threshold written down. If the distance favours one generator and recall favours the other, the answer reports both and says the comparison is unresolved on the axis that matters for the intended use, rather than averaging the two into a single rank.
Then the answer separates the claims. A two-sample test on the features can support the claim that the two sample sets are hard to distinguish at the chosen sample size; it cannot support the claim that either generator reaches every mode of the data. Recall addresses the second question only relative to the chosen manifold, and precision addresses it only for the regions the reference set covers. The answer states one supported conclusion, for example that generator A is closer in the frozen feature metric while generator B covers more of the reference categories, and one unsupported conclusion, for example that A produces better pictures for a use the reference data does not represent.
The answer names a failure of its own protocol. The feature network was trained for classification, so it can ignore texture and small objects; two generators can tie on its distance while a person sees a clear difference. A fixed sample count can miss a rare mode, so recall is a lower bound at that count rather than a property of the model. A precision-recall threshold is a choice, and moving it changes both numbers. A likelihood bound is a bound, not the probability, and it can reward a blurred average. Finally, the answer states the cost: ten thousand samples per generator need compute and storage, the reference set must be held out from any model selection, and a human study would be needed to support the claim about what people see. A reader may conclude that the protocol ranks these two generators on the axes it measured and under the instruments it fixed. A reader may not conclude that the ranking transfers to another dataset, another resolution, or another purpose.