c24 · E3

Open: Scale, Language Models and What They Learned

c24.E3.1E3 · open

The chapter shows three facts that pull in different directions. Loss falls along a power law as parameters, data and compute grow. Hidden vectors carry structure that a probe or a sparse autoencoder can read. An explanation can be plausible to a person and still fail when the model is intervened on. The field has no agreed answer to whether larger models are easier or harder to explain and align.

Take a position on this question: as models grow, does interpretability and alignment get easier, harder, or neither? Design a measurement that would support your position, say what result would change your mind, and name one thing your measurement cannot establish.

Rubric. Tick each criterion your answer meets, then save the self-assessment. The platform does not grade it.

Other tiers

E1 Practise · E2 Derive

Every hint opened, solution viewed and answer checked is recorded in this browser only. See Progress to export or delete it.