Deeper

Lab · c24.s04.L1

Embedding explorer

Words used in similar ways sit near each other in a hidden layer's space, so the map of word vectors falls into separate clumps with a wide empty gap between them.

Question: What does the arrangement of a hidden layer's word vectors say about the words?

Embedding explorer24 words sit as dots in a hidden-layer plane, forming 3 unlabelled clumps: animals on the right, food at the lower left, weather at the upper left. The selected word wears a russet ring in the overview; a detail panel enlarges its clump, with gold lines and numbered badges to its nearest neighbours, whose names and distances are listed alongside. Moving the selection redraws the lines; the axis numbers are arbitrary.Embedding explorerNearness in a hidden layer's space means similarity of use.all 24 wordshidden coordinate 1 (arbitrary units)hidden coordinate 2 (arbitrary units)the selected clump, enlarged; distances stay in map units123catdoghorsecowfoxbearmouserabbitcat · 3 nearest wordsrank · word · distance1. dog0.212. rabbit0.633. mouse0.79Overview: 24 words in 3 clumps. The detail enlarges only the selected clump; axes are arbitrary.
Synthetic Synthetic: the coordinates are a designed toy layout from data/scripts/c24.s04.L1.py, not an embedding fitted to text; a real hidden layer has many coordinates, and this map uses 2 axes so the pattern can be drawn. The 24 words form 3 groups of 8, each made of 4 close pairs, with a small seeded jitter. Distances and neighbour lists are computed exactly from the layout. The frames open with cat selected and 3 neighbours shown, then follow the neighbour slider from 1 to 5 for each word.
Lab controls
cat

Up/Left: previous word; Down/Right: next word; Home: cat; End: frost; typing a letter jumps to the next word beginning with that letter

3 words

Left/Down minus one, Right/Up plus one; Page Down minus two, Page Up plus two, clamped to 1 and 5; Home 1; End 5

Frame 1 of 1: cat: 3 of 5 neighbourscat selected with 3 of 5 neighbours shown. Gold lines run to dog at 0.21, rabbit at 0.63, mouse at 0.79. The third neighbour sits between 0.77 and 0.81 away. All 3 lie in the same clump; the nearest word from another clump is 3.19 away.

Keyboard: Space plays or pauses, Left and Right step, Home goes to the first frame, End goes to the last frame.

Text description of this lab

24 words sit as dots in a hidden-layer plane, forming 3 unlabelled clumps: animals on the right, food at the lower left, weather at the upper left. The selected word wears a russet ring in the overview; a detail panel enlarges its clump, with gold lines and numbered badges to its nearest neighbours, whose names and distances are listed alongside. Moving the selection redraws the lines; the axis numbers are arbitrary.

Return to the section that hosts this lab

Caption status: synthetic. Every lab declares one of published, re-simulated, schematic, synthetic or interpolated.