The four subspaces

A table with three rows and four columns holds twelve numbers, and this one carries only two independent rows: the third row is twice the first plus the second.

!The table has four columns and three rows, and its third row is twice the first plus the second, so the rank is two. On the input side the row space has dimension two and the kernel two, and 2 + 2 = 4 is the number of columns. On the output side the column space has dimension two and the left null space one, and 2 + 1 = 3 is the number of rows. The bars are lengths, not directions: each bar places its two spaces by the length of its lists, and the rank link marks the length the two navy segments share. Nothing in the drawing claims the spaces are perpendicular. Status: schematic.

The display says the third row adds nothing, because it is already built from the first two, and the reduced form — the table left after elimination has cleared every pivot column — makes that redundancy visible as a row of zeros.

The section question asks which four spaces a table defines, and the answer is that it defines two in the space of inputs and two in the space of outputs. The first two columns, (1, 0, 2) and (2, 1, 5), span the column space, and the first two rows, (1, 2, 3, 4) and (0, 1, 1, 1), span the row space. The inputs the table sends to zero form the kernel. The fourth space is made of output lists, and the list (2, 1, -1) below shows what it is [S:25:70].

The input here has four slots and the output has three, so the row space and the kernel live among four-entry lists while the column space and the space of cancelling output lists live among three-entry lists. That placement is half the content of the idea: a question about rows is a question about inputs, and a question about columns is a question about outputs.

The dimensions follow one ledger. The rank of the table plus the dimension of its kernel equals the number of columns, which is the number of input slots, and the rank plus the dimension of the fourth space equals the number of rows [S:25:74]. The rank itself can be counted either way, because the number of independent columns equals the number of independent rows [S:27:53]. The rank is therefore an economical characterisation, since neither count depends on which row you happen to write first, and one number sets all four dimensions.

Now fill the ledger for this table. The first two rows are independent and the third is their combination, so the rank is two. The column space is spanned by the first two columns of the original table, which are (1, 0, 2) and (2, 1, 5) [S:25:70], and the row space is spanned by the first two rows. The pivot columns come from the original table and not from its reduced form, which is the mistake the reduced form invites, since its leading columns are the plain unit lists.

The kernel has dimension four minus two, so two, and here is a basis for it. The list (-1, -1, 1, 0) gives zero against every row, and so does (-2, -1, 0, 1); check the first: one times minus one plus two times minus one plus three times one plus four times zero is zero, and the other rows follow.

The display checks one kernel vector against every row, and the second vector checks the same way, so the kernel is a genuine two-dimensional set of inputs that the output cannot see.

The output list (2, 1, -1) has entry-by-entry product zero with every column, because twice the first row plus the second row minus the third row gives zero. So it spans the left null space, which has dimension three minus two, so one. It is called left because its members multiply the table from the left, $y^{\mathsf{T}}A=0$, the mirror of the kernel's $Ax=0$. That list is a recipe for the redundancy in the table, and it is the only recipe up to scaling. The four spaces together are the fundamental subspaces of the table [S:25:70], and the whole ledger now reads two plus two equals four for the inputs and two plus one equals three for the outputs.

In a data table the rows are items and the columns are measurements, and the ledger has a practical reading. The rank says how many genuinely distinct patterns the table contains, so a table whose rank is much smaller than its column count is storing the same information many times over. A nonzero kernel then names a combination of columns that cancels to zero, which is exactly a redundancy among the measurements, and a table with such a redundancy can be stored with fewer numbers than it has slots.

The ledger is not yet a picture. Nothing above used a notion of angle, so the four spaces are counted but not placed, and the natural depiction of them as perpendicular pairs needs the next chapter's dot product. The bases are also choices, since any independent selection of the right size will do; the four spaces themselves are not. The kernel and the column space are determined by the table, and so are the row space and the left null space, and a change of coordinates leaves all four and their dimensions alone.

The formal statement. Let $A$ be an $m\times n$ matrix. It defines four subspaces: the column space inside the output space, the kernel inside the input space, the row space inside the input space, and the left null space inside the output space [S:25:70]. Write $C(A)$ for the column space and $N(A)$ for the kernel, and write $r$ for the rank. Then

$$\dim C(A)=\dim C(A^{\mathsf{T}})=r,\dim N(A)=n-r,\dim N(A^{\mathsf{T}})=m-r.$$

The display says one number fixes all four dimensions, and the two sides of the table, inputs and outputs, are counted separately.

The derivation. Apply rank-nullity to $A$ and to its transpose, and the kernel and left-null dimensions drop out [S:23:215]. The step that needs work is the equality of row rank and column rank [S:27:53]. Elimination supplies it. Row moves do not change the row space, and the pivot columns of the original table are a basis of its column space [S:25:70]. The pivot count is therefore both the number of independent rows and the number of independent columns. Treil writes the argument with the echelon form in front of you, and the point to watch is that the pivot columns come from the original table, not from the reduced one [S:25:70].

The edge case. The four dimensions are exact integers, and data does not arrive exact. Perturb one entry and the rank can jump to the number of rows, while the table still behaves like a rank-two table on the data you care about. For a symmetric positive-semidefinite table the effective rank is the sum of the eigenvalues divided by the largest eigenvalue, and away from the zero table it is continuous in the entries [S:20:172][S:20:179]. For a data table that number often describes the table better than the exact rank, and it is what a practitioner means by "how many patterns are really here".

What L1 hid. L1 counted the four spaces and said that drawing them needs the next chapter. That is right, and the missing fact is stronger than a picture. The row space and the kernel are orthogonal complements inside the input space, and the column space and the left null space are orthogonal complements inside the output space. So each pair fills its whole space, not merely the rank. That statement needs the dot product, which Chapter 8 supplies.

Where the analogy breaks. A ledger suggests four independent accounts. They are not independent: one number sets all four, and changing the rank changes all four at once. The ledger also suggests that the four sit side by side. Two live among inputs and two among outputs, with different ambient sizes, and L1 said so. That placement is the part most easily lost when the four are drawn in a row.

Where the idea comes from. Treil names the four subspaces and proves the relations between their dimensions [S:25:70]. The version that places them is often called the fundamental theorem of linear algebra. Thomas warns that the name does not pick out one universally agreed statement, and his version adds the orthogonal decompositions and the SVD bases [S:26:21]. That is the version this book reaches in Chapter 9, once angles are available.

What the field has not settled. How many dimensions of a weight matrix actually matter. Exact rank counts nonzero eigenvalues, but for a positive semidefinite matrix the effective rank, the total divided by the largest, is continuous in the entries and can be far smaller [S:20:179]. Whether the updates produced by a given optimiser are low-rank is under test. One report finds that Adam's update has a skewed spectrum with a few dominant singular values, while Muon's update has equal singular values and full effective rank [S:400:30]. Evidence that would settle it is spectra measured across optimisers, widths and training runs, plus ablations that delete the small directions and record the loss.

What depends on it. Compression and adaptation both read the four spaces as a budget. The same ledger that gives a table its rank also says how many directions you must keep to keep its action, and Schmidt's approximation theorem, rediscovered by Eckart and Young and usually called the Eckart–Young theorem, says the leading terms are the best choice in the Frobenius norm [S:52:28]. A rank chosen for a weight matrix is therefore a statement about which of the four spaces you are willing to shrink.

What to read next. Read chapter 7 of Treil [S:25:70]. Read section 3.13 of Thomas for the SVD version of the theorem [S:26:21]. Read section 5.6 of Vershynin for effective rank and its stability [S:20:179]. Then take a small matrix, compute its four dimensions, and add noise until the exact rank changes. Watch which of the four answers moves first.