Derive: Directions That Stay Put

If the roots are ${\lambda }_1$ and ${\lambda }_2$, the polynomial factors as $$(\lambda -{\lambda }_1)(\lambda -{\lambda }_2)={\lambda }^2-({\lambda }_1+{\lambda }_2)\lambda +{\lambda }_1{\lambda }_2.$$ The characteristic polynomial is ${\lambda }^2-(\mathrm{tr}A)\lambda +\det A$, which you get by expanding $\det (A-\lambda I)=(a-\lambda )(d-\lambda )-bc$ and collecting terms. Two polynomials of degree two that agree at every $\lambda$ have the same coefficients, so the coefficient of $\lambda$ gives ${\lambda }_1+{\lambda }_2=\mathrm{tr}A$ and the constant term gives ${\lambda }_1{\lambda }_2=\det A$. This is the whole proof; no matrix entries beyond the trace and the determinant are used.

Now classify. If $\det A={\lambda }_1{\lambda }_2<0$, the two eigenvalues have opposite signs, so the form rises along one eigen-direction and falls along a perpendicular one. That is a saddle. If $\det A>0$, the two eigenvalues share a sign, and the shared sign equals the sign of their sum, which is the trace. A positive trace therefore gives two positive eigenvalues and a form positive in every direction; a negative trace gives two negative eigenvalues and a form negative in every direction. If $\det A=0$, one eigenvalue is zero, and along its eigenvector the form is zero while the vector is nonzero: a flat direction, which makes a trough rather than a bowl. The step from eigenvalues to the sign of $x^{\top }Ax$ everywhere uses section 9.5's formula $x^{\top }Ax={\lambda }_1{y}_1^2+{\lambda }_2{y}_2^2$ in the eigenbasis, and that formula needs the eigenbasis to be orthonormal, which the spectral theorem supplies for a symmetric matrix.

For the given matrix, $\mathrm{tr}A=8+1=9$ and $\det A=8\cdot 1-3\cdot 3=-1$. The determinant is negative, so the eigenvalues have opposite signs and the form is a saddle. The eigenvalues are $(9\pm \sqrt{85})/2$, about $9.11$ and $-0.11$; the trace and the determinant found the shape without them. Now evaluate. At $x=(1,-1)$ the product $Ax=(8-3,3-1)=(5,2)$, and $x^{\top }Ax=1\cdot 5+(-1)\cdot 2=3$. At $x=(1,-3)$ the product $Ax=(8-9,3-3)=(-1,0)$, and $x^{\top }Ax=1\cdot (-1)+(-3)\cdot 0=-1$. The value $3$ is positive and the value $-1$ is negative, which is exactly what a saddle does: it curves up in some directions and down in others. A positive value at one vector is not evidence of a bowl. The bowl condition quantifies over every nonzero vector, and one negative value is enough to refute it. The determinant settled the question before either value was computed, which is why section 9.5 calls the negative determinant a one-line test.

For $t=1$ the formula reads $\left(\begin{array}{cc}2& 1\\ 0& 2\end{array}\right)$, which is $A$. Suppose it holds at $t$. Then $$A^{t+1}=A^{t}A=\left(\begin{array}{cc}{2}^t& t{2}^{t-1}\\ 0& 2^t\end{array}\right)\left(\begin{array}{cc}2& 1\\ 0& 2\end{array}\right)=\left(\begin{array}{cc}{2}^{t+1}& 2^t+t{2}^t\\ 0& 2^{t+1}\end{array}\right).$$ The top-right entry is $2^t(1+t)=(t+1)2^t$, which is $(t+1)2^{(t+1)-1}$, the formula at $t+1$. The two diagonal entries are $2^{t+1}$, and the bottom-left entry stays zero. So the formula holds at $t+1$ whenever it holds at $t$, and it holds at $t=1$; by induction it holds for every $t\ge 1$.

Now the eigenvectors. The equation $Av=2v$ is $(A-2I)v=0$ with $A-2I=\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)$. The first row forces $v_2=0$ and leaves $v_1$ free, so every eigenvector is a nonzero multiple of $(1,0)$. The eigenspace has dimension $1$, not $2$, and there is no eigenbasis. A two-term formula would need a second independent eigenvector to carry the part of a start vector that points off the line. There is none. The formula would also need two eigenvalues to be different, and here they are equal, so the two terms would have the same growth and could not separate.

At $t=10$ the corner entry is $10\cdot 2^9=5120$, and the diagonal entry is $2^{10}=1024$. The ratio is $5120/1024=5$, which is $t/2$. That ratio is the extra factor section 9.2 mentions. A diagonal matrix with the same eigenvalue $2$ has powers $\left(\begin{array}{cc}{2}^t& 0\\ 0& 2^t\end{array}\right)$, whose entries grow like $2^t=1024$ at $t=10$. The shear's corner grows like $t{2}^{t-1}$, so it is larger than $2^t$ by the factor $t/2$. The growth rate is still set by $2^t$ in the long run, because a linear factor loses to any exponential base above one. But at every fixed $t$ the defective matrix is ahead, and the gap widens with $t$. So a defective matrix keeps the rate of its eigenvalue and adds a polynomial factor on top; that factor is the part the two-term formula cannot express.

Because $A^{\top }A$ is symmetric, the spectral theorem supplies an orthonormal basis $u_1,u_2$ of eigenvectors with real eigenvalues. Those eigenvalues are the squares of the singular values, so write them as ${\sigma }_1^2\ge {\sigma }_2^2\ge 0$, ordered so that ${\sigma }_1$ is the largest. For any vector $v$, write $v=a_1{u}_1+a_2{u}_2$. Orthonormality gives $\Vert v{\Vert }^2=a_1^2+a_2^2$. Then $$\Vert Av{\Vert }^2=v^{\top }{A}^{\top }Av={\sigma }_1^2{a}_1^2+{\sigma }_2^2{a}_2^2\le {\sigma }_1^2\left(a_1^2+a_2^2\right)={\sigma }_1^2\Vert v{\Vert }^2.$$ The middle equality holds because $u_1$ and $u_2$ are eigenvectors of $A^{\top }A$ and are perpendicular, so the cross terms vanish. Taking square roots gives $\Vert Av\Vert \le {\sigma }_1\Vert v\Vert$. Equality holds exactly when every term with ${\sigma }_i^2<{\sigma }_1^2$ has $a_i=0$, that is, when $v$ is a multiple of $u_1$. So ${\sigma }_1$ is the largest value of $\Vert Av\Vert$ on the set of unit vectors, which is the claim that the top singular value is the largest stretch.

For the given matrix, $$A^{\top }A=\left(\begin{array}{cc}2\cdot 2+1\cdot 1& 2\cdot 2+1\cdot (-1)\\ 2\cdot 2+(-1)\cdot 1& 2\cdot 2+(-1)(-1)\end{array}\right)=\left(\begin{array}{cc}5& 3\\ 3& 5\end{array}\right).$$ Its trace is $10$ and its determinant is $25-9=16$, so its eigenvalues solve ${\lambda }^2-10\lambda +16=0$, which factors as $(\lambda -8)(\lambda -2)$. The eigenvalues are $8$ and $2$, and the singular values are $\sqrt{8}=2\sqrt{2}\approx 2.828$ and $\sqrt{2}\approx 1.414$. Their product is $4$, and $|\det A|=|2\cdot (-1)-2\cdot 1|=|-4|=4$, as the check requires. The right singular vector for ${\sigma }_1$ is $(1,1)/\sqrt{2}$, and $A(1,1)/\sqrt{2}=(4,0)/\sqrt{2}$, whose length is $4/\sqrt{2}=2\sqrt{2}$, the bound met exactly. On the first axis the stretch is smaller: $A(1,0)=(2,1)$, of length $\sqrt{5}\approx 2.236$, below $2.828$. A bound that is met at one vector and slack at another is what a maximum looks like, and the spectral theorem is what turns the search over all unit vectors into a single eigenproblem.