Why A Transpose A Is a Special Matrix
In the study of the Singular Value Decomposition, the matrix \(\mathbf{A}^{\top}\mathbf{A}\) is important because it turns an arbitrary real matrix into a symmetric square matrix. That change makes the real Spectral Theorem available. The key fact behind the eigenvalue signs is an identity: the quadratic form of \(\mathbf{A}^{\top}\mathbf{A}\) at \(\mathbf{x}\) is the squared length of \(\mathbf{A}\mathbf{x}\). A squared length cannot be negative, and that observation controls the eigenvalues.
Let \(\mathbf{A}\in\mathbb{R}^{m\times n}\), where \(m,n\) are positive integers. Then \(\mathbf{A}^{\top}\in\mathbb{R}^{n\times m}\), so \(\mathbf{A}^{\top}\mathbf{A}\) is an \(n\times n\) matrix. For every \(\mathbf{x}\in\mathbb{R}^n\), the product \(\mathbf{A}\mathbf{x}\) lies in \(\mathbb{R}^m\), and the scalar \(\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}\) is therefore defined. The dimensions matter: this quadratic form is evaluated on vectors in the domain \(\mathbb{R}^n\).
The Quadratic-Form Identity
The Euclidean norm of a real column vector \(\mathbf{y}\in\mathbb{R}^m\) satisfies \(\|\mathbf{y}\|^2=\mathbf{y}^{\top}\mathbf{y}\). Apply this to \(\mathbf{y}=\mathbf{A}\mathbf{x}\). Using the rule \((\mathbf{A}\mathbf{x})^{\top}=\mathbf{x}^{\top}\mathbf{A}^{\top}\) gives the identity for every \(\mathbf{x}\in\mathbb{R}^n\).
Proof. Set \(\mathbf{B}=\mathbf{A}^{\top}\mathbf{A}\). The transpose rule for a product gives \(\mathbf{B}^{\top}=(\mathbf{A}^{\top}\mathbf{A})^{\top}=\mathbf{A}^{\top}(\mathbf{A}^{\top})^{\top}=\mathbf{A}^{\top}\mathbf{A}=\mathbf{B}\), so \(\mathbf{B}\) is symmetric. Now take any \(\mathbf{x}\in\mathbb{R}^n\). By the definition of the Euclidean norm and the same transpose rule, \(\mathbf{x}^{\top}\mathbf{B}\mathbf{x}=\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}=(\mathbf{A}\mathbf{x})^{\top}(\mathbf{A}\mathbf{x})=\|\mathbf{A}\mathbf{x}\|^2\ge0\). Thus \(\mathbf{B}\) is positive semidefinite, and the displayed identity holds for every \(\mathbf{x}\). \(\square\)
Symmetry and nonnegative quadratic-form values are separate requirements in the definition. The identity proves the second requirement, while the transpose calculation proves the first. Neither step depends on \(\mathbf{A}\) being square, invertible, or having independent columns. In particular, the argument applies to rectangular matrices and to matrices that send some nonzero vectors to zero.
Worked Example: A Rectangular Matrix and Its Quadratic Form
Let \(\mathbf{A}=\begin{bmatrix}1&2&0\\0&1&1\end{bmatrix}\). Compute \(\mathbf{A}^{\top}\mathbf{A}\), then verify the quadratic-form identity for an arbitrary \(\mathbf{x}\in\mathbb{R}^3\).
The columns of \(\mathbf{A}\) are \((1,0)^{\top}\), \((2,1)^{\top}\), and \((0,1)^{\top}\). Their pairwise dot products give the entries of \(\mathbf{A}^{\top}\mathbf{A}\):
Write \(\mathbf{x}=(x_1,x_2,x_3)^{\top}\). Direct multiplication gives \(\mathbf{A}\mathbf{x}=(x_1+2x_2,\;x_2+x_3)^{\top}\), so
Multiplying by the computed matrix gives the same expression: \(\mathbf{x}^{\top}(\mathbf{A}^{\top}\mathbf{A})\mathbf{x} =x_1(x_1+2x_2)+x_2(2x_1+5x_2+x_3)+x_3(x_2+x_3)\). Combining terms yields \(x_1^2+4x_1x_2+5x_2^2+2x_2x_3+x_3^2\), exactly as above.
Check: both calculations give the sum of two squares, \((x_1+2x_2)^2+(x_2+x_3)^2\), which is nonnegative for every real choice of \(x_1,x_2,x_3\). The displayed Gram matrix is symmetric, as the proposition requires.
Why Its Eigenvalues Are Real and Nonnegative
The real Spectral Theorem states that every real symmetric \(n\times n\) matrix has an orthonormal basis of real eigenvectors. Equivalently, it can be written as \(\mathbf{Q}\mathbf{D}\mathbf{Q}^{\top}\), where \(\mathbf{Q}\) is orthogonal and \(\mathbf{D}\) is diagonal with real entries. We use this standard result here; the theorem is a central result of real spectral theory. The proposition just proved supplies its hypothesis for \(\mathbf{A}^{\top}\mathbf{A}\).
Proof. By the proposition, \(\mathbf{B}=\mathbf{A}^{\top}\mathbf{A}\) is a real symmetric positive semidefinite \(n\times n\) matrix. The real Spectral Theorem gives an orthogonal matrix \(\mathbf{Q}\) and a real diagonal matrix \(\mathbf{D}\) such that \(\mathbf{B}=\mathbf{Q}\mathbf{D}\mathbf{Q}^{\top}\). Write the columns of \(\mathbf{Q}\) as \(\mathbf{q}_1,\ldots,\mathbf{q}_n\), and the diagonal entries of \(\mathbf{D}\) as \(\lambda_1,\ldots,\lambda_n\). Orthogonality gives \(\mathbf{Q}^{\top}\mathbf{q}_i=\mathbf{e}_i\), so \(\mathbf{B}\mathbf{q}_i=\mathbf{Q}\mathbf{D}\mathbf{e}_i=\lambda_i\mathbf{q}_i\). Thus each \(\lambda_i\) is an eigenvalue, and each is real because \(\mathbf{D}\) has real entries. Also \(\|\mathbf{q}_i\|=1\), so positive semidefiniteness gives \(0\le\mathbf{q}_i^{\top}\mathbf{B}\mathbf{q}_i =\mathbf{q}_i^{\top}(\lambda_i\mathbf{q}_i) =\lambda_i\|\mathbf{q}_i\|^2 =\lambda_i\). Therefore every diagonal entry of \(\mathbf{D}\) is nonnegative. The Spectral Theorem says these entries account for all eigenvalues of \(\mathbf{B}\), proving the claim. \(\square\)
The order of the reasoning is useful: symmetry lets the Spectral Theorem identify real eigenvalues, and positive semidefiniteness forces their signs. For an eigenvector \(\mathbf{v}\ne\mathbf{0}\) with eigenvalue \(\lambda\), the same sign argument reads \(\lambda\|\mathbf{v}\|^2=\mathbf{v}^{\top}\mathbf{B}\mathbf{v}\ge0\). Since \(\|\mathbf{v}\|^2>0\), it follows that \(\lambda\ge0\), whenever \(\lambda\) is known to be real. The Spectral Theorem supplies that reality for symmetric \(\mathbf{B}\).
Worked Example: A Matrix with Two Positive Eigenvalues
Take \(\mathbf{A}=\begin{bmatrix}1&1\\0&1\end{bmatrix}\). Compute the eigenvalues of \(\mathbf{A}^{\top}\mathbf{A}\) and check eigenvectors for both.
First, \(\mathbf{A}^{\top}=\begin{bmatrix}1&0\\1&1\end{bmatrix}\), hence
The characteristic equation is \(\det\begin{bmatrix}1-\lambda&1\\1&2-\lambda\end{bmatrix} =(1-\lambda)(2-\lambda)-1 =\lambda^2-3\lambda+1=0\). The quadratic formula gives \(\lambda_1=(3+\sqrt{5})/2\) and \(\lambda_2=(3-\sqrt{5})/2\). Both are positive because \(3>\sqrt{5}\), which follows by squaring the positive numbers and comparing \(9>5\).
For either root \(\lambda\), set \(\mathbf{v}=(1,\lambda-1)^{\top}\), which is nonzero. Multiplying back gives \(\begin{bmatrix}1&1\\1&2\end{bmatrix}\mathbf{v} =(\lambda,\,2\lambda-1)^{\top}\). The characteristic equation says \(\lambda^2-3\lambda+1=0\), or \(2\lambda-1=\lambda(\lambda-1)\). Therefore the product is \((\lambda,\,\lambda(\lambda-1))^{\top}=\lambda\mathbf{v}\).
Check: the two values satisfy the computed characteristic equation, and the multiplication verifies an eigenvector for each root. Their positivity agrees with the theorem.
Zero Eigenvalues and the Difference from Positive Definite
Positive semidefinite does not mean positive definite. The identity gives an exact test for when the quadratic form is zero: \(\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}=0\) precisely when \(\|\mathbf{A}\mathbf{x}\|^2=0\), which occurs precisely when \(\mathbf{A}\mathbf{x}=\mathbf{0}\). Thus, if \(\mathbf{A}\) sends a nonzero vector to zero, then \(\mathbf{A}^{\top}\mathbf{A}\) has a zero quadratic-form value at that vector and cannot be positive definite. Its positive semidefiniteness still holds.
Worked Example: A Gram Matrix with Zero Eigenvalues
Let \(\mathbf{A}=\begin{bmatrix}1&2&3\\2&4&6\end{bmatrix}\). Compute \(\mathbf{A}^{\top}\mathbf{A}\) and exhibit its eigenvalues, including the zeros.
Multiplication gives
For \(\mathbf{v}=(1,2,3)^{\top}\), the matrix-vector product is \((\mathbf{A}^{\top}\mathbf{A})\mathbf{v}=5(1,2,3)^{\top}(1+4+9)=70\mathbf{v}\). So \(70\) is an eigenvalue. Now take \(\mathbf{u}=(-2,1,0)^{\top}\) and \(\mathbf{w}=(-3,0,1)^{\top}\). Since \(1(-2)+2(1)+3(0)=0\) and \(1(-3)+2(0)+3(1)=0\), the displayed outer-product form gives \((\mathbf{A}^{\top}\mathbf{A})\mathbf{u}=\mathbf{0}\) and \((\mathbf{A}^{\top}\mathbf{A})\mathbf{w}=\mathbf{0}\). Both are nonzero vectors, so \(0\) is an eigenvalue.
The three vectors are linearly independent: \(\mathbf{u}\) and \(\mathbf{w}\) are not multiples of one another, and both are orthogonal to the nonzero vector \(\mathbf{v}\). They therefore form a basis of \(\mathbb{R}^3\), so these eigenvectors account for the full set of eigenvalues: \(70,0,0\).
Check: \(\mathbf{A}\mathbf{u}=\mathbf{0}\) and \(\mathbf{A}\mathbf{w}=\mathbf{0}\), while \(\mathbf{A}\mathbf{v}=(14,28)^{\top}\). Multiplying by \(\mathbf{A}^{\top}\) gives \(\mathbf{A}^{\top}(14,28)^{\top}=(70,140,210)^{\top}=70\mathbf{v}\), confirming the nonzero eigenvalue calculation as well.
A Common Mistake: Replacing Semidefinite with Definite
A frequent error is to infer that \(\mathbf{A}^{\top}\mathbf{A}\) is positive definite simply because its quadratic form is a squared norm. A squared norm is nonnegative, but it can be zero for a nonzero vector if \(\mathbf{A}\mathbf{x}=\mathbf{0}\). For a concrete example, let \(\mathbf{A}=\begin{bmatrix}1&2\\2&4\end{bmatrix}\). Then \(\mathbf{A}^{\top}\mathbf{A}=\begin{bmatrix}5&10\\10&20\end{bmatrix}\). The incorrect calculation would claim strict positivity for every nonzero \(\mathbf{x}\). But with \(\mathbf{x}=(-2,1)^{\top}\ne\mathbf{0}\),
The correction is to conclude positive semidefiniteness, not positive definiteness, unless strict positivity has separately been established for every nonzero \(\mathbf{x}\). The example also illustrates why zero eigenvalues are permitted by the theorem.
Why This Fact Is Useful
The matrix \(\mathbf{A}^{\top}\mathbf{A}\) is square even when \(\mathbf{A}\) is rectangular, and its symmetry makes its eigenvalues accessible through the real Spectral Theorem. The nonnegative sign conclusion is not an extra assumption about \(\mathbf{A}\); it follows directly from the squared-norm identity. This is the foundation for using the eigenvalues of \(\mathbf{A}^{\top}\mathbf{A}\) in the Singular Value Decomposition. The present argument establishes the sign and reality of those eigenvalues without requiring \(\mathbf{A}\) to be square or invertible.
Check Your Understanding
Use the quadratic-form identity and the real Spectral Theorem to justify your conclusions.
- If \(\mathbf{A}\in\mathbb{R}^{4\times 2}\), what are the dimensions of \(\mathbf{A}^{\top}\mathbf{A}\), and why is it symmetric?
- For \(\mathbf{A}=\begin{bmatrix}1&0\\1&1\end{bmatrix}\), compute \(\mathbf{A}^{\top}\mathbf{A}\) and evaluate its quadratic form at \((2,-1)^{\top}\).
- Explain why positive semidefiniteness alone does not imply positive definiteness.
- State which part of the argument establishes that the eigenvalues are real, and which part establishes that they are nonnegative.
- Give a condition on a nonzero \(\mathbf{x}\) that makes \(\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}=0\), and verify it from the identity.
