Two Gram Matrices, One Positive Spectrum
For a rectangular matrix, the products \(\mathbf{A}^{*}\mathbf{A}\) and \(\mathbf{A}\mathbf{A}^{*}\) have different sizes. The first acts on the input space of \(\mathbf{A}\); the second acts on its output space. Their eigenvectors therefore live in different spaces, and the matrices need not have the same zero eigenvalues. Yet their positive eigenvalues agree, including multiplicities. The key is that multiplication by \(\mathbf{A}\) carries an eigenvector of one product to an eigenvector of the other, and multiplication by \(\mathbf{A}^{*}\) carries it back after rescaling.
Work over \(\mathbb{C}\). Let \(m,n\in\mathbb{N}\), and let \(\mathbf{A}\in\mathbb{C}^{m\times n}\). Its conjugate transpose \(\mathbf{A}^{*}\) is the \(n\times m\) matrix obtained by transposing and conjugating each entry. Thus \(\mathbf{A}^{*}\mathbf{A}\) is \(n\times n\), while \(\mathbf{A}\mathbf{A}^{*}\) is \(m\times m\). Both are Hermitian: each equals its own conjugate transpose.
First, these two products cannot have negative or nonreal eigenvalues. For any \(\mathbf{x}\in\mathbb{C}^{n}\), \(\mathbf{x}^{*}\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\|\mathbf{A}\mathbf{x}\|^{2}\), which is a nonnegative real number. If \(\mathbf{x}\ne\mathbf{0}\) is an eigenvector with eigenvalue \(\lambda\), then this equality becomes \(\lambda\|\mathbf{x}\|^{2}=\|\mathbf{A}\mathbf{x}\|^{2}\). Since \(\|\mathbf{x}\|^{2}>0\), \(\lambda\) is real and nonnegative. The same argument applied to \(\mathbf{A}^{*}\) shows that every eigenvalue of \(\mathbf{A}\mathbf{A}^{*}\) is real and nonnegative.
The Eigenvector Maps
Fix \(\lambda>0\). If \(\mathbf{x}\in E_{\lambda}\) for \(\mathbf{A}^{*}\mathbf{A}\), then \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\lambda\mathbf{x}\). Associativity of matrix multiplication gives \(\mathbf{A}\mathbf{A}^{*}(\mathbf{A}\mathbf{x})=\lambda\mathbf{A}\mathbf{x}\). So \(\mathbf{A}\mathbf{x}\) belongs to the \(\lambda\)-eigenspace of \(\mathbf{A}\mathbf{A}^{*}\). It is important to handle the zero vector precisely: if \(\mathbf{x}\ne\mathbf{0}\), then \(\mathbf{A}\mathbf{x}\ne\mathbf{0}\), because \(\mathbf{A}\mathbf{x}=\mathbf{0}\) would imply \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\mathbf{0}=\lambda\mathbf{x}\), and \(\lambda>0\) would force \(\mathbf{x}=\mathbf{0}\). The zero vector maps to the zero vector.
Proof. Fix \(\lambda>0\). If \(\mathbf{x}\in E_{\lambda}(\mathbf{A}^{*}\mathbf{A})\), then \(\mathbf{A}\mathbf{A}^{*}(\mathbf{A}\mathbf{x})=\mathbf{A}(\mathbf{A}^{*}\mathbf{A}\mathbf{x})=\lambda\mathbf{A}\mathbf{x}\), so \(\mathbf{A}\mathbf{x}\in E_{\lambda}(\mathbf{A}\mathbf{A}^{*})\). This also holds when \(\mathbf{x}=\mathbf{0}\). In the reverse direction, if \(\mathbf{y}\in E_{\lambda}(\mathbf{A}\mathbf{A}^{*})\), then \(\mathbf{A}^{*}\mathbf{A}(\mathbf{A}^{*}\mathbf{y})=\mathbf{A}^{*}(\mathbf{A}\mathbf{A}^{*}\mathbf{y})=\lambda\mathbf{A}^{*}\mathbf{y}\), so \(\mathbf{A}^{*}\mathbf{y}\in E_{\lambda}(\mathbf{A}^{*}\mathbf{A})\), including when \(\mathbf{y}=\mathbf{0}\).
Define \(S(\mathbf{x})=\mathbf{A}\mathbf{x}\) on the first eigenspace and \(T(\mathbf{y})=\frac{1}{\lambda}\mathbf{A}^{*}\mathbf{y}\) on the second. For every \(\mathbf{x}\in E_{\lambda}(\mathbf{A}^{*}\mathbf{A})\) and every \(\mathbf{y}\in E_{\lambda}(\mathbf{A}\mathbf{A}^{*})\), including the zero vectors, \(T(S(\mathbf{x}))=\frac{1}{\lambda}\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\mathbf{x}\) and \(S(T(\mathbf{y}))=\frac{1}{\lambda}\mathbf{A}\mathbf{A}^{*}\mathbf{y}=\mathbf{y}\). Thus the maps are inverses and the eigenspaces have equal dimension. Both Gram matrices are Hermitian. By the Spectral Theorem, each is diagonalizable, so for each eigenvalue its algebraic multiplicity equals its geometric multiplicity. Equal eigenspace dimensions therefore give equal algebraic multiplicities for every \(\lambda>0\). The two implications also show that \(\lambda>0\) is an eigenvalue of one product exactly when it is an eigenvalue of the other. \(\square\)
Worked Examples
Worked Example: A Tall Matrix with a Repeated Positive Eigenvalue
Let \(\mathbf{A}=\begin{bmatrix}1&0\\0&1\\0&0\end{bmatrix}\in\mathbb{C}^{3\times2}\). Compute both products and compare their spectra and multiplicities.
Here \(\mathbf{A}^{*}=\begin{bmatrix}1&0&0\\0&1&0\end{bmatrix}\). Direct multiplication gives
The first matrix has eigenvalue \(1\) with eigenspace \(\mathbb{C}^{2}\), so its algebraic and geometric multiplicities are both \(2\). The second has eigenvalue \(1\) with eigenspace \(\{(y_1,y_2,0)^{\top}:y_1,y_2\in\mathbb{C}\}\), also of dimension \(2\), and eigenvalue \(0\) with eigenspace \(\operatorname{span}\{(0,0,1)^{\top}\}\). For an explicit check of the map, \(\mathbf{A}(x_1,x_2)^{\top}=(x_1,x_2,0)^{\top}\); this carries the entire \(1\)-eigenspace of \(\mathbf{A}^{*}\mathbf{A}\) bijectively onto the \(1\)-eigenspace of \(\mathbf{A}\mathbf{A}^{*}\). Conversely, \(\mathbf{A}^{*}(y_1,y_2,0)^{\top}=(y_1,y_2)^{\top}\). Thus the positive eigenvalue and its multiplicity agree. The extra zero eigenvalue occurs in the larger, \(3\times3\) product.
Worked Example: A Wide Matrix with Different Zero Multiplicities
Let \(\mathbf{A}=\begin{bmatrix}1&0&0\\0&2&0\end{bmatrix}\in\mathbb{C}^{2\times3}\). Find the spectra and verify the positive eigenvectors map in both directions.
Since \(\mathbf{A}^{*}=\begin{bmatrix}1&0\\0&2\\0&0\end{bmatrix}\), multiplication gives
The positive eigenvalues are \(1\) and \(4\), each with multiplicity \(1\) in both products. For \(\lambda=1\), the eigenvector \(\mathbf{e}_1=(1,0,0)^{\top}\) of \(\mathbf{A}^{*}\mathbf{A}\) maps to \(\mathbf{A}\mathbf{e}_1=(1,0)^{\top}\), and \(\mathbf{A}\mathbf{A}^{*}(1,0)^{\top}=(1,0)^{\top}\). Applying the proposed inverse gives \(\frac{1}{1}\mathbf{A}^{*}(1,0)^{\top}=(1,0,0)^{\top}\).
For \(\lambda=4\), \(\mathbf{e}_2=(0,1,0)^{\top}\) maps to \(\mathbf{A}\mathbf{e}_2=(0,2)^{\top}\), and \(\mathbf{A}\mathbf{A}^{*}(0,2)^{\top}=(0,8)^{\top}=4(0,2)^{\top}\). The inverse map returns \(\frac14\mathbf{A}^{*}(0,2)^{\top}=\frac14(0,4,0)^{\top}=(0,1,0)^{\top}\). The \(3\times3\) product has an additional zero eigenvalue, with eigenspace \(\operatorname{span}\{\mathbf{e}_3\}\); the \(2\times2\) product has none. This difference is consistent with the theorem, which concerns only positive eigenvalues.
Worked Example: Matching Eigenvalues Without Matching Entries
Take \(\mathbf{A}=\begin{bmatrix}1&1&0\\0&1&0\end{bmatrix}\). Compute the two products and verify their shared positive eigenvalues using eigenvectors.
Multiplication gives
The nonzero \(2\times2\) block of the first product and the second product both have characteristic polynomial \(\lambda^2-3\lambda+1\): for the first block, \((1-\lambda)(2-\lambda)-1=\lambda^2-3\lambda+1\); for the second, \((2-\lambda)(1-\lambda)-1=\lambda^2-3\lambda+1\). Its roots are \(\lambda_{+}=\frac{3+\sqrt5}{2}\) and \(\lambda_{-}=\frac{3-\sqrt5}{2}\), both positive since \(0<\sqrt5<3\), and distinct. The \(3\times3\) product also has one zero eigenvalue from its third coordinate.
For either root \(\lambda\), the vector \(\mathbf{x}=(1,\lambda-1,0)^{\top}\) is an eigenvector of \(\mathbf{A}^{*}\mathbf{A}\). Indeed, \(\lambda^2-3\lambda+1=0\) gives \(\lambda^2=3\lambda-1\), and hence
Its image is \(\mathbf{A}\mathbf{x}=(\lambda,\lambda-1)^{\top}=\lambda(1,\lambda-2)^{\top}\), which is nonzero because \(\lambda>0\). To check the target eigenvector directly, let \(\mathbf{y}=(1,\lambda-2)^{\top}\). Then \(\mathbf{A}\mathbf{A}^{*}\mathbf{y}=(\lambda,\lambda-1)^{\top}=\lambda\mathbf{y}\), using \(\lambda(\lambda-2)=\lambda-1\), which follows from \(\lambda^2-3\lambda+1=0\). Finally, \(\frac{1}{\lambda}\mathbf{A}^{*}(\mathbf{A}\mathbf{x})=\mathbf{x}\). Each of the two distinct positive eigenvalues therefore appears once in each product, although the displayed matrices have different sizes and entries.
Why the Zero Eigenvalues Can Differ
The eigenvector correspondence depends on dividing by \(\lambda\), so it applies when \(\lambda>0\), not when \(\lambda=0\). Indeed, zero eigenvectors behave differently: \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\mathbf{0}\) holds exactly when \(\mathbf{A}\mathbf{x}=\mathbf{0}\), since \(\mathbf{x}^{*}\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\|\mathbf{A}\mathbf{x}\|^{2}\). Likewise, \(\mathbf{A}\mathbf{A}^{*}\mathbf{y}=\mathbf{0}\) holds exactly when \(\mathbf{A}^{*}\mathbf{y}=\mathbf{0}\). These are null spaces in different dimensions. In particular, the two full spectra, including zero, need not be identical as lists with multiplicities.
There is a quick dimension check that makes the distinction visible. The matrices have sizes \(n\times n\) and \(m\times m\), so their total eigenvalue multiplicities are \(n\) and \(m\), respectively, by the Spectral Theorem. Their positive eigenvalues use the same total multiplicity on both sides. Any difference in the remaining multiplicity must therefore be at zero: the difference between the zero multiplicities is \(n-m\). This statement allows either product to have more zeros, depending on whether \(n\) or \(m\) is larger.
Common Pitfall: The Zero Vector Is in Every Eigenspace
A careless proof might say, “For every \(\mathbf{x}\in E_{\lambda}(\mathbf{A}^{*}\mathbf{A})\), \(\mathbf{A}\mathbf{x}\ne\mathbf{0}\).” That is false: every eigenspace contains \(\mathbf{0}\), and \(\mathbf{A}\mathbf{0}=\mathbf{0}\). The correct statement is that \(\mathbf{A}\mathbf{x}\ne\mathbf{0}\) when \(\mathbf{x}\ne\mathbf{0}\) and \(\lambda>0\). If \(\mathbf{A}\mathbf{x}=\mathbf{0}\), then \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\mathbf{0}\); the eigenvalue equation also gives \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\lambda\mathbf{x}\), so \(\lambda\mathbf{x}=\mathbf{0}\). Positivity of \(\lambda\) forces \(\mathbf{x}=\mathbf{0}\). The zero vector itself maps to zero, as it must for a linear map.
A second error is to conclude that the two matrices have identical spectra with all multiplicities just because their positive eigenvalues match. The worked examples show why that inference fails: the products can have different sizes, and their zero multiplicities can differ. The precise conclusion is equality of the positive eigenvalues with multiplicities; the zero eigenvalues must be considered separately.
Check Your Understanding
Use the eigenvector maps and, where appropriate, verify each claim by multiplying the matrices.
- For \(\mathbf{A}=\begin{bmatrix}2&0\\0&0\\0&1\end{bmatrix}\), compute \(\mathbf{A}^{*}\mathbf{A}\) and \(\mathbf{A}\mathbf{A}^{*}\). Compare their positive eigenvalues and zero multiplicities.
- Explain why \(\mathbf{A}\mathbf{x}\) is nonzero if \(\mathbf{x}\ne\mathbf{0}\), \(\mathbf{A}^{*}\mathbf{A}\mathbf{x}=\lambda\mathbf{x}\), and \(\lambda>0\).
- For a fixed \(\lambda>0\), write down the proposed inverse of the map \(\mathbf{x}\mapsto\mathbf{A}\mathbf{x}\) between the two \(\lambda\)-eigenspaces and verify both compositions.
- Why does the argument mapping eigenvectors through \(\mathbf{A}\) not prove a correspondence for \(\lambda=0\)?
- If \(m=4\) and \(n=2\), what can be said about the difference between the zero multiplicities once the shared positive eigenvalues are counted?
