Why Four Equations Give One Matrix
A rectangular matrix may have no ordinary inverse, and even when a matrix is singular, a useful generalized inverse can still be defined. The Moore-Penrose equations specify the properties such a matrix must satisfy. The central question here is whether those properties determine exactly one matrix. The answer is yes: the equations force the products with the original matrix to be particular orthogonal projectors, and those projectors determine the candidate completely.
Work over \(\mathbb{F}=\mathbb{R}\) or \(\mathbb{C}\), with the conjugate transpose denoted by \(*\). Let \(\mathbf{A}\in\mathbb{F}^{m\times n}\), and let \(\mathbf{G}\in\mathbb{F}^{n\times m}\). The dimensions matter: \(\mathbf{A}\mathbf{G}\) is \(m\times m\), while \(\mathbf{G}\mathbf{A}\) is \(n\times n\).
A square matrix \(\mathbf{P}\) is an orthogonal projector if \(\mathbf{P}^2=\mathbf{P}\) and \(\mathbf{P}^*=\mathbf{P}\). Idempotence means applying the matrix twice has the same effect as applying it once; the Hermitian condition makes the projection orthogonal with respect to the usual inner product. We will use the fact that an orthogonal projector is uniquely determined by either its image or its kernel.
Proof. For any vector \(\mathbf{x}\), write \(\mathbf{x}=\mathbf{P}\mathbf{x}+(\mathbf{I}-\mathbf{P})\mathbf{x}\). The first term lies in \(\operatorname{im}\mathbf{P}\). The second lies in \(\ker\mathbf{P}\), since \(\mathbf{P}(\mathbf{I}-\mathbf{P})\mathbf{x}=(\mathbf{P}-\mathbf{P}^2)\mathbf{x}=\mathbf{0}\). If \(\mathbf{y}=\mathbf{P}\mathbf{z}\) lies in the image and \(\mathbf{w}\) lies in the kernel, then \(\langle\mathbf{y},\mathbf{w}\rangle=\langle\mathbf{P}\mathbf{z},\mathbf{w}\rangle=\langle\mathbf{z},\mathbf{P}\mathbf{w}\rangle=0\), using \(\mathbf{P}^*=\mathbf{P}\). Thus the image and kernel are orthogonal complements. For a fixed subspace, there is only one orthogonal projection onto it: each vector decomposes uniquely as a vector in that subspace plus a vector in its orthogonal complement. So equal images give equal projectors. Equal kernels also give equal orthogonal complements, hence equal images and equal projectors. \(\square\)
The SVD Candidate and Its Projectors
Take a singular value decomposition \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^*\), where \(\mathbf{U}\) and \(\mathbf{V}\) are unitary (orthogonal over \(\mathbb{R}\)). Suppose \(\mathbf{A}\) has rank \(r\), so its positive singular values are \(\sigma_1,\dots,\sigma_r\). The SVD-defined pseudoinverse is \(\mathbf{A}^{+}=\mathbf{V}\boldsymbol{\Sigma}^{+}\mathbf{U}^*\), where \(\boldsymbol{\Sigma}^{+}\) has reciprocal entries \(1/\sigma_i\) in the positions corresponding to the positive singular values and zeros elsewhere.
In the SVD coordinates, \(\mathbf{A}\mathbf{A}^{+}\) and \(\mathbf{A}^{+}\mathbf{A}\) are diagonal projections: the first keeps the left singular-vector coordinates for positive singular values, and the second keeps the corresponding right singular-vector coordinates. In particular, both are Hermitian and idempotent. The diagonal products also give \(\mathbf{A}\mathbf{A}^{+}\mathbf{A}=\mathbf{A}\) and \(\mathbf{A}^{+}\mathbf{A}\mathbf{A}^{+}=\mathbf{A}^{+}\), so \(\mathbf{A}^{+}\) satisfies all four equations.
Proof. Let \(\mathbf{G}\) satisfy the four equations. Set \(\mathbf{P}=\mathbf{A}\mathbf{G}\) and \(\mathbf{Q}=\mathbf{G}\mathbf{A}\). The equations give \(\mathbf{P}^*=\mathbf{P}\) and \(\mathbf{Q}^*=\mathbf{Q}\). They also give \(\mathbf{P}^2=\mathbf{A}\mathbf{G}\mathbf{A}\mathbf{G}=\mathbf{A}\mathbf{G}=\mathbf{P}\) and \(\mathbf{Q}^2=\mathbf{G}\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{G}\mathbf{A}=\mathbf{Q}\). Thus both products are orthogonal projectors.
The image of \(\mathbf{P}\) is \(\operatorname{im}\mathbf{A}\). Indeed, \(\mathbf{P}\mathbf{x}=\mathbf{A}(\mathbf{G}\mathbf{x})\) belongs to \(\operatorname{im}\mathbf{A}\), so \(\operatorname{im}\mathbf{P}\subseteq\operatorname{im}\mathbf{A}\). Conversely, if \(\mathbf{y}=\mathbf{A}\mathbf{x}\), then \(\mathbf{P}\mathbf{y}=\mathbf{A}\mathbf{G}\mathbf{A}\mathbf{x}=\mathbf{A}\mathbf{x}=\mathbf{y}\). Therefore every vector in \(\operatorname{im}\mathbf{A}\) is in \(\operatorname{im}\mathbf{P}\).
Next, \(\ker\mathbf{Q}=\ker\mathbf{A}\). If \(\mathbf{A}\mathbf{x}=\mathbf{0}\), then \(\mathbf{Q}\mathbf{x}=\mathbf{G}\mathbf{A}\mathbf{x}=\mathbf{0}\). Conversely, if \(\mathbf{Q}\mathbf{x}=\mathbf{0}\), then \(\mathbf{A}\mathbf{x}=\mathbf{A}\mathbf{G}\mathbf{A}\mathbf{x}=\mathbf{A}\mathbf{Q}\mathbf{x}=\mathbf{0}\).
The SVD shows that \(\mathbf{A}\mathbf{A}^{+}\) is an orthogonal projector with image \(\operatorname{im}\mathbf{A}\), and \(\mathbf{A}^{+}\mathbf{A}\) is an orthogonal projector with kernel \(\ker\mathbf{A}\): in right singular-vector coordinates, the latter keeps precisely the coordinates for positive singular values. By the lemma, projectors with the same image or kernel are equal. Hence \(\mathbf{A}\mathbf{G}=\mathbf{A}\mathbf{A}^{+}\) and \(\mathbf{G}\mathbf{A}=\mathbf{A}^{+}\mathbf{A}\).
Finally, use the second Moore-Penrose equation and then these projector equalities: \(\mathbf{G}=\mathbf{G}\mathbf{A}\mathbf{G}=\mathbf{G}(\mathbf{A}\mathbf{G})=\mathbf{G}(\mathbf{A}\mathbf{A}^{+})=(\mathbf{G}\mathbf{A})\mathbf{A}^{+}=(\mathbf{A}^{+}\mathbf{A})\mathbf{A}^{+}=\mathbf{A}^{+}\mathbf{A}\mathbf{A}^{+}=\mathbf{A}^{+}\). Thus any matrix satisfying the four equations is the SVD-defined pseudoinverse. \(\square\)
Worked Projector Checks
Worked Example: A Rectangular Matrix with a Zero Singular Direction
Let \(\mathbf{A}=\begin{bmatrix}1&0&0\\0&2&0\end{bmatrix}\) and \(\mathbf{G}=\begin{bmatrix}1&0\\0&\tfrac12\\0&0\end{bmatrix}\). Check that \(\mathbf{G}\) satisfies the four equations and identify the two projectors.
First multiply in each order:
Both products are Hermitian and idempotent: the first is \(\mathbf{I}_2\), and squaring the second leaves its diagonal entries \(1,1,0\) unchanged. Then \(\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{I}_2\mathbf{A}=\mathbf{A}\), and \(\mathbf{G}\mathbf{A}\mathbf{G}=(\mathbf{G}\mathbf{A})\mathbf{G}=\mathbf{G}\), since the last row of \(\mathbf{G}\) is zero. Thus all four equations hold. The first projector keeps every vector in \(\mathbb{R}^2\); the second keeps the first two coordinates in \(\mathbb{R}^3\) and removes the third, which is the zero singular direction.
Check: the products above have the required sizes, \(2\times2\) and \(3\times3\), respectively, and direct multiplication gives both triple products exactly as claimed.
Worked Example: A Rank-Deficient Matrix
Take \(\mathbf{A}=\begin{bmatrix}1&1\\0&0\end{bmatrix}\). Its rank-one SVD has left singular vector \((1,0)^{\top}\), singular value \(\sqrt{2}\), and right singular vector \((1,1)^{\top}/\sqrt{2}\). Therefore the SVD formula gives \(\mathbf{A}^{+}=\begin{bmatrix}\tfrac12&0\\\tfrac12&0\end{bmatrix}\). Verify the equations and use the projector argument to see what any other candidate \(\mathbf{H}\) must do.
Multiplication gives
The first matrix is Hermitian and idempotent. For the second, each entry of its square is \(\tfrac14+\tfrac14=\tfrac12\), so its square is itself; it is also symmetric. Thus both products are Hermitian idempotents. Multiplying once more gives \(\mathbf{A}\mathbf{A}^{+}\mathbf{A}=\begin{bmatrix}1&1\\0&0\end{bmatrix}=\mathbf{A}\) and \(\mathbf{A}^{+}\mathbf{A}\mathbf{A}^{+}=\begin{bmatrix}\tfrac12&0\\\tfrac12&0\end{bmatrix}=\mathbf{A}^{+}\).
If \(\mathbf{H}\) satisfies the four equations for this \(\mathbf{A}\), the theorem forces \(\mathbf{A}\mathbf{H}=\mathbf{A}\mathbf{A}^{+}\) and \(\mathbf{H}\mathbf{A}=\mathbf{A}^{+}\mathbf{A}\). Moreover, \(\mathbf{H}=\mathbf{H}(\mathbf{A}\mathbf{H})=(\mathbf{H}\mathbf{A})\mathbf{A}^{+}=(\mathbf{A}^{+}\mathbf{A})\mathbf{A}^{+}=\mathbf{A}^{+}\). The projector constraints therefore leave no freedom to choose a different candidate.
Check: the first product projects onto the span of \((1,0)^{\top}\), the column space of \(\mathbf{A}\). The second projects onto the span of \((1,1)^{\top}\), and its kernel is the span of \((1,-1)^{\top}\), which is exactly \(\ker\mathbf{A}\).
Common Pitfall: Omitting the Hermitian Conditions
Worked Example: The First Two Equations Do Not Ensure Uniqueness
Let \(\mathbf{A}=\begin{bmatrix}1&0\\0&0\end{bmatrix}\) and propose \(\mathbf{H}=\begin{bmatrix}1&1\\0&0\end{bmatrix}\). This is not the Moore-Penrose pseudoinverse of \(\mathbf{A}\). The correct pseudoinverse is \(\mathbf{A}^{+}=\mathbf{A}\).
Indeed, \(\mathbf{A}\mathbf{H}=\mathbf{H}\), so \(\mathbf{A}\mathbf{H}\mathbf{A}=\mathbf{H}\mathbf{A}=\mathbf{A}\). Also \(\mathbf{H}\mathbf{A}=\mathbf{A}\), and therefore \(\mathbf{H}\mathbf{A}\mathbf{H}=\mathbf{A}\mathbf{H}=\mathbf{H}\). So the first two equations hold. But \(\mathbf{A}\mathbf{H}=\begin{bmatrix}1&1\\0&0\end{bmatrix}\) is not symmetric: its \((1,2)\) entry is \(1\), while its \((2,1)\) entry is \(0\). Thus \((\mathbf{A}\mathbf{H})^*\ne\mathbf{A}\mathbf{H}\), and the third equation fails.
Correction: check all four equations, not only the two triple-product identities. The Hermitian requirements are what make the products orthogonal projectors; without them, the projector uniqueness argument does not apply.
What the Uniqueness Proof Contributes
The equations \(\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{A}\) and \(\mathbf{G}\mathbf{A}\mathbf{G}=\mathbf{G}\) alone are not the whole definition. Together with the two Hermitian conditions, they make \(\mathbf{A}\mathbf{G}\) the orthogonal projector onto \(\operatorname{im}\mathbf{A}\) and \(\mathbf{G}\mathbf{A}\) the orthogonal projector whose kernel is \(\ker\mathbf{A}\). Those subspaces are fixed by \(\mathbf{A}\), so the two projectors are fixed as well. The final identity in the proof then recovers \(\mathbf{G}\) from them.
This explains both the logic and the practical check: a candidate cannot be justified by matching only one product or by satisfying only the first two equations. Verify the Hermitian idempotent products, and the uniqueness theorem identifies the candidate with the SVD-defined pseudoinverse.
Check Your Understanding
Use the dimensions and projector properties carefully in each response.
- For \(\mathbf{G}\) satisfying the four Moore-Penrose equations, prove directly that \((\mathbf{A}\mathbf{G})^2=\mathbf{A}\mathbf{G}\).
- Why does \(\ker(\mathbf{G}\mathbf{A})=\ker\mathbf{A}\) require both inclusions?
- For the matrix in the rank-deficient worked example, state the image of \(\mathbf{A}\mathbf{A}^{+}\) and the kernel of \(\mathbf{A}^{+}\mathbf{A}\).
- Explain which Moore-Penrose condition fails for the proposed \(\mathbf{H}\) in the common-pitfall example.
- In the uniqueness proof, identify the equality that allows the final conclusion \(\mathbf{G}=\mathbf{A}^{+}\).
