Why Four Equations Are Needed
For a square invertible matrix, the inverse is characterized by multiplying to the identity on either side. A rectangular or rank-deficient matrix cannot generally have an inverse of that kind, but its singular value decomposition (SVD) gives a natural candidate for a pseudoinverse. The four Moore-Penrose equations specify the properties this candidate must satisfy. Two equations express inverse-like behavior; the other two require symmetry of the products in opposite orders.
Throughout this tutorial, let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). We use the real SVD, so transposes—not conjugate transposes—appear. Write the positive singular values of \(\mathbf{A}\) as \(s_1,\dots,s_r\), where each \(s_i>0\). The corresponding left and right singular vectors are the columns of \(\mathbf{U}_r\in\mathbb{R}^{m\times r}\) and \(\mathbf{V}_r\in\mathbb{R}^{n\times r}\). These columns are orthonormal, so \(\mathbf{U}_r^{\top}\mathbf{U}_r=\mathbf{I}_r\) and \(\mathbf{V}_r^{\top}\mathbf{V}_r=\mathbf{I}_r\).
The dimensions explain why the equations are arranged as they are. The product \(\mathbf{A}\mathbf{G}\) is \(m\times m\), so it can be compared with \(\mathbf{A}\) in \(\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{A}\). The product \(\mathbf{G}\mathbf{A}\) is \(n\times n\), so it can be compared with \(\mathbf{G}\) in \(\mathbf{G}\mathbf{A}\mathbf{G}=\mathbf{G}\). Both symmetry conditions are meaningful even when \(m\ne n\), since each product is square.
The SVD Candidate
In the compact SVD, \(\mathbf{A}=\mathbf{U}_r\mathbf{D}\mathbf{V}_r^{\top}\), where \(\mathbf{D}=\operatorname{diag}(s_1,\dots,s_r)\). Since all \(s_i\) are positive, \(\mathbf{D}^{-1}=\operatorname{diag}(1/s_1,\dots,1/s_r)\) is defined. The SVD-defined candidate reverses the roles of the left and right singular vectors and reciprocates each positive singular value.
The formula has an intuitive structure: \(\mathbf{U}_r^{\top}\) takes coordinates along the left singular vectors, \(\mathbf{D}^{-1}\) rescales those coordinates, and \(\mathbf{V}_r\) returns the result in the domain coordinates. Here the immediate task is not to interpret solutions of linear systems, but to check exactly what the formula guarantees.
Proof. First suppose \(r>0\). Using \(\mathbf{U}_r^{\top}\mathbf{U}_r=\mathbf{I}_r\), \(\mathbf{V}_r^{\top}\mathbf{V}_r=\mathbf{I}_r\), and \(\mathbf{D}\mathbf{D}^{-1}=\mathbf{D}^{-1}\mathbf{D}=\mathbf{I}_r\), compute $$ \begin{aligned} \mathbf{A}\mathbf{G}\mathbf{A} &=\mathbf{U}_r\mathbf{D}\mathbf{V}_r^{\top} \mathbf{V}_r\mathbf{D}^{-1}\mathbf{U}_r^{\top} \mathbf{U}_r\mathbf{D}\mathbf{V}_r^{\top}\\ &=\mathbf{U}_r\mathbf{D}\mathbf{D}^{-1}\mathbf{D}\mathbf{V}_r^{\top} =\mathbf{U}_r\mathbf{D}\mathbf{V}_r^{\top} =\mathbf{A}. \end{aligned} $$ For the second equation, $$ \begin{aligned} \mathbf{G}\mathbf{A}\mathbf{G} &=\mathbf{V}_r\mathbf{D}^{-1}\mathbf{U}_r^{\top} \mathbf{U}_r\mathbf{D}\mathbf{V}_r^{\top} \mathbf{V}_r\mathbf{D}^{-1}\mathbf{U}_r^{\top}\\ &=\mathbf{V}_r\mathbf{D}^{-1}\mathbf{D}\mathbf{D}^{-1}\mathbf{U}_r^{\top} =\mathbf{V}_r\mathbf{D}^{-1}\mathbf{U}_r^{\top} =\mathbf{G}. \end{aligned} $$ The two products in the symmetry conditions reduce to $$ \mathbf{A}\mathbf{G} =\mathbf{U}_r\mathbf{D}\mathbf{D}^{-1}\mathbf{U}_r^{\top} =\mathbf{U}_r\mathbf{U}_r^{\top}, \qquad \mathbf{G}\mathbf{A} =\mathbf{V}_r\mathbf{D}^{-1}\mathbf{D}\mathbf{V}_r^{\top} =\mathbf{V}_r\mathbf{V}_r^{\top}. $$ Taking transposes gives \((\mathbf{U}_r\mathbf{U}_r^{\top})^{\top}=\mathbf{U}_r\mathbf{U}_r^{\top}\) and \((\mathbf{V}_r\mathbf{V}_r^{\top})^{\top}=\mathbf{V}_r\mathbf{V}_r^{\top}\), so both products are symmetric. If \(r=0\), then \(\mathbf{A}=\mathbf{O}\) and \(\mathbf{G}=\mathbf{O}\); all four equations reduce to equalities between zero matrices. Thus the result holds in every case. \(\square\)
This proof also shows the useful formulas \(\mathbf{A}\mathbf{G}=\mathbf{U}_r\mathbf{U}_r^{\top}\) and \(\mathbf{G}\mathbf{A}=\mathbf{V}_r\mathbf{V}_r^{\top}\). Their symmetry is not an extra accident of a particular example: it follows directly from the orthonormal columns in the SVD. The two products need not be the same matrix, since one acts on \(\mathbb{R}^m\) and the other on \(\mathbb{R}^n\).
Worked Checks in Different Matrix Shapes
Worked Example: A Rank-One Rectangular Matrix
Let $$ \mathbf{A}=\begin{bmatrix}3&0&0\\0&0&0\end{bmatrix}, \qquad \mathbf{G}=\begin{bmatrix}\tfrac13&0\\0&0\\0&0\end{bmatrix}. $$ The only positive singular value is \(3\), and these matrices give the SVD candidate. Verify all four equations.
First multiply in the order \(\mathbf{A}\mathbf{G}\): $$ \mathbf{A}\mathbf{G} =\begin{bmatrix}3&0&0\\0&0&0\end{bmatrix} \begin{bmatrix}\tfrac13&0\\0&0\\0&0\end{bmatrix} =\begin{bmatrix}1&0\\0&0\end{bmatrix}. $$ Therefore $$ \mathbf{A}\mathbf{G}\mathbf{A} =\begin{bmatrix}1&0\\0&0\end{bmatrix} \begin{bmatrix}3&0&0\\0&0&0\end{bmatrix} =\begin{bmatrix}3&0&0\\0&0&0\end{bmatrix} =\mathbf{A}. $$ In the reverse order, $$ \mathbf{G}\mathbf{A} =\begin{bmatrix}\tfrac13&0\\0&0\\0&0\end{bmatrix} \begin{bmatrix}3&0&0\\0&0&0\end{bmatrix} =\begin{bmatrix}1&0&0\\0&0&0\\0&0&0\end{bmatrix}, $$ so $$ \mathbf{G}\mathbf{A}\mathbf{G} =\begin{bmatrix}1&0&0\\0&0&0\\0&0&0\end{bmatrix} \begin{bmatrix}\tfrac13&0\\0&0\\0&0\end{bmatrix} =\begin{bmatrix}\tfrac13&0\\0&0\\0&0\end{bmatrix} =\mathbf{G}. $$ Both \(\mathbf{A}\mathbf{G}\) and \(\mathbf{G}\mathbf{A}\) are diagonal, hence equal to their transposes. This verifies the two symmetry equations as well.
Worked Example: A Nonzero Rank-One Matrix in Two Dimensions
Consider $$ \mathbf{A}=\begin{bmatrix}1&1\\1&1\end{bmatrix}. $$ Its positive singular value is \(2\), with left and right singular vector \(\frac{1}{\sqrt{2}}(1,1)^{\top}\). The SVD candidate is therefore $$ \mathbf{G}=\frac12 \begin{bmatrix}\tfrac{1}{\sqrt2}\\[2pt]\tfrac{1}{\sqrt2}\end{bmatrix} \begin{bmatrix}\tfrac{1}{\sqrt2}&\tfrac{1}{\sqrt2}\end{bmatrix} =\begin{bmatrix}\tfrac14&\tfrac14\\\tfrac14&\tfrac14\end{bmatrix} =\tfrac14\mathbf{A}. $$ Check the equations by first noting that $$ \mathbf{A}^2= \begin{bmatrix}1&1\\1&1\end{bmatrix} \begin{bmatrix}1&1\\1&1\end{bmatrix} =\begin{bmatrix}2&2\\2&2\end{bmatrix}=2\mathbf{A}. $$ It follows, by one more multiplication, that \(\mathbf{A}^3=2\mathbf{A}^2=4\mathbf{A}\). Since \(\mathbf{G}=\tfrac14\mathbf{A}\), $$ \mathbf{A}\mathbf{G}\mathbf{A} =\tfrac14\mathbf{A}^3 =\mathbf{A}, \qquad \mathbf{G}\mathbf{A}\mathbf{G} =\tfrac{1}{16}\mathbf{A}^3 =\tfrac14\mathbf{A} =\mathbf{G}. $$ The other two products are $$ \mathbf{A}\mathbf{G}=\tfrac14\mathbf{A}^2 =\begin{bmatrix}\tfrac12&\tfrac12\\\tfrac12&\tfrac12\end{bmatrix}, \qquad \mathbf{G}\mathbf{A}=\tfrac14\mathbf{A}^2 =\begin{bmatrix}\tfrac12&\tfrac12\\\tfrac12&\tfrac12\end{bmatrix}. $$ Each is symmetric, so both transpose equations hold. The equality of these two products here comes from \(\mathbf{A}\) and \(\mathbf{G}\) being scalar multiples of the same matrix; it is not required in the general case.
Worked Example: A Full-Column-Rank Rectangular Matrix
Let $$ \mathbf{A}=\begin{bmatrix}1&0\\0&2\\0&0\end{bmatrix}, \qquad \mathbf{G}=\begin{bmatrix}1&0&0\\0&\tfrac12&0\end{bmatrix}. $$ The singular values are \(1\) and \(2\); the displayed \(\mathbf{G}\) is the SVD candidate. The products are $$ \mathbf{A}\mathbf{G} =\begin{bmatrix}1&0&0\\0&1&0\\0&0&0\end{bmatrix}, \qquad \mathbf{G}\mathbf{A} =\begin{bmatrix}1&0\\0&1\end{bmatrix}. $$ Consequently, $$ \mathbf{A}\mathbf{G}\mathbf{A} =\begin{bmatrix}1&0&0\\0&1&0\\0&0&0\end{bmatrix} \begin{bmatrix}1&0\\0&2\\0&0\end{bmatrix} =\begin{bmatrix}1&0\\0&2\\0&0\end{bmatrix} =\mathbf{A}, $$ and $$ \mathbf{G}\mathbf{A}\mathbf{G} =\begin{bmatrix}1&0\\0&1\end{bmatrix} \begin{bmatrix}1&0&0\\0&\tfrac12&0\end{bmatrix} =\begin{bmatrix}1&0&0\\0&\tfrac12&0\end{bmatrix} =\mathbf{G}. $$ The displayed \(\mathbf{A}\mathbf{G}\) and \(\mathbf{G}\mathbf{A}\) are both diagonal, so each equals its transpose. All four equations are verified, including in this case where the two products have different sizes.
Why the Symmetry Conditions Cannot Be Dropped
The first equation alone does not specify the SVD candidate. It permits many matrices \(\mathbf{G}\), even when \(\mathbf{A}\) is fixed. For example, take $$ \mathbf{A}=\begin{bmatrix}1&0\\0&0\end{bmatrix}, \qquad \mathbf{G}=\begin{bmatrix}1&1\\0&0\end{bmatrix}. $$ Direct multiplication gives $$ \mathbf{A}\mathbf{G} =\begin{bmatrix}1&1\\0&0\end{bmatrix}, \qquad \mathbf{A}\mathbf{G}\mathbf{A} =\begin{bmatrix}1&1\\0&0\end{bmatrix} \begin{bmatrix}1&0\\0&0\end{bmatrix} =\begin{bmatrix}1&0\\0&0\end{bmatrix} =\mathbf{A}. $$ Thus the first equation holds. But $$ (\mathbf{A}\mathbf{G})^{\top} =\begin{bmatrix}1&0\\1&0\end{bmatrix} \ne \begin{bmatrix}1&1\\0&0\end{bmatrix} =\mathbf{A}\mathbf{G}. $$ So it fails the third equation and is not a solution of all four. This is the common mistake: checking \(\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{A}\) and declaring the proposed matrix the pseudoinverse. The correction is to test each equation separately; an inverse-like identity does not imply either symmetry condition.
The two symmetry equations are also distinct conditions in general. One concerns a matrix acting on the codomain, and the other concerns a matrix acting on the domain. Their dimensions may differ, and neither transpose equality should be inferred solely from the other. The SVD verification is effective precisely because it computes both products explicitly.
A Reliable Verification Checklist
For a proposed candidate \(\mathbf{G}\in\mathbb{R}^{n\times m}\), a careful check can be organized as follows:
- Confirm the dimensions: \(\mathbf{A}\mathbf{G}\) must be \(m\times m\), and \(\mathbf{G}\mathbf{A}\) must be \(n\times n\).
- Multiply \(\mathbf{A}\mathbf{G}\mathbf{A}\) and compare every entry with \(\mathbf{A}\).
- Multiply \(\mathbf{G}\mathbf{A}\mathbf{G}\) and compare every entry with \(\mathbf{G}\).
- Transpose \(\mathbf{A}\mathbf{G}\) and check that it equals \(\mathbf{A}\mathbf{G}\).
- Transpose \(\mathbf{G}\mathbf{A}\) and check that it equals \(\mathbf{G}\mathbf{A}\).
For an SVD-defined candidate, the theorem supplies a general verification without having to expand every entry: orthonormality reduces the inverse-like products to identities in the \(r\)-dimensional singular-vector coordinates, and the remaining products have the forms \(\mathbf{U}_r\mathbf{U}_r^{\top}\) and \(\mathbf{V}_r\mathbf{V}_r^{\top}\), which are symmetric by transposition. The examples show how the same checks appear in direct arithmetic. The four equations are the defining conditions under study here; establishing that a matrix satisfying them is unique is a separate result.
Check Your Understanding
For each question, track the dimensions as well as the matrix entries.
- If \(\mathbf{A}\) is \(4\times 2\), what are the dimensions of a proposed \(\mathbf{G}\), \(\mathbf{A}\mathbf{G}\), and \(\mathbf{G}\mathbf{A}\)?
- In the compact SVD formula, why is \(\mathbf{D}^{-1}\) defined for the retained singular values?
- For the SVD candidate, write the simplified expressions for \(\mathbf{A}\mathbf{G}\) and \(\mathbf{G}\mathbf{A}\), then verify their symmetry by transposing them.
- For \(\mathbf{A}=\begin{bmatrix}2&0\\0&0\end{bmatrix}\), compute the SVD candidate and check all four equations.
- Give a reason that checking only \(\mathbf{A}\mathbf{G}\mathbf{A}=\mathbf{A}\) is insufficient, and identify which equation fails in the concrete pitfall above.
