Two Formulas, Two Different Rank Conditions
For a rectangular matrix, the Moore–Penrose pseudoinverse provides a single matrix that supports both least-squares and exact-solution questions. When the matrix has full rank in one of two particular senses, its pseudoinverse can be written using an ordinary inverse of a square matrix. The key is to choose the square matrix that is guaranteed to be invertible: \(\mathbf{A}^{\top}\mathbf{A}\) for full column rank, or \(\mathbf{A}\mathbf{A}^{\top}\) for full row rank.
Throughout, let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). Its columns are vectors in \(\mathbb{R}^m\). It has full column rank if its \(n\) columns are linearly independent, so \(n\le m\). It has full row rank if its \(m\) rows are linearly independent, so \(m\le n\). These are distinct conditions for rectangular matrices; a wide matrix cannot have full column rank, and a tall matrix cannot have full row rank.
The formulas below follow by proposing a candidate and checking all four equations. In particular, the symmetry conditions matter: merely finding a matrix that reverses multiplication on one side is not enough to establish that it is the pseudoinverse.
Full Column Rank: The Left-Inverse Formula
Suppose \(\mathbf{A}\in\mathbb{R}^{m\times n}\) has full column rank. Then the square matrix \(\mathbf{A}^{\top}\mathbf{A}\in\mathbb{R}^{n\times n}\) is invertible. Indeed, for any nonzero \(\mathbf{x}\in\mathbb{R}^n\), full column rank gives \(\mathbf{A}\mathbf{x}\ne\mathbf{0}\), and therefore \(\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}=\|\mathbf{A}\mathbf{x}\|^2>0\). If \(\mathbf{A}^{\top}\mathbf{A}\) had a nonzero vector in its null space, this quadratic expression would be zero for that vector. Thus its null space is trivial, so the square matrix is invertible.
Proof. Put \(\mathbf{B}=(\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}\). The inverse exists by the argument above, and \(\mathbf{B}\) has size \(n\times m\). First, \(\mathbf{B}\mathbf{A}=(\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}\mathbf{A}=\mathbf{I}_n\). Consequently, \(\mathbf{A}\mathbf{B}\mathbf{A}=\mathbf{A}\), and \(\mathbf{B}\mathbf{A}\mathbf{B}=\mathbf{I}_n\mathbf{B}=\mathbf{B}\). The matrix \(\mathbf{A}^{\top}\mathbf{A}\) is symmetric, so its inverse is symmetric. Hence \(\mathbf{B}\mathbf{A}=\mathbf{I}_n\) is symmetric, and \(\mathbf{A}\mathbf{B}=\mathbf{A}(\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}\) is symmetric as well, since transposing this expression leaves it unchanged. Thus \(\mathbf{B}\) satisfies all four equations, and by uniqueness it equals \(\mathbf{A}^{+}\). \(\square\)
Worked Example: A Tall Full-Column-Rank Matrix
Let \(\mathbf{A}=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}\). Its columns are independent: if \(c_1(1,0,1)+c_2(0,1,1)=\mathbf{0}\), the first two coordinates force \(c_1=c_2=0\). Compute its pseudoinverse.
First form the Gram matrix and its inverse:
Multiplying by \(\mathbf{A}^{\top}\) gives
Check. Multiplication on the right by \(\mathbf{A}\) gives \(\mathbf{A}^{+}\mathbf{A}=\mathbf{I}_2\). In the other order,
This product is symmetric, as required by the Moore–Penrose equations. It also fixes every vector in \(\operatorname{Col}\mathbf{A}\): for \(\mathbf{A}\mathbf{x}\), multiplication gives \(\mathbf{A}\mathbf{A}^{+}\mathbf{A}\mathbf{x}=\mathbf{A}\mathbf{x}\).
Full Row Rank: The Right-Inverse Formula
Now suppose \(\mathbf{A}\in\mathbb{R}^{m\times n}\) has full row rank. In this case \(\mathbf{A}\mathbf{A}^{\top}\), rather than \(\mathbf{A}^{\top}\mathbf{A}\), is guaranteed to be invertible. For nonzero \(\mathbf{y}\in\mathbb{R}^m\), independence of the rows means \(\mathbf{A}^{\top}\mathbf{y}\ne\mathbf{0}\). Thus \(\mathbf{y}^{\top}\mathbf{A}\mathbf{A}^{\top}\mathbf{y}=\|\mathbf{A}^{\top}\mathbf{y}\|^2>0\), which proves that the square matrix \(\mathbf{A}\mathbf{A}^{\top}\) has trivial null space and is invertible.
Proof. Put \(\mathbf{B}=\mathbf{A}^{\top}(\mathbf{A}\mathbf{A}^{\top})^{-1}\), which has size \(n\times m\). The inverse exists as shown above. Direct multiplication gives \(\mathbf{A}\mathbf{B}=\mathbf{A}\mathbf{A}^{\top}(\mathbf{A}\mathbf{A}^{\top})^{-1}=\mathbf{I}_m\). It follows that \(\mathbf{A}\mathbf{B}\mathbf{A}=\mathbf{A}\) and \(\mathbf{B}\mathbf{A}\mathbf{B}=\mathbf{B}\mathbf{I}_m=\mathbf{B}\). The product \(\mathbf{A}\mathbf{B}=\mathbf{I}_m\) is symmetric. Also, \(\mathbf{A}\mathbf{A}^{\top}\) and its inverse are symmetric, so \(\mathbf{B}\mathbf{A}=\mathbf{A}^{\top}(\mathbf{A}\mathbf{A}^{\top})^{-1}\mathbf{A}\) is symmetric. All four equations hold, and uniqueness gives \(\mathbf{B}=\mathbf{A}^{+}\). \(\square\)
Worked Example: A Wide Full-Row-Rank Matrix
Let \(\mathbf{A}=\begin{bmatrix}1&0&1\\0&1&1\end{bmatrix}\). Its two rows are independent, since a linear combination of them equal to zero has both coefficients zero. Compute \(\mathbf{A}^{+}\) from the full-row-rank formula.
Compute the \(2\times2\) Gram matrix and its inverse:
Then
Check. Direct multiplication gives \(\mathbf{A}\mathbf{A}^{+}=\mathbf{I}_2\). The other product is
which is symmetric. The four Moore–Penrose equations therefore hold, also as guaranteed by the theorem.
What the Formulas Do to a Right-Hand Side
The distinction between the two cases is visible in the products. For full column rank, \(\mathbf{A}^{+}\mathbf{A}=\mathbf{I}_n\), while \(\mathbf{A}\mathbf{A}^{+}\) is the orthogonal projection onto \(\operatorname{Col}\mathbf{A}\). For full row rank, \(\mathbf{A}\mathbf{A}^{+}=\mathbf{I}_m\), while \(\mathbf{A}^{+}\mathbf{A}\) is the orthogonal projection onto the row space of \(\mathbf{A}\), viewed as a subspace of \(\mathbb{R}^n\). These projection statements follow because the products are symmetric and idempotent, and their images are respectively \(\operatorname{Col}\mathbf{A}\) and the image of \(\mathbf{A}^{\top}\).
For the tall matrix in the first example, take \(\mathbf{b}=(1,2,4)^{\top}\). Then
The latter vector is the projection of \(\mathbf{b}\) onto \(\operatorname{Col}\mathbf{A}\). The residual is \(\mathbf{b}-\mathbf{A}\mathbf{A}^{+}\mathbf{b}=(-1/3,-1/3,1/3)^{\top}\), and its dot products with the columns \((1,0,1)^{\top}\) and \((0,1,1)^{\top}\) are both zero. Thus \(\mathbf{A}^{+}\mathbf{b}\) is the least-squares solution. It is the unique such solution because full column rank makes \(\mathbf{A}\mathbf{x}\) different for distinct \(\mathbf{x}\).
For the wide matrix in the second example, take \(\mathbf{b}=(2,1)^{\top}\). The formula gives \(\mathbf{A}^{+}\mathbf{b}=(1,0,1)^{\top}\), and multiplying back gives \(\mathbf{A}\mathbf{A}^{+}\mathbf{b}=(2,1)^{\top}=\mathbf{b}\). It is an exact solution. Every solution has the form \(\mathbf{x}=(2-t,1-t,t)^{\top}\), because the equations are \(x_1+x_3=2\) and \(x_2+x_3=1\). Relative to \((1,0,1)^{\top}\), the difference is \(s(-1,-1,1)^{\top}\) for some \(s\in\mathbb{R}\). These two vectors are orthogonal, so the squared norm of any solution is \(2+3s^2\). It is minimized uniquely at \(s=0\). Thus the full-row-rank formula gives the minimum-norm exact solution.
When Both Formulas Apply—and a Common Error
If \(\mathbf{A}\) is square and invertible, it has both full column rank and full row rank. Both formulas then give the ordinary inverse. For example, let \(\mathbf{A}=\begin{bmatrix}1&2\\3&5\end{bmatrix}\). Computing from the first formula gives
Computing from the second formula gives
Check: multiplying the resulting matrix by \(\mathbf{A}\) on either side gives \(\mathbf{I}_2\), confirming that it is \(\mathbf{A}^{-1}\).
Common mistake: using the wrong Gram matrix. For the full-row-rank \(2\times3\) matrix \(\begin{bmatrix}1&0&1\\0&1&1\end{bmatrix}\), a tempting but invalid step is to try \((\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}\). In fact, \(\mathbf{A}(-1,-1,1)^{\top}=\mathbf{0}\), so \((\mathbf{A}^{\top}\mathbf{A})(-1,-1,1)^{\top}=\mathbf{0}\). The \(3\times3\) matrix \(\mathbf{A}^{\top}\mathbf{A}\) is singular and cannot be inverted. The correction is to use the full-row-rank formula, which inverts the \(2\times2\) matrix \(\mathbf{A}\mathbf{A}^{\top}\). Choose the Gram matrix to match the rank condition; do not assume both are invertible.
Check Your Understanding
For each question, identify the dimensions and rank condition before choosing a formula.
- For a full-column-rank matrix \(\mathbf{A}\in\mathbb{R}^{5\times3}\), what are the dimensions of \(\mathbf{A}^{\top}\mathbf{A}\) and \(\mathbf{A}^{+}\)?
- Why does full row rank guarantee that \(\mathbf{A}\mathbf{A}^{\top}\) is invertible?
- For a full-column-rank matrix, which product is the identity, and onto which subspace does the other product project?
- For a full-row-rank matrix, what solution does \(\mathbf{A}^{+}\mathbf{b}\) give for a consistent equation \(\mathbf{A}\mathbf{x}=\mathbf{b}\)?
- Give a reason the full-column-rank formula cannot be used for every wide matrix.
