Why Invertible Changes Preserve Rank
Rank measures how many independent directions a matrix produces. Multiplying a matrix by another matrix changes its entries, and can change its column or row spaces. But if the multiplier is invertible, the change cannot collapse distinct directions into one: the original information can be recovered by multiplying by the inverse. This tutorial proves that rank is preserved under invertible multiplication on either side, and distinguishes what happens to the column and row spaces in each case.
Throughout, matrices have real entries. Let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). Its rank, \(\operatorname{rank}\mathbf{A}\), is the dimension of its column space \(\operatorname{Col}\mathbf{A}\), the subspace of \(\mathbb{R}^m\) spanned by the columns of \(\mathbf{A}\). A square matrix is invertible if it has a two-sided inverse. To multiply \(\mathbf{A}\) on the left by an invertible matrix requires a matrix \(\mathbf{P}\in\mathbb{R}^{m\times m}\); to multiply on the right requires a matrix \(\mathbf{Q}\in\mathbb{R}^{n\times n}\). These sizes ensure that the products are defined.
Proof. First consider left multiplication. Each column of \(\mathbf{P}\mathbf{A}\) is \(\mathbf{P}\) times the corresponding column of \(\mathbf{A}\). Thus \(\operatorname{Col}(\mathbf{P}\mathbf{A})=\{\mathbf{P}\mathbf{x}:\mathbf{x}\in\operatorname{Col}\mathbf{A}\}\). Let \(\mathbf{x}_1,\ldots,\mathbf{x}_r\) be a basis of \(\operatorname{Col}\mathbf{A}\), where \(r=\operatorname{rank}\mathbf{A}\). The vectors \(\mathbf{P}\mathbf{x}_1,\ldots,\mathbf{P}\mathbf{x}_r\) span \(\operatorname{Col}(\mathbf{P}\mathbf{A})\), because applying \(\mathbf{P}\) to any linear combination of the basis vectors gives the corresponding linear combination of their images. They are also independent: if \(\sum_{i=1}^r c_i\mathbf{P}\mathbf{x}_i=\mathbf{0}\), then \(\mathbf{P}(\sum_{i=1}^r c_i\mathbf{x}_i)=\mathbf{0}\). Since \(\mathbf{P}\) is invertible, it is one-to-one, so \(\sum_{i=1}^r c_i\mathbf{x}_i=\mathbf{0}\). Independence of the original basis gives \(c_i=0\) for every \(i\). Therefore the image vectors are a basis of \(\operatorname{Col}(\mathbf{P}\mathbf{A})\), and \(\operatorname{rank}(\mathbf{P}\mathbf{A})=r\). For right multiplication, every column of \(\mathbf{A}\mathbf{Q}\) is a linear combination of the columns of \(\mathbf{A}\), so \(\operatorname{Col}(\mathbf{A}\mathbf{Q})\subseteq\operatorname{Col}\mathbf{A}\). Since \(\mathbf{Q}\) is invertible, \(\mathbf{A}=(\mathbf{A}\mathbf{Q})\mathbf{Q}^{-1}\). Applying the same column-combination observation to this equality gives \(\operatorname{Col}\mathbf{A}\subseteq\operatorname{Col}(\mathbf{A}\mathbf{Q})\). The column spaces are equal, so their dimensions, and hence their ranks, are equal. \(\square\)
Left Multiplication: Invertible Row Changes
Left multiplication by \(\mathbf{P}\) forms each row of \(\mathbf{P}\mathbf{A}\) as a linear combination of the rows of \(\mathbf{A}\). When \(\mathbf{P}\) is invertible, the original matrix can be recovered as \(\mathbf{A}=\mathbf{P}^{-1}(\mathbf{P}\mathbf{A})\). Thus this is a reversible change of rows. In particular, elementary row operations are reversible and can be represented by multiplication on the left by invertible elementary matrices. The theorem explains why such row changes preserve rank.
Worked Example: An Invertible Row Change
Let \(\mathbf{A}=\begin{bmatrix}1&2&0\\0&1&1\end{bmatrix}\) and \(\mathbf{P}=\begin{bmatrix}1&1\\0&1\end{bmatrix}\). The matrix \(\mathbf{P}\) is invertible because \(\mathbf{P}^{-1}=\begin{bmatrix}1&-1\\0&1\end{bmatrix}\): multiplying in either order gives \(\mathbf{I}_2\).
Compute the left product by taking the first row of \(\mathbf{P}\mathbf{A}\) as the sum of the two rows of \(\mathbf{A}\), and keeping the second row unchanged:
To check the rank, reduce each matrix. For \(\mathbf{A}\), replace \(R_1\) by \(R_1-2R_2\). For \(\mathbf{P}\mathbf{A}\), replace \(R_1\) by \(R_1-3R_2\):
Each reduced matrix has two pivots, so both ranks are \(2\). The multiplication changed the rows and the entries, but the two independent directions were not lost.
Right Multiplication: Invertible Column Changes
Right multiplication by \(\mathbf{Q}\) forms each column of \(\mathbf{A}\mathbf{Q}\) as a linear combination of the columns of \(\mathbf{A}\). Invertibility makes this column change reversible: multiplying \(\mathbf{A}\mathbf{Q}\) by \(\mathbf{Q}^{-1}\) recovers \(\mathbf{A}\). The proof above gives a stronger statement than equal rank in this case: the column spaces themselves are exactly equal.
Worked Example: An Invertible Column Change
Use \(\mathbf{A}=\begin{bmatrix}1&2&0\\0&1&1\end{bmatrix}\) and let \(\mathbf{Q}=\begin{bmatrix}1&1&0\\0&1&1\\0&0&1\end{bmatrix}\). The matrix \(\mathbf{Q}\) is invertible, with \(\mathbf{Q}^{-1}=\begin{bmatrix}1&-1&1\\0&1&-1\\0&0&1\end{bmatrix}\). Indeed, multiplying these two matrices in either order gives \(\mathbf{I}_3\).
The columns of \(\mathbf{A}\mathbf{Q}\) are, in order, the first column of \(\mathbf{A}\), the sum of its first two columns, and the sum of its second and third columns. Direct multiplication gives
Row reduction confirms the ranks. Subtract \(2R_2\) from \(R_1\) for \(\mathbf{A}\), and \(3R_2\) from \(R_1\) for \(\mathbf{A}\mathbf{Q}\):
Both have two pivot columns and rank \(2\). The column spaces are equal as well: the columns of the product are combinations of the original columns, and multiplication by \(\mathbf{Q}^{-1}\) expresses the original columns as combinations of the new ones.
What Happens to the Row and Column Spaces?
The rank result can be read alongside the effects on both fundamental spaces. Recall that \(\operatorname{Row}\mathbf{A}\) is the span of the rows of \(\mathbf{A}\), regarded as row vectors. Left multiplication and right multiplication have different effects on these spaces.
Proof. The statements about column spaces follow from the column-combination argument in the rank theorem: right multiplication gives both inclusions because \(\mathbf{Q}\) has an inverse, while left multiplication applies \(\mathbf{P}\) to every column-space vector. For the row spaces, each row of \(\mathbf{P}\mathbf{A}\) is a linear combination of rows of \(\mathbf{A}\), so \(\operatorname{Row}(\mathbf{P}\mathbf{A})\subseteq\operatorname{Row}\mathbf{A}\). The identity \(\mathbf{A}=\mathbf{P}^{-1}(\mathbf{P}\mathbf{A})\) gives the reverse inclusion. Each row of \(\mathbf{A}\mathbf{Q}\) is a row of \(\mathbf{A}\) multiplied by \(\mathbf{Q}\); linear combinations of those rows therefore have the form \(\mathbf{x}\mathbf{Q}\) with \(\mathbf{x}\in\operatorname{Row}\mathbf{A}\), and every such \(\mathbf{x}\) is a combination of rows of \(\mathbf{A}\). This proves the final equality. \(\square\)
In short, an invertible left multiplication preserves the row space exactly and carries the column space to an invertible image of itself. An invertible right multiplication preserves the column space exactly and carries the row space to an invertible image of itself. The spaces need not remain the same in the latter two transformations, but their dimensions are preserved because the transformations are one-to-one and onto their images. This is another way to understand the rank invariance theorem.
Why the Invertibility Condition Matters
Worked Example: A Singular Multiplier Can Lower Rank
Let \(\mathbf{A}=\mathbf{I}_2\), which has rank \(2\), and take the noninvertible matrix \(\mathbf{S}=\begin{bmatrix}1&0\\0&0\end{bmatrix}\). Since \(\mathbf{S}\mathbf{e}_2=\mathbf{0}\), the matrix \(\mathbf{S}\) is not one-to-one and cannot be invertible. The product is
The first matrix has two pivot columns, while the product has only one. Thus \(\operatorname{rank}(\mathbf{A})=2\) but \(\operatorname{rank}(\mathbf{S}\mathbf{A})=1\). The singular multiplier has collapsed one direction to zero. This does not contradict the theorem: its hypothesis that the multiplier is invertible is not satisfied.
Common mistake: “Multiplication by a square matrix preserves rank.” This claim omits the essential condition. The calculation above is a counterexample: \(\mathbf{S}\) is square, but singular, and \(\mathbf{S}\mathbf{I}_2\) has lower rank than \(\mathbf{I}_2\). The correction is to check both compatibility of dimensions and invertibility of the multiplier before applying rank invariance.
Also avoid reversing the theorem without checking what is being claimed. It says that multiplication by an invertible matrix preserves rank; it does not say that every matrix which happens to preserve the rank of a particular \(\mathbf{A}\) must be invertible. For instance, multiplying the zero matrix by any compatible matrix still gives the zero matrix and rank \(0\). The theorem provides a sufficient condition for rank preservation, not a converse for every individual matrix.
Using the Result
Rank invariance is useful whenever a matrix is replaced by a more convenient representation through reversible row or column changes. It permits rank comparisons without requiring the new matrix to have the same entries, row space, or column space as the original. For a product \(\mathbf{P}\mathbf{A}\mathbf{Q}\) with both multipliers invertible, apply the theorem first to the left product and then to the right product; the rank remains \(\operatorname{rank}\mathbf{A}\) at each step.
The distinction between the two sides is worth retaining. Invertible right multiplication leaves the column space exactly unchanged, while invertible left multiplication generally transforms it. Invertible left multiplication leaves the row space exactly unchanged, while invertible right multiplication generally transforms it. In every case, the transformed space has the same dimension as before, which is the quantity rank records.
Check Your Understanding
For each question, identify the relevant space or condition before making a rank claim.
- If \(\mathbf{A}\in\mathbb{R}^{4\times 3}\), what size must an invertible matrix \(\mathbf{P}\) have for \(\mathbf{P}\mathbf{A}\) to be defined? What size must an invertible \(\mathbf{Q}\) have for \(\mathbf{A}\mathbf{Q}\) to be defined?
- Explain why \(\operatorname{Col}(\mathbf{A}\mathbf{Q})\subseteq\operatorname{Col}\mathbf{A}\) for any compatible \(\mathbf{Q}\), and identify where invertibility is needed to obtain equality.
- Suppose \(\mathbf{P}\) is invertible. Which space is preserved exactly by left multiplication, \(\operatorname{Row}(\mathbf{P}\mathbf{A})\) or \(\operatorname{Col}(\mathbf{P}\mathbf{A})\)?
- Give a noninvertible \(2\times2\) matrix that lowers the rank of \(\mathbf{I}_2\), and calculate the product and its rank.
- State the rank of \(\mathbf{P}\mathbf{A}\mathbf{Q}\) when both compatible multipliers are invertible, and give the theorem that justifies the conclusion.
