Four Subspaces in Two Ambient Spaces
An \(m\times n\) matrix relates vectors in two different spaces: multiplication by \(\mathbf{A}\) takes an input from \(\mathbb{R}^n\) to an output in \(\mathbb{R}^m\). Its row space and null space therefore live in \(\mathbb{R}^n\), while its column space and left null space live in \(\mathbb{R}^m\). Keeping these ambient spaces straight reveals a useful structure: in each space, one fundamental subspace is exactly the orthogonal complement of another.
Throughout this tutorial, \(\mathbf{A}\in\mathbb{R}^{m\times n}\), and orthogonality uses the usual dot product on the relevant real coordinate space. The row space \(\operatorname{Row}\mathbf{A}\) is the span in \(\mathbb{R}^n\) of the rows of \(\mathbf{A}\), treated as vectors. The column space \(\operatorname{Col}\mathbf{A}\) is the span in \(\mathbb{R}^m\) of its columns. The null space \(\operatorname{Nul}\mathbf{A}\) consists of the vectors \(\mathbf{x}\in\mathbb{R}^n\) for which \(\mathbf{A}\mathbf{x}=\mathbf{0}\). The left null space is \(\operatorname{Nul}(\mathbf{A}^{\top})\), a subspace of \(\mathbb{R}^m\).
There are two orthogonal-complement pairs. In \(\mathbb{R}^n\), the row space pairs with the null space. In \(\mathbb{R}^m\), the column space pairs with the left null space. These are equalities of subspaces, not just statements that vectors from the paired spaces happen to be orthogonal.
The Orthogonal-Complement Identities
We use the double orthogonal complement property for finite-dimensional real coordinate spaces: if \(S\subseteq\mathbb{R}^q\) is a subspace, then \((S^\perp)^\perp=S\). This follows by taking an orthonormal basis \(\mathbf{u}_1,\dots,\mathbf{u}_k\) of \(S\) and extending it to an orthonormal basis \(\mathbf{u}_1,\dots,\mathbf{u}_q\) of \(\mathbb{R}^q\), as is possible by Gram-Schmidt and orthonormal basis extension. A vector is orthogonal to \(S\) precisely when its first \(k\) coordinates in this basis are zero, so \(S^\perp=\operatorname{span}\{\mathbf{u}_{k+1},\dots,\mathbf{u}_q\}\). Its orthogonal complement is therefore \(\operatorname{span}\{\mathbf{u}_1,\dots,\mathbf{u}_k\}=S\). This argument also gives \(\dim S^\perp=q-\dim S\), including \(k=0\) and \(k=q\).
Proof. Write the rows of \(\mathbf{A}\), regarded as vectors in \(\mathbb{R}^n\), as \(\mathbf{r}_1,\dots,\mathbf{r}_m\). For any \(\mathbf{x}\in\mathbb{R}^n\), the \(i\)-th entry of \(\mathbf{A}\mathbf{x}\) is \(\mathbf{r}_i\cdot\mathbf{x}\). Thus \(\mathbf{A}\mathbf{x}=\mathbf{0}\) holds exactly when \(\mathbf{x}\) is orthogonal to every row. Being orthogonal to every row is equivalent to being orthogonal to every linear combination of rows, so \(\operatorname{Nul}\mathbf{A}=(\operatorname{Row}\mathbf{A})^\perp\). Taking orthogonal complements and using the double orthogonal complement property gives \((\operatorname{Nul}\mathbf{A})^\perp=(\operatorname{Row}\mathbf{A})^{\perp\perp}=\operatorname{Row}\mathbf{A}\).
Now regard the columns of \(\mathbf{A}\) as vectors \(\mathbf{a}_1,\dots,\mathbf{a}_n\in\mathbb{R}^m\). For any \(\mathbf{y}\in\mathbb{R}^m\), the \(j\)-th entry of \(\mathbf{A}^{\top}\mathbf{y}\) is \(\mathbf{a}_j\cdot\mathbf{y}\). Therefore \(\mathbf{A}^{\top}\mathbf{y}=\mathbf{0}\) exactly when \(\mathbf{y}\) is orthogonal to every column, or equivalently to every linear combination of columns. Hence \(\operatorname{Nul}(\mathbf{A}^{\top})=(\operatorname{Col}\mathbf{A})^\perp\). Taking orthogonal complements in \(\mathbb{R}^m\) gives \((\operatorname{Nul}(\mathbf{A}^{\top}))^\perp=(\operatorname{Col}\mathbf{A})^{\perp\perp}=\operatorname{Col}\mathbf{A}\). Both identities follow. \(\square\)
The proof establishes the equalities by first identifying the null space as the set of vectors orthogonal to the corresponding rows or columns. The double-complement step is what upgrades orthogonality to equality. Without it, the direct calculation alone gives the reverse-oriented statements \(\operatorname{Nul}\mathbf{A}=(\operatorname{Row}\mathbf{A})^\perp\) and \(\operatorname{Nul}(\mathbf{A}^{\top})=(\operatorname{Col}\mathbf{A})^\perp\).
Dimensions from Rank-Nullity
For \(\mathbf{A}\in\mathbb{R}^{m\times n}\), let \(r=\operatorname{rank}\mathbf{A}=\dim(\operatorname{Col}\mathbf{A})\). The Rank-Nullity Theorem says that the dimension of the null space of a linear map plus the dimension of its image equals the dimension of its domain. Apply it first to the map \(\mathbf{x}\mapsto\mathbf{A}\mathbf{x}\) from \(\mathbb{R}^n\) to \(\mathbb{R}^m\), and then to \(\mathbf{y}\mapsto\mathbf{A}^{\top}\mathbf{y}\) from \(\mathbb{R}^m\) to \(\mathbb{R}^n\).
Proof. Rank-nullity for \(\mathbf{x}\mapsto\mathbf{A}\mathbf{x}\) gives \(\dim(\operatorname{Nul}\mathbf{A})=n-r\). The theorem just proved gives \(\operatorname{Row}\mathbf{A}=(\operatorname{Nul}\mathbf{A})^\perp\); the dimension formula for orthogonal complements in \(\mathbb{R}^n\) then yields \(\dim(\operatorname{Row}\mathbf{A})=n-(n-r)=r\). By definition, \(\dim(\operatorname{Col}\mathbf{A})=r\). The columns of \(\mathbf{A}^{\top}\) are the transposes of the rows of \(\mathbf{A}\), so \(\operatorname{Col}(\mathbf{A}^{\top})\) and \(\operatorname{Row}\mathbf{A}\) have the same vectors under the usual identification of row coordinates with column vectors. Thus the map \(\mathbf{y}\mapsto\mathbf{A}^{\top}\mathbf{y}\) has image dimension \(r\). Rank-nullity for this map gives \(\dim(\operatorname{Nul}(\mathbf{A}^{\top}))=m-r\). \(\square\)
These dimensions describe complementary pairs in their respective spaces: the row space and null space dimensions add to \(n\), while the column space and left null space dimensions add to \(m\). Rank counts the dimensions of both the row space and column space; the two null spaces record the remaining dimensions in the input and output spaces.
Worked Examples in Both Ambient Spaces
Worked Example: A Full Row-Rank Matrix
Let \(\mathbf{A}=\begin{bmatrix}1&2&1\\0&1&1\end{bmatrix}\). Find its four fundamental subspaces and check the complementary dimensions.
The rows are \((1,2,1)\) and \((0,1,1)\). They are independent: if \(a(1,2,1)+b(0,1,1)=(0,0,0)\), the first coordinate gives \(a=0\), and the second then gives \(b=0\). Hence \(\operatorname{Row}\mathbf{A}=\operatorname{span}\{(1,2,1),(0,1,1)\}\) has dimension \(2\).
To find the null space, solve \(\mathbf{A}\mathbf{x}=\mathbf{0}\). Writing \(\mathbf{x}=(x_1,x_2,x_3)^\top\), the equations are \(x_1+2x_2+x_3=0\) and \(x_2+x_3=0\). Set \(x_3=t\). Then \(x_2=-t\), and \(x_1-2t+t=0\), so \(x_1=t\). Therefore \(\operatorname{Nul}\mathbf{A}=\operatorname{span}\{(1,-1,1)^\top\}\).
Check: \(\mathbf{A}(1,-1,1)^\top=(1-2+1,-1+1)^\top=(0,0)^\top\). Also, \((1,2,1)\cdot(1,-1,1)=1-2+1=0\) and \((0,1,1)\cdot(1,-1,1)=-1+1=0\).
The columns \((1,0)^\top\) and \((2,1)^\top\) span \(\mathbb{R}^2\): for any \((u,v)^\top\), \( (u-2v)(1,0)^\top+v(2,1)^\top=(u,v)^\top\). Thus \(\operatorname{Col}\mathbf{A}=\mathbb{R}^2\). If \(\mathbf{y}=(s,t)^\top\), then \(\mathbf{A}^{\top}\mathbf{y}=(s,2s+t,s+t)^\top\). Setting this equal to zero gives \(s=0\), then \(t=0\), so \(\operatorname{Nul}(\mathbf{A}^{\top})=\{\mathbf{0}\}\).
Dimension check: \(2+1=3\) in \(\mathbb{R}^3\), and \(2+0=2\) in \(\mathbb{R}^2\). The column space is the entire output space, so its orthogonal complement is just the zero vector.
Worked Example: Rank One and Two Nontrivial Complements
Let \(\mathbf{B}=\begin{bmatrix}1&2\\2&4\\0&0\end{bmatrix}\). Determine the four spaces, including the left null space.
The rows are \((1,2)\), \((2,4)\), and \((0,0)\). Since the second row is twice the first and the third is zero, \(\operatorname{Row}\mathbf{B}=\operatorname{span}\{(1,2)\}\), of dimension \(1\). The columns are \((1,2,0)^\top\) and \((2,4,0)^\top=2(1,2,0)^\top\), so \(\operatorname{Col}\mathbf{B}=\operatorname{span}\{(1,2,0)^\top\}\), also of dimension \(1\).
For the null space, \(\mathbf{B}(x_1,x_2)^\top=(x_1+2x_2,2x_1+4x_2,0)^\top\). The first equation gives \(x_1=-2x_2\); the second is then automatically satisfied. Thus \(\operatorname{Nul}\mathbf{B}=\operatorname{span}\{(-2,1)^\top\}\).
For the left null space, \(\mathbf{B}^{\top}\mathbf{y}=\mathbf{0}\) with \(\mathbf{y}=(y_1,y_2,y_3)^\top\) gives \(y_1+2y_2=0\) and \(2y_1+4y_2=0\). The second equation is twice the first, and \(y_3\) is unrestricted. Setting \(y_2=s\) and \(y_3=t\) gives \(\mathbf{y}=s(-2,1,0)^\top+t(0,0,1)^\top\). Hence \(\operatorname{Nul}(\mathbf{B}^{\top})=\operatorname{span}\{(-2,1,0)^\top,(0,0,1)^\top\}\).
Check: \(\mathbf{B}(-2,1)^\top=(-2+2,-4+4,0)^\top=\mathbf{0}\). Also \(\mathbf{B}^{\top}(-2,1,0)^\top=(-2+2,-4+4)^\top=\mathbf{0}\), and \(\mathbf{B}^{\top}(0,0,1)^\top=(0,0)^\top\). The row vector \((1,2)\) is orthogonal to \((-2,1)\), since \(-2+2=0\), and the column vector \((1,2,0)^\top\) is orthogonal to both displayed generators of the left null space.
Here \(r=1\). The dimensions are \(1+1=2\) in \(\mathbb{R}^2\) and \(1+2=3\) in \(\mathbb{R}^3\), as required.
Worked Example: Full Column Rank with a Nonzero Left Null Space
Let \(\mathbf{C}=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}\). This example has two independent columns but has outputs in \(\mathbb{R}^3\).
The rows \((1,0)\) and \((0,1)\) span \(\mathbb{R}^2\), so \(\operatorname{Row}\mathbf{C}=\mathbb{R}^2\). Solving \(\mathbf{C}\mathbf{x}=\mathbf{0}\) gives \(x_1=0\), \(x_2=0\), and \(x_1+x_2=0\); thus \(\operatorname{Nul}\mathbf{C}=\{\mathbf{0}\}\). Consequently \(\operatorname{Row}\mathbf{C}=(\operatorname{Nul}\mathbf{C})^\perp=\mathbb{R}^2\).
The columns are \((1,0,1)^\top\) and \((0,1,1)^\top\), and they are independent because a combination \(a(1,0,1)^\top+b(0,1,1)^\top=\mathbf{0}\) has first coordinate \(a=0\) and second coordinate \(b=0\). For \(\mathbf{y}=(y_1,y_2,y_3)^\top\), the left null equations are \(y_1+y_3=0\) and \(y_2+y_3=0\). Setting \(y_3=t\) gives \(\mathbf{y}=t(-1,-1,1)^\top\), so \(\operatorname{Nul}(\mathbf{C}^{\top})=\operatorname{span}\{(-1,-1,1)^\top\}\).
Check: \(\mathbf{C}^{\top}(-1,-1,1)^\top=(-1+1,-1+1)^\top=(0,0)^\top\). The dot products with the columns are \((1,0,1)\cdot(-1,-1,1)=-1+1=0\) and \((0,1,1)\cdot(-1,-1,1)=-1+1=0\). Here the dimensions add to \(2\) in \(\mathbb{R}^2\) and to \(3\) in \(\mathbb{R}^3\).
A Common Confusion: Match the Ambient Space
A frequent mistake is to write \(\operatorname{Col}\mathbf{A}=(\operatorname{Nul}\mathbf{A})^\perp\). This pairs the wrong spaces. For the first example, \(\operatorname{Nul}\mathbf{A}\subseteq\mathbb{R}^3\), so its orthogonal complement is also in \(\mathbb{R}^3\); meanwhile, \(\operatorname{Col}\mathbf{A}\subseteq\mathbb{R}^2\). In that example, \((\operatorname{Nul}\mathbf{A})^\perp=\operatorname{Row}\mathbf{A}\) has dimension \(2\), while \(\operatorname{Col}\mathbf{A}=\mathbb{R}^2\). The dimensions happen to agree, but these subspaces belong to different ambient spaces and are not the asserted pair.
The correction is to match each subspace with the null space in its own coordinate space. In \(\mathbb{R}^n\), use \(\operatorname{Row}\mathbf{A}=(\operatorname{Nul}\mathbf{A})^\perp\). In \(\mathbb{R}^m\), use \(\operatorname{Col}\mathbf{A}=(\operatorname{Nul}(\mathbf{A}^{\top}))^\perp\). The transpose is essential in the second identity because \(\mathbf{A}^{\top}\) acts on vectors in the output space \(\mathbb{R}^m\).
| Ambient space | Subspace | Orthogonal complement | Dimensions |
|---|---|---|---|
| \(\mathbb{R}^n\) | \(\operatorname{Row}\mathbf{A}\) | \(\operatorname{Nul}\mathbf{A}\) | \(r\) and \(n-r\) |
| \(\mathbb{R}^m\) | \(\operatorname{Col}\mathbf{A}\) | \(\operatorname{Nul}(\mathbf{A}^{\top})\) | \(r\) and \(m-r\) |
The four spaces summarize different aspects of the same matrix. The column space describes the outputs that can occur, and the null space describes inputs sent to zero. The row space describes the linear constraints imposed on an input by the rows, while the left null space consists of output-space vectors orthogonal to every attainable output. Their dimensions are tied together by rank-nullity, but their locations in \(\mathbb{R}^n\) or \(\mathbb{R}^m\) determine which orthogonal-complement identity applies.
Check Your Understanding
For each question, identify the ambient space before using an orthogonal-complement identity.
- For a real \(4\times 3\) matrix \(\mathbf{A}\), in which coordinate spaces do \(\operatorname{Row}\mathbf{A}\), \(\operatorname{Nul}\mathbf{A}\), \(\operatorname{Col}\mathbf{A}\), and \(\operatorname{Nul}(\mathbf{A}^{\top})\) lie?
- If \(\operatorname{rank}\mathbf{A}=2\) for a \(5\times 4\) matrix, find the dimensions of all four fundamental subspaces.
- Explain why \(\mathbf{A}\mathbf{x}=\mathbf{0}\) means that \(\mathbf{x}\) is orthogonal to every row of \(\mathbf{A}\).
- For \(\mathbf{B}=\begin{bmatrix}1&2\\2&4\\0&0\end{bmatrix}\), verify that \((-2,1,0)^\top\) belongs to \(\operatorname{Nul}(\mathbf{B}^{\top})\).
- State the correct orthogonal-complement identity for the column space, and explain why it involves \(\mathbf{A}^{\top}\).
