Null Spaces And Images From Singular Vectors

What Singular Vectors Tell Us About a Matrix

A singular value decomposition describes how a matrix acts on orthonormal directions. Each right singular vector is an input direction; its singular value tells us whether the matrix sends that direction to zero or stretches it by a positive amount. The corresponding left singular vector gives the output direction. This makes the SVD a direct way to describe both which inputs are annihilated and which outputs are attainable.

Let \(\mathbf{A}\in\mathbb{F}^{m\times n}\), where \(\mathbb{F}=\mathbb{R}\) or \(\mathbb{C}\), and set \(p=\min(m,n)\). A full SVD is a factorization \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}\), where \(\mathbf{U}\in\mathbb{F}^{m\times m}\) and \(\mathbf{V}\in\mathbb{F}^{n\times n}\) are unitary (orthogonal when \(\mathbb{F}=\mathbb{R}\)). The columns of \(\mathbf{U}\) are \(\mathbf{u}_1,\dots,\mathbf{u}_m\); those of \(\mathbf{V}\) are \(\mathbf{v}_1,\dots,\mathbf{v}_n\). The rectangular diagonal matrix \(\boldsymbol{\Sigma}\in\mathbb{R}^{m\times n}\) has entries \(\sigma_1\ge\cdots\ge\sigma_p\ge0\) on its diagonal and zeros elsewhere. For complex matrices, \(\mathbf{V}^{*}\) is the conjugate transpose; for real matrices, it is \(\mathbf{V}^{\top}\).

In a full SVD, there may be right singular vectors \(\mathbf{v}_{p+1},\dots,\mathbf{v}_n\) when \(n>p\), and left singular vectors \(\mathbf{u}_{p+1},\dots,\mathbf{u}_m\) when \(m>p\). These unpaired vectors matter: the extra right vectors belong to the null space, while the extra left vectors cannot belong to the image. When a list of spanning vectors has no members, its span is understood to be \(\{\mathbf{0}\}\).

The Span Descriptions

Proof. Since the columns of \(\mathbf{V}\) form an orthonormal basis of \(\mathbb{F}^n\), every \(\mathbf{x}\in\mathbb{F}^n\) has a unique expression \(\mathbf{x}=\sum_{j=1}^{n}c_j\mathbf{v}_j\). Multiplying by \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}\) gives \[ \mathbf{A}\mathbf{x}=\sum_{i=1}^{p}\sigma_i c_i\mathbf{u}_i. \] The vectors \(\mathbf{u}_1,\dots,\mathbf{u}_p\) are orthonormal, so this sum is zero exactly when \(\sigma_i c_i=0\) for every \(i\le p\). If \(i\in P\), then \(\sigma_i>0\), forcing \(c_i=0\). If \(i\in Z\), then \(\sigma_i=0\), so \(c_i\) may be arbitrary. The coefficients \(c_{p+1},\dots,c_n\), when present, do not occur in the product and are also arbitrary. This proves the null-space description in both directions. For the image, the displayed formula shows that every \(\mathbf{A}\mathbf{x}\) is a linear combination of the \(\mathbf{u}_i\) with \(i\in P\). Conversely, if \(\mathbf{y}=\sum_{i\in P}d_i\mathbf{u}_i\), set \(\mathbf{x}=\sum_{i\in P}(d_i/\sigma_i)\mathbf{v}_i\). Each denominator is nonzero by the definition of \(P\), and multiplication gives \(\mathbf{A}\mathbf{x}=\sum_{i\in P}d_i\mathbf{u}_i=\mathbf{y}\). Thus every vector in that span is in the image. \(\square\)

The two descriptions use different sides of the SVD. The null space is a subspace of the input space \(\mathbb{F}^n\), so it is described by right singular vectors. The image is a subspace of the output space \(\mathbb{F}^m\), so it is described by left singular vectors. A positive singular value pairs a right direction with an output direction; a zero singular value means that the paired right direction is sent to zero.

Worked Examples

Orthogonality and Rectangular Cases

Proof. The full-SVD theorem shows that the null space is spanned by a subset of the orthonormal basis \(\mathbf{v}_1,\dots,\mathbf{v}_n\): the vectors indexed by \(Z\) and the extra right vectors, if any. Every remaining basis vector is indexed by \(P\). In an orthonormal basis, a vector is perpendicular to the span of a selected subset exactly when its coefficients on that subset are zero. Therefore the orthogonal complement of the null space is the span of the remaining vectors, those indexed by \(P\). The image is spanned by the subset \(\{\mathbf{u}_i:i\in P\}\) of the orthonormal basis \(\mathbf{u}_1,\dots,\mathbf{u}_m\). The vectors in the complementary subset are exactly those indexed by \(Z\) and the extra left vectors with indices greater than \(p\). The same orthonormal-basis argument shows that their span is the orthogonal complement of the image. \(\square\)

The extra vectors explain why the full SVD is useful for these descriptions. For a wide matrix, the rectangular diagonal \(\boldsymbol{\Sigma}\) has fewer diagonal positions than there are columns in \(\mathbf{V}\); its extra right singular vectors are automatically sent to zero. For a tall matrix, there are extra columns in \(\mathbf{U}\), but the matrix cannot produce a component in any of those output directions. The theorem also covers the zero matrix: all singular values are zero, every right direction lies in the null space, and the image is \(\{\mathbf{0}\}\).

Common Pitfall: Forgetting the Extra Right Vectors

For the \(2\times3\) matrix in the first worked example, a tempting but wrong conclusion is that the null space is \(\{\mathbf{0}\}\), because both listed singular values are positive. This treats the two diagonal entries as if they described all three input directions. But \(p=2\), while \(\mathbf{V}\) has three columns. The third right singular vector is unpaired with a diagonal entry and is sent to zero: \[ \mathbf{A}\mathbf{v}_3 =\begin{bmatrix}1&0&1\\0&2&0\end{bmatrix} \begin{bmatrix}1/\sqrt{2}\\0\\-1/\sqrt{2}\end{bmatrix} =\begin{bmatrix}0\\0\end{bmatrix}. \] Thus the correct null space is \(\operatorname{span}\{\mathbf{v}_3\}\), not \(\{\mathbf{0}\}\). The correction is to use a full SVD and include every extra right singular vector. In a compact SVD those extra vectors are omitted, so a compact factorization alone does not list the whole null space.

Another error is to use left singular vectors to describe the null space. Their coordinates lie in the output space \(\mathbb{F}^m\), whereas null-space vectors must be inputs in \(\mathbb{F}^n\). Match the space to the task: right singular vectors for inputs and the null space; left singular vectors for outputs and the image.