Constructing Left Singular Vectors

From a Right Singular Vector to a Left One

For a real matrix \(\mathbf{A}\), a right singular vector describes a direction in the input space, while its image under \(\mathbf{A}\) lies in the output space. When the right singular vector corresponds to a positive singular value, that image is nonzero and has length equal to the singular value. Dividing the image by that length produces a unit vector in the output space: the corresponding left singular vector.

Let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). The matrix \(\mathbf{A}^{\top}\mathbf{A}\) is \(n\times n\), so its eigenvectors are vectors in \(\mathbb{R}^n\), the input space of \(\mathbf{A}\). A unit right singular vector \(\mathbf{v}\) associated with a positive singular value \(\sigma\) is a unit eigenvector satisfying \(\mathbf{A}^{\top}\mathbf{A}\mathbf{v}=\sigma^2\mathbf{v}\), where \(\sigma>0\). The positive singular values are the positive square roots of the eigenvalues of \(\mathbf{A}^{\top}\mathbf{A}\). The construction below turns \(\mathbf{v}\in\mathbb{R}^n\) into a vector \(\mathbf{u}\in\mathbb{R}^m\).

The positivity condition matters: division by \(\sigma\) is valid only when \(\sigma\ne0\). Also, \(\mathbf{A}\mathbf{v}\) is not usually already a unit vector. Its length is \(\sigma\), which is why the formula includes division by \(\sigma\), rather than simply using the image as the left vector.

Why the Formula Works

The key calculation is the squared length of the image of a unit right singular vector. For any \(\mathbf{x}\in\mathbb{R}^n\), the Euclidean norm satisfies \(\|\mathbf{A}\mathbf{x}\|^2=(\mathbf{A}\mathbf{x})^{\top}(\mathbf{A}\mathbf{x})=\mathbf{x}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{x}\). Applying this to \(\mathbf{v}\), and using its eigenvector equation, gives the required length.

Proof. Assume first that \(\|\mathbf{v}\|=1\) and \(\mathbf{A}^{\top}\mathbf{A}\mathbf{v}=\sigma^2\mathbf{v}\), with \(\sigma>0\). Using the definition of the Euclidean norm and the eigenvector equation,

Since a norm is nonnegative and \(\sigma>0\), this gives \(\|\mathbf{A}\mathbf{v}\|=\sigma\). Therefore \(\|\mathbf{u}\|=\|\mathbf{A}\mathbf{v}/\sigma\|=\sigma/\sigma=1\). The definition also gives \(\mathbf{A}\mathbf{v}=\sigma\mathbf{u}\). Finally,

For the converse, suppose \(\mathbf{A}\mathbf{v}=\sigma\mathbf{u}\) and \(\mathbf{A}^{\top}\mathbf{u}=\sigma\mathbf{v}\). Then \(\mathbf{A}^{\top}\mathbf{A}\mathbf{v}=\mathbf{A}^{\top}(\sigma\mathbf{u})=\sigma(\mathbf{A}^{\top}\mathbf{u})=\sigma(\sigma\mathbf{v})=\sigma^2\mathbf{v}\), as required. \(\square\)

The theorem explains both the construction and the check for a computed answer. Starting with a right singular vector, calculate \(\mathbf{A}\mathbf{v}/\sigma\), then verify that the result has unit length and that the two equations hold. The first equation checks the image and its scale; the second checks that the constructed vector is paired with the original right singular vector.

Computing the Left Singular Vectors

What the Construction Does—and Does Not—Provide

For every chosen unit right singular vector associated with a positive singular value, the formula gives a specific left singular vector. If a right singular vector is replaced by its negative, its image and the resulting left singular vector both change sign, while the singular value stays positive. The pair of equations continues to hold. More generally, the construction is attached to the particular right vector chosen; it is not a rule for selecting right singular vectors in the first place.

The dimensions are a useful check on calculations. If \(\mathbf{A}\in\mathbb{R}^{m\times n}\), then \(\mathbf{v}\) has \(n\) entries, \(\mathbf{A}\mathbf{v}\) has \(m\) entries, and \(\mathbf{u}\) must have \(m\) entries. A proposed answer of the wrong length cannot be a left singular vector for that matrix. The second equation, \(\mathbf{A}^{\top}\mathbf{u}=\sigma\mathbf{v}\), is another efficient way to catch an arithmetic or scaling error.

This construction establishes that each resulting \(\mathbf{u}\) is a unit vector and forms the stated pair with \(\mathbf{v}\). It does not, by itself, establish that left vectors obtained from different right vectors are orthogonal; that is a separate property requiring its own argument. Keeping the claims distinct prevents the construction formula from being used as a substitute for a proof of additional structure.

Common Pitfall: Dividing by the Wrong Quantity

A common error is to divide \(\mathbf{A}\mathbf{v}\) by \(\sigma^2\), perhaps because the eigenvalue of \(\mathbf{A}^{\top}\mathbf{A}\) is \(\sigma^2\). In the third example, \(\|\mathbf{A}\mathbf{v}_1\|=\sqrt{3}\) and \(\sigma=\sqrt{3}\). Dividing that image by the eigenvalue \(3\) would give a vector of length \(\sqrt{3}/3=1/\sqrt{3}\), not \(1\). The correction is to divide by the singular value \(\sigma\), giving a unit vector. The eigenvalue supplies the scale through its positive square root; it is the square root, not the eigenvalue itself, that belongs in the denominator.

A second error is trying to apply the formula when \(\sigma=0\). In that case \(\mathbf{A}\mathbf{v}=\mathbf{0}\), so the quotient \(\mathbf{A}\mathbf{v}/\sigma\) is undefined. The construction in this tutorial is specifically for positive singular values. It neither defines nor selects a left singular vector for a zero singular value.