How Orthogonal Changes Preserve Singular Values

Orthogonal Changes of Coordinates

An orthogonal matrix changes coordinates without changing lengths or angles. In a matrix product, an orthogonal matrix on the left changes the output coordinates, while one on the right changes the input coordinates. The singular values measure how much a matrix stretches in its principal directions, so it is natural to ask whether these coordinate changes alter those amounts. They do not: multiplying by an orthogonal matrix on either side preserves every singular value.

Throughout this tutorial, matrices and vectors have real entries. A square matrix \(\mathbf{Q}\in\mathbb{R}^{m\times m}\) is orthogonal if \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}_m\). This condition also gives \(\mathbf{Q}\mathbf{Q}^{\top}=\mathbf{I}_m\), so \(\mathbf{Q}^{-1}=\mathbf{Q}^{\top}\). For a matrix \(\mathbf{A}\in\mathbb{R}^{m\times n}\), multiplication on the left must use an \(m\times m\) orthogonal matrix; multiplication on the right must use an \(n\times n\) orthogonal matrix.

The singular values are the diagonal entries in an SVD’s middle factor, counted with multiplicity and arranged in nonincreasing order. The SVD theory established earlier ensures that such a factorization exists. The key idea here is to transform the outer factors while keeping the same middle factor.

The Preservation Theorem

Proof. Take a full SVD \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\). For left multiplication,

The new left factor is orthogonal, since

Thus this is an SVD of \(\mathbf{Q}\mathbf{A}\) with the same middle factor \(\boldsymbol{\Sigma}\). For right multiplication, write

Here \(\mathbf{R}^{\top}\mathbf{V}\) is orthogonal because \((\mathbf{R}^{\top}\mathbf{V})^{\top}(\mathbf{R}^{\top}\mathbf{V})=\mathbf{V}^{\top}\mathbf{R}\mathbf{R}^{\top}\mathbf{V}=\mathbf{I}_n\). So this is an SVD with the same \(\boldsymbol{\Sigma}\). Applying both transformations gives

Both outer factors are orthogonal, and the middle factor has not changed. In all three cases the singular values—the entries of that middle factor—are therefore unchanged. \(\square\)

This proof gives more than equality of the singular values: it tells us how the singular vectors change. The left vectors are multiplied by \(\mathbf{Q}\), and the right vectors are multiplied by \(\mathbf{R}^{\top}\) when the matrix is multiplied on the right by \(\mathbf{R}\).

Proof. The displayed product equals \(\mathbf{Q}\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\mathbf{R}=\mathbf{Q}\mathbf{A}\mathbf{R}\), by the definition of transpose and matrix multiplication. The preservation theorem’s orthogonality calculations show that \(\mathbf{Q}\mathbf{U}\) and \(\mathbf{R}^{\top}\mathbf{V}\) are orthogonal. The middle matrix is still diagonal in the rectangular sense with nonnegative, nonincreasing entries. These are precisely the requirements for an SVD. \(\square\)

Left Multiplication: Changing Output Coordinates

Right Multiplication: Changing Input Coordinates

Simplifying an SVD by Undoing Orthogonal Factors

The theorem is especially useful when a matrix looks complicated because its input and output coordinates have been rotated. If we can write \(\mathbf{A}=\mathbf{Q}\mathbf{D}\mathbf{R}\), where \(\mathbf{Q}\) and \(\mathbf{R}\) are orthogonal and \(\mathbf{D}\) has a simple SVD, then \(\mathbf{A}\) has the same singular values as \(\mathbf{D}\). Equivalently, multiplying by \(\mathbf{Q}^{\top}\) on the left and \(\mathbf{R}^{\top}\) on the right recovers \(\mathbf{D}\), since \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}\) and \(\mathbf{R}\mathbf{R}^{\top}=\mathbf{I}\).

Common Pitfalls

Orthogonal is essential. It is incorrect to conclude that any invertible matrix preserves singular values. For example, \(\mathbf{I}_2\) has singular values \(1,1\), while \(2\mathbf{I}_2\) has singular values \(2,2\). The factor \(2\mathbf{I}_2\) is invertible but not orthogonal, because \((2\mathbf{I}_2)^{\top}(2\mathbf{I}_2)=4\mathbf{I}_2\ne\mathbf{I}_2\). The preservation theorem requires orthogonality, not merely invertibility.

On the right, the transformed right factor uses a transpose. If \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\), then \(\mathbf{A}\mathbf{R}=\mathbf{U}\boldsymbol{\Sigma}(\mathbf{R}^{\top}\mathbf{V})^{\top}\). A common wrong step is to use \(\mathbf{R}\mathbf{V}\) as the new right factor. For a concrete check, take \(\mathbf{A}=\mathbf{I}_2\) and \(\mathbf{R}=\begin{bmatrix}0&-1\\1&0\end{bmatrix}\). The correct right factor is \(\mathbf{R}^{\top}\), giving \(\mathbf{I}_2\mathbf{I}_2(\mathbf{R}^{\top})^{\top}=\mathbf{R}\). Using \(\mathbf{R}\) instead gives \(\mathbf{R}^{\top}\), which is not \(\mathbf{R}\). The singular values happen to remain \(1,1\), but the proposed factorization is wrong.

The practical method is to identify the orthogonal factors, check their dimensions and verify \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}\) and \(\mathbf{R}^{\top}\mathbf{R}=\mathbf{I}\). Then simplify the middle matrix, retain its singular values, and transform the singular-vector factors with the stated formulas. This avoids repeating a full SVD calculation when the matrix is only a rotated or otherwise orthogonally re-expressed version of a simpler one.