Singular Values Under Scaling And Transposition

Two Transformations, Two Useful Invariances

Singular values describe how a matrix stretches directions, but they do not change arbitrarily when we alter the matrix in simple ways. Multiplying a matrix by a scalar changes every stretch by the scalar’s size, while transposing the matrix preserves its positive singular values. These facts are useful checks on an SVD and help separate the information carried by a matrix from effects caused only by its orientation or dimensions.

Throughout, let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). An SVD of \(\mathbf{A}\) is a factorization \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\), where \(\mathbf{U}\in\mathbb{R}^{m\times m}\) and \(\mathbf{V}\in\mathbb{R}^{n\times n}\) are orthogonal, and \(\boldsymbol{\Sigma}\in\mathbb{R}^{m\times n}\) is diagonal in the rectangular sense: its only potentially nonzero entries are on its main diagonal. Write \(p=\min(m,n)\). The singular values are the \(p\) diagonal entries \(\sigma_1\ge\cdots\ge\sigma_p\ge0\) in such a factorization. The positive singular values are the entries in this list that are strictly greater than zero.

The sign of a scalar and the sign of a singular value play different roles. Singular values are always nonnegative, so multiplying a matrix by a negative scalar cannot make its singular values negative. Instead, the sign is absorbed into an orthogonal factor of the SVD. Transposition exchanges the left and right singular-vector roles; it does not change the nonzero diagonal entries in the SVD.

Scaling a Matrix

Proof. First suppose \(c\ne0\), and take an SVD \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\). Put \(s=c/|c|\), so \(s\) equals \(1\) or \(-1\). Then \(s\mathbf{U}\) is orthogonal because \((s\mathbf{U})^{\top}(s\mathbf{U})=s^2\mathbf{U}^{\top}\mathbf{U}=\mathbf{I}_m\). Also, \(|c|\boldsymbol{\Sigma}\) has diagonal entries \(|c|\sigma_1,\dots,|c|\sigma_p\), still nonnegative and in descending order. Consequently \(c\mathbf{A}=(s\mathbf{U})(|c|\boldsymbol{\Sigma})\mathbf{V}^{\top}\) is an SVD, proving the claim when \(c\ne0\). If \(c=0\), then \(c\mathbf{A}\) is the zero matrix. Its SVD has all diagonal entries zero, which are exactly \(|c|\sigma_i=0\) for every \(i\). \(\square\)

When \(c=0\), every direction is collapsed and every singular value becomes zero. When \(c\ne0\), the ordering is preserved because multiplying nonnegative numbers by the same positive number \(|c|\) preserves their order. In particular, scaling does not change which singular values are zero unless the scalar itself is zero.

Transposing a Matrix

Proof. Take a full SVD \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\), with \(\mathbf{U}\) and \(\mathbf{V}\) square orthogonal matrices. Transposing both sides gives \(\mathbf{A}^{\top}=\mathbf{V}\boldsymbol{\Sigma}^{\top}\mathbf{U}^{\top}\). The factors \(\mathbf{V}\) and \(\mathbf{U}\) are orthogonal, so this is an SVD of \(\mathbf{A}^{\top}\). The rectangular diagonal matrix \(\boldsymbol{\Sigma}^{\top}\) has the same entries on its main diagonal as \(\boldsymbol{\Sigma}\); its shape is simply \(n\times m\) instead of \(m\times n\). Thus the diagonal entries, including every positive entry and its multiplicity, are unchanged. \(\square\)

Under the convention used here, both \(\mathbf{A}\) and \(\mathbf{A}^{\top}\) have \(p=\min(m,n)\) singular values. Their entire lists therefore agree, including any zero entries in those lists. The theorem emphasizes positive singular values because the matrices \(\mathbf{A}^{\top}\mathbf{A}\) and \(\mathbf{A}\mathbf{A}^{\top}\), which are square matrices of different sizes when \(m\ne n\), can have different numbers of zero eigenvalues. Their positive eigenvalues correspond to the squares of the positive singular values.

Using Both Rules Together

The scaling and transposition results can be combined: the positive singular values of \(c\mathbf{A}^{\top}\) are \(|c|\) times the positive singular values of \(\mathbf{A}\). The order of these operations does not affect the conclusion. The example below checks both the numerical values and the rectangular shapes.

A Common Mistake: Comparing Zero Eigenvalues Across Shapes

Wrong calculation: For the \(2\times3\) matrix \(\mathbf{A}=\begin{bmatrix}1&0&0\\0&0&0\end{bmatrix}\), one might compute \(\mathbf{A}^{\top}\mathbf{A}=\operatorname{diag}(1,0,0)\) and \(\mathbf{A}\mathbf{A}^{\top}=\operatorname{diag}(1,0)\), then conclude that transposition changes the singular values because the first product has two zero eigenvalues and the second has one.

Correction: Those are eigenvalue lists for square matrices of different sizes, not singular-value lists of the same length. The singular-value list for each \(2\times3\) or \(3\times2\) matrix has length \(\min(2,3)=2\), so in both cases it is \((1,0)\). The positive singular value agrees, and the zero counts in the two Gram-matrix eigenvalue lists merely reflect their different dimensions. Do not compare full eigenvalue lists from different-sized matrices as though their lengths must match.

For a reliable check, first identify the matrix dimensions and the singular-value convention being used. Then use the SVD: scaling multiplies the nonnegative diagonal entries by \(|c|\), while transposition replaces \(\boldsymbol{\Sigma}\) by \(\boldsymbol{\Sigma}^{\top}\) without changing its diagonal entries. This also handles repeated singular values and rank-deficient matrices without requiring separate cases.