Diagnosing And Verifying A Proposed SVD

A Verification Checklist

A proposed singular value decomposition can look plausible and still fail in several different ways. The dimensions might not allow the multiplication, a column might not have unit length, or the diagonal entries might be out of order. Even if all those checks pass, the product may not reconstruct the matrix in question. A reliable diagnosis separates these conditions rather than treating “the factors look right” as a single test.

Work over \(\mathbb{R}\) or \(\mathbb{C}\). For a complex matrix, use the conjugate transpose \(\mathbf{A}^{*}\); for a real matrix, this is the transpose \(\mathbf{A}^{\top}\). A square matrix \(\mathbf{U}\) is unitary if \(\mathbf{U}^{*}\mathbf{U}=\mathbf{U}\mathbf{U}^{*}=\mathbf{I}\). A real unitary matrix is called orthogonal. A list of columns is orthonormal when every column has norm \(1\) and distinct columns are orthogonal.

Here “diagonal in the rectangular sense” means that \(\Sigma_{ij}=0\) whenever \(i\ne j\). The numbers \(\sigma_1,\dots,\sigma_q\) are the singular values, listed in nonincreasing order. Recall that the singular values are the nonnegative square roots of the eigenvalues of \(\mathbf{A}^{*}\mathbf{A}\), with the list padded by zeros when needed. The earlier result that \(\mathbf{A}^{*}\mathbf{A}\) has an orthonormal eigenbasis underlies the existence of an SVD; for verification, the checks below are direct.

For a full SVD, dimensions must be exactly \(m\times m\), \(m\times n\), and \(n\times n\) for \(\mathbf{U}\), \(\boldsymbol{\Sigma}\), and \(\mathbf{V}\). A thin SVD uses fewer columns: with \(q=\min(m,n)\), its factors have sizes \(m\times q\), \(q\times q\), and \(n\times q\). The thin \(\mathbf{U}\) and \(\mathbf{V}\) must have orthonormal columns, and the diagonal of the middle factor must still be nonnegative and nonincreasing. Do not mix the dimensions of one convention with the conditions of another. In either convention, the final test is multiplication: the factors must produce the particular matrix being decomposed.

Why the Checks Identify Singular Data

Proof. If the proposed factors form a full SVD, they satisfy these conditions by the definition. Conversely, suppose the stated conditions hold. The matrix product is defined, the unitary factors have orthonormal columns, the middle factor has the required diagonal form and ordered nonnegative entries, and the assumed equality reconstructs \(\mathbf{A}\). These are exactly the defining conditions for a full SVD. To confirm that the listed entries are singular values, put \(q=\min(m,n)\) and let \(d_j=\sigma_j^2\) for \(1\le j\le q\), with \(d_j=0\) for \(q<j\le n\). Then \(\mathbf{A}^{*}\mathbf{A}=\mathbf{V}(\boldsymbol{\Sigma}^{*}\boldsymbol{\Sigma})\mathbf{V}^{*}=\mathbf{V}\operatorname{diag}(d_1,\dots,d_n)\mathbf{V}^{*}\). Thus the columns \(\mathbf{v}_j\) of \(\mathbf{V}\) are eigenvectors of \(\mathbf{A}^{*}\mathbf{A}\) with eigenvalues \(d_j\). Their nonnegative square roots, in order, are the stated \(\sigma_j\), with any additional entries equal to zero. \(\square\)

For a square proposed factor, the column check is usually written as \(\mathbf{U}^{*}\mathbf{U}=\mathbf{I}\). This says precisely that its columns are orthonormal. It also implies unitarity: the columns are linearly independent, so a square matrix with those columns is invertible, and \(\mathbf{U}^{*}\mathbf{U}=\mathbf{I}\) then gives \(\mathbf{U}^{-1}=\mathbf{U}^{*}\), hence \(\mathbf{U}\mathbf{U}^{*}=\mathbf{I}\). The same argument applies to \(\mathbf{V}\). For a thin factor, by contrast, orthonormal columns do not make it square or unitary; the appropriate test is just \(\mathbf{U}^{*}\mathbf{U}=\mathbf{I}_q\), or \(\mathbf{V}^{*}\mathbf{V}=\mathbf{I}_q\).

Proof. Write the columns as \(\mathbf{q}_1,\dots,\mathbf{q}_k\). The entry in row \(i\), column \(j\) of \(\mathbf{Q}^{*}\mathbf{Q}\) is \(\mathbf{q}_i^{*}\mathbf{q}_j\). If the columns are orthonormal, this entry is \(1\) when \(i=j\) and \(0\) when \(i\ne j\), so the product is the identity. Conversely, if the product is the identity, its diagonal entries give \(\mathbf{q}_i^{*}\mathbf{q}_i=\|\mathbf{q}_i\|^2=1\), and its off-diagonal entries give \(\mathbf{q}_i^{*}\mathbf{q}_j=0\) for \(i\ne j\). Therefore the columns are orthonormal. \(\square\)

This test reduces many separate dot products to one small matrix multiplication. For real matrices it uses \(\mathbf{Q}^{\top}\mathbf{Q}\); for complex matrices it must use \(\mathbf{Q}^{*}\mathbf{Q}\), not merely the transpose. The reconstruction check is independent: passing the Gram-matrix test does not establish that the proposed factors multiply to \(\mathbf{A}\).

Worked Checks

Reconstruction as a Sum of Rank-One Terms

A second way to check the multiplication is to expand it by columns. Write \(\mathbf{u}_j\) and \(\mathbf{v}_j\) for the \(j\)th columns of the full factors. Each term \(\mathbf{u}_j\mathbf{v}_j^{*}\) is an \(m\times n\) matrix, so it has the same dimensions as \(\mathbf{A}\).

Proof. The rectangular diagonal matrix can be written as \(\boldsymbol{\Sigma}=\sum_{j=1}^{q}\sigma_j\mathbf{e}_j\mathbf{e}_j^{\top}\), where the standard basis vectors in each term have the dimensions required by the rectangular product. Multiplying on the left by \(\mathbf{U}\) and on the right by \(\mathbf{V}^{*}\) gives \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}=\sum_{j=1}^{q}\sigma_j(\mathbf{U}\mathbf{e}_j)(\mathbf{V}\mathbf{e}_j)^{*}=\sum_{j=1}^{q}\sigma_j\mathbf{u}_j\mathbf{v}_j^{*}\). Since \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}=\mathbf{A}\), the formula follows. \(\square\)

This identity is useful when an entry-by-entry multiplication is awkward: it checks that each singular-vector pair contributes the right matrix term. It does not replace the other tests. If the columns are not orthonormal, or if the diagonal entries are misordered, a sum that reconstructs \(\mathbf{A}\) still does not satisfy the full SVD conditions.

Common Mistakes and a Reliable Order of Work

Using a transpose for a complex factor. For complex matrices, \(\mathbf{V}^{\top}\) is not generally \(\mathbf{V}^{*}\). For example, with \(\mathbf{v}=(1,i)^{\top}\), we have \(\mathbf{v}^{*}\mathbf{v}=1+(-i)i=2\), whereas \(\mathbf{v}^{\top}\mathbf{v}=1+i^2=0\). The transpose would give the wrong inner-product test. Use conjugate transpose in every complex orthonormality and reconstruction check.

Checking only that the middle factor is diagonal. A diagonal matrix can have a negative diagonal entry or entries in the wrong order. Either violates the SVD convention. Check each diagonal entry and compare successive entries; for a real diagonal \(s_1,\dots,s_q\), require \(s_j\ge0\) and \(s_j\ge s_{j+1}\) for every \(j<q\).

Changing the order of multiplication. Matrix multiplication is not commutative. The SVD reconstruction is \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}\), not \(\mathbf{U}\mathbf{V}^{*}\boldsymbol{\Sigma}\) or \(\boldsymbol{\Sigma}\mathbf{U}\mathbf{V}^{*}\). Check the dimensions of each adjacent product before multiplying; then calculate in the stated order and compare every entry with \(\mathbf{A}\).

Keeping these tests separate makes the diagnosis precise: report which condition fails, rather than simply declaring that a proposed SVD is incorrect. When every condition passes, the verification criterion establishes that the factors form an SVD and that the diagonal entries are indeed the singular values in the stated order.