The Frobenius Norm From Singular Values

Total Matrix Size and the Frobenius Norm

A matrix can be large in several different senses: it can stretch some vectors strongly, or it can have many entries that contribute modestly. The Frobenius norm records the combined size of all its entries. The singular values provide a second way to measure that same total, even when the matrix is rectangular or has complex entries. The link between these descriptions is useful both for calculating a norm and for checking an SVD.

Let \(\mathbf{A}=(a_{ij})\in\mathbb{C}^{m\times n}\), where \(m,n\in\mathbb{N}\). The real case is included by taking all entries to be real. The absolute value of a complex number \(z\) is denoted by \(|z|\), so \(|z|^2=z\overline{z}\). The Frobenius norm squares the absolute value of each entry, adds those squares, and then takes the nonnegative square root.

For an \(m\times n\) matrix, write \(p=\min(m,n)\). Its singular values are the nonnegative square roots of the eigenvalues of \(\mathbf{A}^{*}\mathbf{A}\), counted with multiplicity and arranged in nonincreasing order. There are \(p\) singular values when zeros are included; any additional eigenvalues of \(\mathbf{A}^{*}\mathbf{A}\) beyond those \(p\) are also zero. Equivalently, a full SVD writes \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}\), where \(\mathbf{U}\) and \(\mathbf{V}\) are unitary and the diagonal entries of the rectangular matrix \(\boldsymbol{\Sigma}\), up to \(p\), are the singular values. In the real case, the unitary matrices are orthogonal and the conjugate transpose is the transpose.

The Frobenius Norm Identity

Proof. For a square matrix \(\mathbf{B}\), its trace \(\operatorname{tr}\mathbf{B}\) is the sum of its diagonal entries. First, the diagonal entry in position \(j\) of \(\mathbf{A}^{*}\mathbf{A}\) is \(\sum_{i=1}^{m}\overline{a_{ij}}a_{ij}=\sum_{i=1}^{m}|a_{ij}|^2\). Adding these diagonal entries gives \(\operatorname{tr}(\mathbf{A}^{*}\mathbf{A})=\sum_{j=1}^{n}\sum_{i=1}^{m}|a_{ij}|^2=\|\mathbf{A}\|_F^2\).

Now use a full SVD \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{*}\), with \(\mathbf{U}^{*}\mathbf{U}=\mathbf{I}_m\) and \(\mathbf{V}^{*}\mathbf{V}=\mathbf{I}_n\). Then \(\mathbf{A}^{*}\mathbf{A}=\mathbf{V}\boldsymbol{\Sigma}^{*}\boldsymbol{\Sigma}\mathbf{V}^{*}\). For compatible matrices \(\mathbf{X}\) and \(\mathbf{Y}\), the identity \(\operatorname{tr}(\mathbf{X}\mathbf{Y})=\operatorname{tr}(\mathbf{Y}\mathbf{X})\) follows by writing either side as \(\sum_i\sum_j x_{ij}y_{ji}\); both sums contain the same terms. Apply this identity to the product above, using \(\mathbf{V}^{*}\mathbf{V}=\mathbf{I}_n\), to obtain \(\operatorname{tr}(\mathbf{A}^{*}\mathbf{A})=\operatorname{tr}(\boldsymbol{\Sigma}^{*}\boldsymbol{\Sigma})\). The diagonal entries of \(\boldsymbol{\Sigma}^{*}\boldsymbol{\Sigma}\) are \(\sigma_1^2,\dots,\sigma_p^2\), followed, if needed, by zeros. Its trace is therefore \(\sum_{k=1}^{p}\sigma_k^2\). Combining the two trace calculations proves the squared identity. Both sides are nonnegative, so taking their nonnegative square roots gives the norm formula. \(\square\)

The theorem uses the squares of the singular values, not their sum. It also applies without requiring \(\mathbf{A}\) to be square or invertible: zero singular values contribute zero, so rank deficiency causes no exception.

Reading the Identity from a Rectangular Matrix

This example illustrates why a rectangular matrix can have zero singular values even when its displayed Gram matrix has more eigenvalues than there are singular values in the list of length \(p\). Including or omitting extra zeros does not change the sum of squares.

Complex Entries and Absolute Values

For complex matrices, replacing \(|a_{ij}|^2\) by \(a_{ij}^2\) is not valid: for example, \(i^2=-1\), whereas \(|i|^2=1\). The conjugation in \(\mathbf{A}^{*}\mathbf{A}\) is what makes its diagonal entries nonnegative and makes the entrywise formula agree with the singular values.

A Square Example and a Useful Entrywise Check

There is also a direct way to organize the entrywise sum. If \(\mathbf{a}_1,\dots,\mathbf{a}_n\) are the columns of \(\mathbf{A}\), then each column contributes the squared lengths of its entries. In particular, this formula can help catch an omitted entry or an incorrect conjugate.

Proof. By the definition of the Frobenius norm, \(\|\mathbf{A}\|_F^2=\sum_{i=1}^{m}\sum_{j=1}^{n}|a_{ij}|^2\). Reordering this finite sum by columns gives \(\sum_{j=1}^{n}\left(\sum_{i=1}^{m}|a_{ij}|^2\right)\). The inner sum is \(\|\mathbf{a}_j\|^2\) by the stated definition of vector length, so the result follows. \(\square\)

Common Mistake: Adding Singular Values

The Frobenius norm is not the sum of the singular values. For example, take \(\mathbf{D}=\begin{bmatrix}3&0\\0&4\end{bmatrix}\). Its singular values are \(4\) and \(3\), since \(\mathbf{D}^{*}\mathbf{D}=\begin{bmatrix}9&0\\0&16\end{bmatrix}\) has eigenvalues \(16\) and \(9\). The incorrect calculation \(\|\mathbf{D}\|_F=4+3=7\) adds the singular values instead of combining their squares.

The correct calculation is \(\|\mathbf{D}\|_F^2=|3|^2+|0|^2+|0|^2+|4|^2=9+16=25\), so \(\|\mathbf{D}\|_F=\sqrt{25}=5\). Using the singular values gives the same answer: \(\sqrt{4^2+3^2}=\sqrt{16+9}=5\). A related error is to report \(25\) as the norm: \(25\) is the squared Frobenius norm, while the Frobenius norm itself is \(5\).

Why the Identity Is Useful

The entry formula is often convenient when a matrix is already written out. The singular-value formula is convenient when an SVD or the eigenvalues of \(\mathbf{A}^{*}\mathbf{A}\) are already known. Agreement between these calculations gives a quick consistency check: a discrepancy signals an error in the entrywise sum, the Gram matrix, or the singular values. For a matrix with many entries, the singular-value expression also summarizes its total squared size using a list of nonnegative numbers.

When checking an answer, keep three details in view: use absolute-value squares for complex entries; include zero singular values if needed to make a list of length \(\min(m,n)\); and take a square root only when moving from the squared norm to the norm itself. These distinctions make the same identity reliable across square, rectangular, real, and complex matrices.