From an SVD to a Sum
An SVD packages a matrix into orthonormal vectors and nonnegative singular values. The same information can be read one singular value at a time: each positive singular value scales a single outer product, and adding those contributions recovers the original matrix. This viewpoint is useful for understanding what each term contributes and for checking a proposed reconstruction without losing track of matrix dimensions.
Throughout, let \(\mathbf{A}\in\mathbb{R}^{m\times n}\). A real SVD writes \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\), where \(\mathbf{U}\) and \(\mathbf{V}\) are orthogonal matrices and \(\boldsymbol{\Sigma}\) is diagonal apart from any rectangular padding. Write \(\mathbf{u}_i\) for the relevant columns of \(\mathbf{U}\), \(\mathbf{v}_i\) for the corresponding columns of \(\mathbf{V}\), and \(\sigma_i\geq 0\) for the corresponding diagonal entries. The vectors are columns; consequently, \(\mathbf{u}_i\mathbf{v}_i^{\top}\) is an \(m\times n\) matrix.
Thus \(\sigma_i\mathbf{u}_i\mathbf{v}_i^{\top}\) is an \(m\times n\) contribution. When \(\sigma_i>0\), it has rank one; when \(\sigma_i=0\), it is the zero matrix. The expansion is simply the SVD matrix product written as the sum of these contributions.
Proof. In the full SVD, the rectangular diagonal matrix can be written as \(\boldsymbol{\Sigma}=\sum_{i=1}^{k}\sigma_i\mathbf{e}_i\mathbf{e}_i^{\top}\), with the standard coordinate vectors interpreted in the corresponding dimensions. Multiplying on the left by \(\mathbf{U}\) and on the right by \(\mathbf{V}^{\top}\) gives \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}=\sum_{i=1}^{k}\sigma_i(\mathbf{U}\mathbf{e}_i)(\mathbf{V}\mathbf{e}_i)^{\top}\). By the definition of matrix columns, \(\mathbf{U}\mathbf{e}_i=\mathbf{u}_i\) and \(\mathbf{V}\mathbf{e}_i=\mathbf{v}_i\); the SVD identity therefore gives the full expansion. Terms with \(\sigma_i=0\) vanish, so removing them leaves the compact sum. If there are no positive singular values, every full-SVD term is zero, and hence \(\mathbf{A}=\mathbf{O}\), as stated for the empty sum. \(\square\)
Reading Each Contribution
The outer product makes the direction and scale of a term explicit. For any \(\mathbf{x}\in\mathbb{R}^n\), its action on \(\mathbf{x}\) is \(\sigma_i\mathbf{u}_i\mathbf{v}_i^{\top}\mathbf{x}=\sigma_i\mathbf{u}_i(\mathbf{v}_i^{\top}\mathbf{x})\). The scalar \(\mathbf{v}_i^{\top}\mathbf{x}\) is multiplied by \(\sigma_i\), and the result points along \(\mathbf{u}_i\). Entry by entry, the same term contributes \(\sigma_i u_{i,j}v_{i,k}\) to entry \((j,k)\). These are two ways to read the same matrix contribution.
Proof. By the definition of transpose and matrix multiplication, the entry in row \(j\), column \(k\) of \(\mathbf{u}\mathbf{v}^{\top}\) is \(u_jv_k\); multiplication by \(\sigma\) makes it \(\sigma u_jv_k\). For the action on \(\mathbf{x}\), associativity of matrix multiplication gives \((\sigma\mathbf{u}\mathbf{v}^{\top})\mathbf{x}=\sigma\mathbf{u}(\mathbf{v}^{\top}\mathbf{x})\). Since \(\mathbf{v}^{\top}\mathbf{x}\) is a scalar, this is the claimed scalar multiple of \(\mathbf{u}\). \(\square\)
In a compact SVD, the columns \(\mathbf{u}_i\) are orthonormal, as are the columns \(\mathbf{v}_i\), and each pair is associated with the same \(\sigma_i\). The expansion does not pair a left vector with an arbitrary right vector: the index \(i\) must match across all three factors. That pairing is essential for the product to reconstruct \(\mathbf{A}\).
Worked Reconstructions
Worked Example: A Symmetric Two-By-Two Matrix
Use the SVD data \(\sigma_1=3\), \(\sigma_2=1\), \(\mathbf{u}_1=\mathbf{v}_1=\tfrac{1}{\sqrt{2}}(1,1)^{\top}\), and \(\mathbf{u}_2=\mathbf{v}_2=\tfrac{1}{\sqrt{2}}(1,-1)^{\top}\) to reconstruct \(\mathbf{A}\).
First form the two outer products. For the first pair, each entry is a product of the corresponding coordinates:
For the second pair, the negative coordinates give negative off-diagonal entries:
Add corresponding entries of the two contributions:
Check: The two displayed vectors form the columns of \(\mathbf{U}=\mathbf{V}=\tfrac{1}{\sqrt{2}}\begin{bmatrix}1&1\\1&-1\end{bmatrix}\). Then \(\mathbf{U}\boldsymbol{\Sigma}=\tfrac{1}{\sqrt{2}}\begin{bmatrix}3&1\\3&-1\end{bmatrix}\), where \(\boldsymbol{\Sigma}=\begin{bmatrix}3&0\\0&1\end{bmatrix}\). Multiplying by \(\mathbf{V}^{\top}\) gives \(\tfrac12\begin{bmatrix}3&1\\3&-1\end{bmatrix}\begin{bmatrix}1&1\\1&-1\end{bmatrix}=\begin{bmatrix}2&1\\1&2\end{bmatrix}\), agreeing with the sum.
Worked Example: A Rectangular Matrix
Let \(\mathbf{u}_1=(1,0)^{\top}\), \(\mathbf{u}_2=(0,1)^{\top}\), \(\mathbf{v}_1=\tfrac{1}{\sqrt{2}}(1,0,1)^{\top}\), \(\mathbf{v}_2=(0,1,0)^{\top}\), and let \(\sigma_1=2,\ \sigma_2=1\). These left and right vectors are each orthonormal lists. Compute the \(2\times3\) matrix given by the two terms.
The first outer product and its scaling are
The second term is
Adding the matrices gives \(\mathbf{A}=\begin{bmatrix}\sqrt{2}&0&\sqrt{2}\\0&1&0\end{bmatrix}\). The different row and column counts are consistent: each outer product is \(2\times3\), so their sum is \(2\times3\).
Check: Here \(\mathbf{U}=\mathbf{I}_2\), \(\boldsymbol{\Sigma}=\begin{bmatrix}2&0\\0&1\end{bmatrix}\), and \(\mathbf{V}^{\top}=\begin{bmatrix}\frac{1}{\sqrt{2}}&0&\frac{1}{\sqrt{2}}\\0&1&0\end{bmatrix}\). Thus \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top} =\begin{bmatrix}2&0\\0&1\end{bmatrix} \begin{bmatrix}\frac{1}{\sqrt{2}}&0&\frac{1}{\sqrt{2}}\\0&1&0\end{bmatrix} =\begin{bmatrix}\sqrt{2}&0&\sqrt{2}\\0&1&0\end{bmatrix}\), as reconstructed.
Worked Example: A Zero Singular Value
Consider the full SVD data \(\mathbf{u}_1=(\tfrac35,\tfrac45)^{\top}\), \(\mathbf{u}_2=(-\tfrac45,\tfrac35)^{\top}\), \(\mathbf{v}_1=(1,0)^{\top}\), \(\mathbf{v}_2=(0,1)^{\top}\), with \(\sigma_1=5,\ \sigma_2=0\). The two lists are orthonormal. Reconstruct the matrix and explain what happens to the second term.
The first contribution is \(5\mathbf{u}_1\mathbf{v}_1^{\top} =5\begin{bmatrix}\frac35\\\frac45\end{bmatrix}\begin{bmatrix}1&0\end{bmatrix} =\begin{bmatrix}3&0\\4&0\end{bmatrix}\). The second is \(0\mathbf{u}_2\mathbf{v}_2^{\top} =\mathbf{O}\). Therefore \(\mathbf{A}=\begin{bmatrix}3&0\\4&0\end{bmatrix}\).
Check: With \(\mathbf{U}=\begin{bmatrix}\frac35&-\frac45\\\frac45&\frac35\end{bmatrix}\), \(\boldsymbol{\Sigma}=\begin{bmatrix}5&0\\0&0\end{bmatrix}\), and \(\mathbf{V}=\mathbf{I}_2\), we have \(\mathbf{U}\boldsymbol{\Sigma}=\begin{bmatrix}3&0\\4&0\end{bmatrix}\). Multiplication by \(\mathbf{V}^{\top}=\mathbf{I}_2\) leaves that matrix unchanged. Omitting the zero term therefore changes nothing.
A Reliable Calculation and a Common Pitfall
For a supplied SVD, a careful reconstruction follows the same sequence every time. First check that each \(\mathbf{u}_i\) has \(m\) entries and each \(\mathbf{v}_i\) has \(n\) entries. Next transpose the right vector, form the outer product, and multiply every entry by its singular value. Finally add the resulting \(m\times n\) matrices entry by entry. If a singular value is zero, its entire contribution is zero and can be left out of the addition.
Use \(\sigma_i,\mathbf{u}_i,\mathbf{v}_i\) with the same index \(i\).
Calculate \(\sigma_i\mathbf{u}_i\mathbf{v}_i^{\top}\), checking that its shape is \(m\times n\).
Sum the contributions, then compare with \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\) or check every entry.
Common mistake: reversing the outer product. In the rectangular example, a tempting but incorrect first term is \(2\mathbf{v}_1\mathbf{u}_1^{\top}\). Since \(\mathbf{v}_1\) has three entries and \(\mathbf{u}_1\) has two, this product is \(3\times2\), not \(2\times3\). It cannot be added to \(\mathbf{A}\). The correct order is \(2\mathbf{u}_1\mathbf{v}_1^{\top}\), which has two rows and three columns. A useful check before calculating entries is therefore to confirm that the left vector comes first and the transposed right vector comes second.
Another error is to multiply the entries of \(\mathbf{u}_i\) and \(\mathbf{v}_i\) coordinate by coordinate. That operation does not form an \(m\times n\) matrix. The outer product instead pairs every coordinate of \(\mathbf{u}_i\) with every coordinate of \(\mathbf{v}_i\), producing all \(mn\) entries. Scaling and then adding these matrices is precisely the matrix multiplication encoded by the SVD.
What the Expansion Keeps Visible
The decomposition makes three features of a reconstruction explicit: the left direction of each contribution, its right direction, and its scale. It also works without alteration for rectangular matrices and for SVDs containing zero singular values. The expansion is an exact identity, not an approximation: retaining all nonzero terms recovers \(\mathbf{A}\), while any zero terms contribute nothing. Approximation questions involving selected terms belong to a separate topic; here the essential task is to form and add the terms correctly.
Check Your Understanding
For each question, keep track of vector dimensions and use the matching index for every singular value and vector pair.
- If \(\mathbf{u}\in\mathbb{R}^3\) and \(\mathbf{v}\in\mathbb{R}^2\), what are the dimensions of \(\mathbf{u}\mathbf{v}^{\top}\), and what is its entry in position \((2,1)\)?
- Write the two outer-product terms in the full SVD expansion of a \(2\times2\) matrix with singular values \(\sigma_1,\sigma_2\).
- For \(\mathbf{u}=(1,2)^{\top}\), \(\mathbf{v}=(3,-1)^{\top}\), and \(\sigma=2\), calculate \(\sigma\mathbf{u}\mathbf{v}^{\top}\).
- Why can terms with zero singular value be omitted from the reconstruction sum?
- For a \(3\times2\) matrix, what dimensions must \(\mathbf{u}_i\) and \(\mathbf{v}_i\) have so that \(\mathbf{u}_i\mathbf{v}_i^{\top}\) has the same shape as the matrix?
