From SVD Factors to Geometry
A matrix acts on every point of the unit sphere in its input space. The singular value decomposition makes the resulting shape visible: one orthogonal change of coordinates selects special input directions, the diagonal singular values stretch or collapse those directions, and a second orthogonal change places the resulting axes in the output space. The image is an ellipsoid, possibly confined to a lower-dimensional subspace or filled rather than just its boundary.
Throughout, take \(\mathbf{A}\in\mathbb{R}^{m\times n}\). The unit sphere in \(\mathbb{R}^n\) is the set of vectors \(\mathbf{x}\) satisfying \(\mathbf{x}^{\top}\mathbf{x}=1\). Use a full real SVD \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\), where \(\mathbf{U}\in\mathbb{R}^{m\times m}\) and \(\mathbf{V}\in\mathbb{R}^{n\times n}\) are orthogonal, and \(\boldsymbol{\Sigma}\in\mathbb{R}^{m\times n}\) has nonnegative singular values on its diagonal. Write \(p=\min(m,n)\), order the singular values as \(\sigma_1\ge\cdots\ge\sigma_p\ge0\), and let \(r\) be the number of positive singular values. Denote the columns of \(\mathbf{U}\) by \(\mathbf{u}_i\) and those of \(\mathbf{V}\) by \(\mathbf{v}_j\).
Orthogonal matrices preserve lengths: for example, if \(\mathbf{z}=\mathbf{V}^{\top}\mathbf{x}\), then \(\mathbf{z}^{\top}\mathbf{z}=\mathbf{x}^{\top}\mathbf{V}\mathbf{V}^{\top}\mathbf{x}=\mathbf{x}^{\top}\mathbf{x}\). Thus \(\mathbf{V}^{\top}\) only changes the coordinates used to describe an input; it does not change whether the input is on the unit sphere. The diagonal factor \(\boldsymbol{\Sigma}\) then scales the coordinates, and \(\mathbf{U}\) expresses the output in the output directions.
Principal Directions and Semiaxis Lengths
Applying \(\mathbf{A}\) to an input direction \(\mathbf{v}_j\) gives the basic geometric action. The SVD says that each positive singular value scales its corresponding input direction and sends it to its corresponding output direction. Any input direction belonging to a zero singular value is collapsed to zero.
Proof. Since \(\mathbf{V}\) is orthogonal, \(\mathbf{V}^{\top}\mathbf{v}_i=\mathbf{e}_i\), where \(\mathbf{e}_i\) is the \(i\)th standard coordinate vector. Therefore \(\mathbf{A}\mathbf{v}_i=\mathbf{U}\boldsymbol{\Sigma}\mathbf{e}_i\). For \(i\le p\), the vector \(\boldsymbol{\Sigma}\mathbf{e}_i\) has entry \(\sigma_i\) in position \(i\) and zero in every other position, so multiplication by \(\mathbf{U}\) gives \(\sigma_i\mathbf{u}_i\). If \(n>m\) and \(j>m\), the \(j\)th column of the \(m\times n\) matrix \(\boldsymbol{\Sigma}\) is zero, so \(\boldsymbol{\Sigma}\mathbf{e}_j=\mathbf{0}\), and hence \(\mathbf{A}\mathbf{v}_j=\mathbf{0}\). When \(\sigma_i=0\), the same formula gives \(\mathbf{A}\mathbf{v}_i=\mathbf{0}\). For \(\sigma_i>0\), the output is a positive scalar multiple of the unit vector \(\mathbf{u}_i\), with length \(\sigma_i\). \(\square\)
Thus \(\sigma_i\) is the semiaxis length in the output direction \(\mathbf{u}_i\), and \(\mathbf{v}_i\) is the input direction that reaches the endpoint \(\sigma_i\mathbf{u}_i\). The singular values are listed in decreasing order, so the largest one gives the longest semiaxis. A zero singular value gives a collapsed axis, not an axis of positive length.
Worked Example: A Rotated Input and Output
Consider the full SVD factors
Here \(\mathbf{U}\) is \(3\times3\), \(\boldsymbol{\Sigma}\) is \(3\times2\), and \(\mathbf{V}\) is \(2\times2\). First multiply the left two factors, then include \(\mathbf{V}^{\top}\):
The first two columns of \(\mathbf{U}\) are \(\mathbf{u}_1=(0,1,0)^{\top}\) and \(\mathbf{u}_2=(1,0,0)^{\top}\); the columns of \(\mathbf{V}\) are \(\mathbf{v}_1=(0,1)^{\top}\) and \(\mathbf{v}_2=(-1,0)^{\top}\). Checking the action directly with the displayed \(\mathbf{A}\),
An input \(\mathbf{x}=(x_1,x_2)^{\top}\) on the unit circle therefore maps to \((-2x_1,4x_2,0)^{\top}\). Its image lies in the first two coordinate directions of \(\mathbb{R}^3\), with semiaxis length \(2\) along \((1,0,0)^{\top}\) and length \(4\) along \((0,1,0)^{\top}\). The check above confirms the endpoints: the unit inputs \(\mathbf{v}_1,\mathbf{v}_2\) reach \(4\mathbf{u}_1,2\mathbf{u}_2\), respectively.
When the Image Is a Surface or a Filled Ellipsoid
The image has a boundary-only description when every input coordinate is scaled by a positive singular value. If at least one input direction is collapsed, the original sphere can use that direction to make up for input length while leaving the output unchanged. As a result, interior points of the ellipsoid can also be reached.
Proof. Let \(\mathbf{x}\) be any unit input and set \(\mathbf{z}=\mathbf{V}^{\top}\mathbf{x}\), so \(\mathbf{z}^{\top}\mathbf{z}=1\). Since only the first \(r\) singular values are positive, \(\mathbf{A}\mathbf{x}=\sum_{i=1}^{r}\sigma_i z_i\mathbf{u}_i\). The sum \(z_1^2+\cdots+z_r^2\) equals \(1\) if \(r=n\); if \(r<n\), it is at most \(1\), because the remaining squared coordinates are nonnegative. This proves that every image point has the claimed form. Conversely, take coefficients \(t_1,\ldots,t_r\) satisfying the relevant condition. If \(r=n\), set \(z_i=t_i\); then \(\mathbf{z}^{\top}\mathbf{z}=1\). If \(r<n\), choose an index \(j>r\) among the \(n\) input coordinates, set \(z_i=t_i\) for \(1\le i\le r\), set \(z_j=\sqrt{1-\sum_{i=1}^{r}t_i^2}\), and set every other coordinate to zero. The square root is defined because the sum is at most \(1\), and the resulting \(\mathbf{z}\) has \(\mathbf{z}^{\top}\mathbf{z}=1\). In either case \(\mathbf{x}=\mathbf{V}\mathbf{z}\) is a unit input and maps to the stated output. If \(r=0\), every singular value is zero and \(\mathbf{A}\mathbf{x}=\mathbf{0}\) for every input, so the image is exactly \(\{\mathbf{0}\}\). \(\square\)
When \(r<n\), the image is a filled ellipsoid in the \(r\)-dimensional subspace spanned by \(\mathbf{u}_1,\ldots,\mathbf{u}_r\). In its output coordinates \(s_i=\mathbf{u}_i^{\top}\mathbf{y}\), it is described by \(\sum_{i=1}^{r}s_i^2/\sigma_i^2\le1\). When \(r=n\), the image is only the ellipsoid boundary, described by the same equation with equality. In either case, outputs have zero coordinates along \(\mathbf{u}_{r+1},\ldots,\mathbf{u}_m\), when those vectors exist. If \(r=0\), there are no positive-length axes at all.
Worked Example: A Collapsed Direction Fills the Ellipse
Take \(\mathbf{A}=\operatorname{diag}(3,1,0)\in\mathbb{R}^{3\times3}\). This is already an SVD with \(\mathbf{U}=\mathbf{V}=\mathbf{I}_3\) and singular values \(3,1,0\). A unit input \(\mathbf{x}=(x_1,x_2,x_3)^{\top}\) satisfies \(x_1^2+x_2^2+x_3^2=1\), and its output is \(\mathbf{y}=(3x_1,x_2,0)^{\top}\). Therefore every output satisfies \(y_1^2/9+y_2^2=x_1^2+x_2^2\le1\) and \(y_3=0\).
This inequality describes the filled ellipse in the plane \(y_3=0\), not just its boundary. For example, let \(\mathbf{y}=(3/2,1/2,0)^{\top}\). Then \(y_1^2/9+y_2^2=1/4+1/4=1/2\). To verify that this interior point is reached, choose \(\mathbf{x}=(1/2,1/2,1/\sqrt{2})^{\top}\). Its squared length is \(1/4+1/4+1/2=1\), and
The input’s third coordinate uses the remaining length without changing the output. The semiaxis endpoints also check the lengths: \(\mathbf{A}\mathbf{e}_1=3\mathbf{e}_1\) and \(\mathbf{A}\mathbf{e}_2=\mathbf{e}_2\). The third direction \(\mathbf{e}_3\) is collapsed, since \(\mathbf{A}\mathbf{e}_3=\mathbf{0}\).
Worked Example: A Wide Matrix Maps onto a Filled Ellipse
Consider \(\mathbf{A}=\begin{bmatrix}2&0&0\\0&1&0\end{bmatrix}\), with \(\mathbf{U}=\mathbf{I}_2\), \(\boldsymbol{\Sigma}=\begin{bmatrix}2&0&0\\0&1&0\end{bmatrix}\), and \(\mathbf{V}=\mathbf{I}_3\). Multiplying gives \(\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}=\mathbf{A}\), so these factors do give the stated SVD. For a unit input \(\mathbf{x}=(x_1,x_2,x_3)^{\top}\), the output is \(\mathbf{y}=(2x_1,x_2)^{\top}\), and \(y_1^2/4+y_2^2=x_1^2+x_2^2\le1\).
Every point satisfying this inequality is attained: given \((y_1,y_2)^{\top}\) with \(y_1^2/4+y_2^2\le1\), set \(x_1=y_1/2\), \(x_2=y_2\), and \(x_3=\sqrt{1-y_1^2/4-y_2^2}\). Then \(\mathbf{x}^{\top}\mathbf{x}=1\) and \(\mathbf{A}\mathbf{x}=\mathbf{y}\). For a direct check, \(\mathbf{y}=(1,1/2)^{\top}\) gives \(x_3=1/2\), so \(\mathbf{x}=(1/2,1/2,1/2)^{\top}\) is unit and \(\mathbf{A}\mathbf{x}=(1,1/2)^{\top}\). The two semiaxis lengths are \(2\) and \(1\), and the extra input direction is collapsed.
A Common Pitfall: Boundary Equality Is Not Universal
A frequent error is to write an equality for the image of every unit sphere, even when a zero singular value leaves an input direction available to adjust the length. In the \(\operatorname{diag}(3,1,0)\) example, the unit input \((0,0,1)^{\top}\) maps to \(\mathbf{0}\). The incorrect equation \(y_1^2/9+y_2^2=1\) rejects this output, since its left side is \(0\). The correction is the inequality \(y_1^2/9+y_2^2\le1\), together with \(y_3=0\). Equality describes the boundary; the inequality describes the filled ellipsoid.
Also keep the input and output directions distinct. The right singular vector \(\mathbf{v}_i\) is an input direction, while the corresponding output axis points along the left singular vector \(\mathbf{u}_i\). It is \(\mathbf{A}\mathbf{v}_i=\sigma_i\mathbf{u}_i\), not generally \(\sigma_i\mathbf{v}_i\). If singular values repeat, the SVD may allow different orthonormal choices within the corresponding directions; the lengths remain the same, even when a particular choice of axes is not unique.
Why the Ellipsoid Description Is Useful
The geometric reading of an SVD translates between three kinds of information: input directions, output directions, and stretches. It also reveals immediately whether the image is full-dimensional in the output space, lies in a lower-dimensional subspace, or collapses entirely. For a unit sphere in an \(n\)-dimensional input space, the key distinction is whether any input direction is collapsed: if there is one, the image is filled; if every input direction has a positive singular value, the image is the boundary. These conclusions come directly from the coordinate representation and do not require guessing the shape from a plotted matrix action.
Check Your Understanding
Use the SVD factors and the unit-sphere condition to justify each answer.
- If \(\mathbf{A}\) is \(3\times3\) with singular values \(5,2,1\), what are the lengths and output directions of its principal semiaxes? Is the unit-sphere image a surface or a filled ellipsoid?
- For \(\mathbf{A}=\operatorname{diag}(4,2,0)\), describe the image of the unit sphere and state the plane in which it lies.
- Explain why a zero singular value allows interior points of the ellipsoid to be reached.
- For a full SVD, derive \(\mathbf{A}\mathbf{v}_i=\sigma_i\mathbf{u}_i\) from \(\mathbf{A}=\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\).
- Suppose \(n>m\) and all \(m\) singular values are positive. Is the image of the unit sphere a boundary or a filled ellipsoid? Explain using the unused input directions.
