Rank, Support, and Degeneracy Under Gaussian Linear Maps

Why Rank Determines the Shape of a Gaussian Law

A Gaussian vector need not spread probability across every direction in its ambient space. A linear transformation may create exact relationships among its coordinates, so the transformed vector lies in a line or plane even when it is written as a vector in a higher-dimensional space. The covariance matrix records which directions vary, but its rank alone does not show where the vector lies. To describe the distribution fully, we connect that rank to the support subspace and to deterministic linear constraints.

Let \(X\) be a Gaussian vector in \(\mathbb{R}^p\) with mean \(\mu\) and positive semidefinite covariance matrix \(\Sigma\), and let \(A\) be an \(m\times p\) matrix. For a fixed vector \(b\in\mathbb{R}^m\), consider \(Y=AX+b\). The covariance of \(Y\) is \(C=A\Sigma A^T\). This tutorial studies what the rank and null space of \(C\) say about the distribution of \(Y\).

Definition: The support of a random vector is the set of points for which every open neighborhood has positive probability. A Gaussian distribution is degenerate in \(\mathbb{R}^m\) when its covariance matrix is singular. In that case its support is contained in a proper affine subspace of \(\mathbb{R}^m\).

The Rank-and-Support Theorem

Because \(\Sigma\) is positive semidefinite, it has a symmetric positive semidefinite square root \(\Sigma^{1/2}\), satisfying \(\Sigma^{1/2}\Sigma^{1/2}=\Sigma\). The transformed covariance can therefore be written as \(C=BB^T\), where \(B=A\Sigma^{1/2}\). This factorization makes the connection between variation and support explicit.

Theorem: Suppose \(X\sim N_p(\mu,\Sigma)\), where \(\Sigma\) is positive semidefinite, and define \(Y=AX+b\) for an \(m\times p\) matrix \(A\) and fixed \(b\in\mathbb{R}^m\). Set \(\eta=A\mu+b\) and \(C=A\Sigma A^T\). Then

\(Y\sim N_m(\eta,C)\), and \(\operatorname{rank}(C)=\operatorname{rank}(A\Sigma^{1/2})\).
The support of \(Y\) is \(\eta+\operatorname{Range}(C)\), an affine subspace of dimension \(\operatorname{rank}(C)\).
For \(c\in\mathbb{R}^m\), \(c^T(Y-\eta)=0\) almost surely if and only if \(c\in\operatorname{Null}(C)\).

Here \(\operatorname{Range}(C)\) is the set of vectors \(Cv\), and \(\operatorname{Null}(C)\) is the set of vectors \(c\) satisfying \(Cc=0\). The third claim means that every null vector of the covariance gives an exact linear constraint on \(Y\). Conversely, every deterministic linear constraint centered at the mean arises this way.

Represent the Gaussian vector.
For a standard normal vector \(Z\sim N_p(0,I_p)\), the distribution of \(X\) is the same as that of \(\mu+\Sigma^{1/2}Z\). This representation also applies when \(\Sigma\) is singular.

Transform it.
It follows that \(Y\) has the same distribution as \(\eta+BZ\), where \(B=A\Sigma^{1/2}\). Its covariance is \(BB^T=C\).

Identify the varying directions.
For any \(c\), \(c^TCc=\|B^Tc\|^2\). Thus \(Cc=0\) exactly when \(B^Tc=0\); the vectors \(c\) in this null space have no random variation in their linear contrasts of \(Y\).

For completeness, the rank equality follows because \(BB^T\) and \(B^T\) have the same null space: \(c^TBB^Tc=\|B^Tc\|^2\). Hence \(\operatorname{rank}(BB^T)=\operatorname{rank}(B)\), by the rank-nullity theorem. Also, \(\operatorname{Var}(c^T(Y-\eta))=c^TCc\). A scalar random variable with variance zero equals its mean almost surely, so \(c^TCc=0\) is equivalent to \(c^T(Y-\eta)=0\) almost surely. Since \(c^TCc=\|B^Tc\|^2\), this is equivalent to \(Cc=0\).

It remains to verify the support claim. Let \(r=\operatorname{rank}(B)\). A singular value decomposition gives \(B=UDV^T\), where the columns of \(U\) and \(V\) are orthonormal, and \(D\) has \(r\) positive diagonal entries. Therefore \(BZ=UDV^TZ\) lies in \(\operatorname{Range}(U)=\operatorname{Range}(B)\). The first \(r\) coordinates of \(V^TZ\) are independent standard normal variables, so their image under the invertible diagonal matrix \(D\) has positive density throughout \(\mathbb{R}^r\). Every neighborhood of every point in \(\operatorname{Range}(B)\) consequently has positive probability, and points outside that subspace have a neighborhood with probability zero. Finally, \(\operatorname{Range}(B)=\operatorname{Range}(BB^T)=\operatorname{Range}(C)\): both spaces have dimension \(r\), and \(\operatorname{Range}(BB^T)\subseteq\operatorname{Range}(B)\). Translating by \(\eta\) proves the support claim. \(\square\)

Reading the Support and Its Density

If \(C\) has rank \(r<m\), then \(Y\) has no density with respect to ordinary \(m\)-dimensional Lebesgue measure: its entire probability is concentrated on an affine subspace of dimension \(r\). It does have a density relative to \(r\)-dimensional volume on that support. To see its form, choose orthonormal eigenvectors \(u_1,\ldots,u_r\) of \(C\) for its positive eigenvalues \(\lambda_1,\ldots,\lambda_r\). A support point \(y\) can be written as \(y=\eta+\sum_{j=1}^r t_j u_j\). The density in these coordinates is

(2\pi)^{-r/2}\left(\prod_{j=1}^r\lambda_j\right)^{-1/2} \exp\left(-\frac{1}{2}\sum_{j=1}^r\frac{t_j^2}{\lambda_j}\right).

The coordinates \(t_j=u_j^T(y-\eta)\) are independent centered normal coordinates with variances \(\lambda_j\). The remaining \(m-r\) coordinates are fixed by the support constraint. When \(r=0\), \(C=0\) and \(Y=\eta\) almost surely; the distribution is a point mass rather than a density on a positive-dimensional space.

Worked Example: A Transformation That Forces an Exact Constraint

Let \(X=(X_1,X_2,X_3)^T\sim N_3(0,I_3)\), and define \(A=\begin{pmatrix}1&1&0\\2&2&0\\0&0&1\end{pmatrix}\) and \(Y=AX\). Find the covariance rank, support, and deterministic constraint.

The transformed vector is \(Y=(X_1+X_2,\;2X_1+2X_2,\;X_3)^T\). Its mean is zero, and its covariance is

C=AA^T= \begin{pmatrix} 2&4&0\\ 4&8&0\\ 0&0&1 \end{pmatrix}.

The first two rows are proportional, while the third contributes an additional independent direction, so \(\operatorname{rank}(C)=2\). The vector \(c=(-2,1,0)^T\) satisfies \(Cc=0\), and therefore \(c^TY=-2Y_1+Y_2=0\) almost surely. In fact this follows directly from the coordinate formula for \(Y\). The support is the plane \(\{(y_1,y_2,y_3):y_2=2y_1\}\), which has dimension two.

Worked Example: A Full-Row-Rank Map Can Still Be Degenerate

Let \(U,V\) be independent \(N(0,1)\) variables and set \(X=(U,V,U+V)^T\). Use \(A=\begin{pmatrix}1&0&0\\0&-1&1\end{pmatrix}\), which has full row rank, to form \(Y=AX\).

The covariance of \(X\) is \(\Sigma=\begin{pmatrix}1&0&1\\0&1&1\\1&1&2\end{pmatrix}\). Directly, \(Y=(U,-V+(U+V))^T=(U,U)^T\), so

C=A\Sigma A^T= \begin{pmatrix} 1&1\\ 1&1 \end{pmatrix}.

This matrix has rank one, since its second row equals its first. Its null vector \((-1,1)^T\) gives \(Y_2-Y_1=0\) almost surely, and the support is the line \(\{(y_1,y_2):y_2=y_1\}\). Full row rank of \(A\) is not enough to guarantee a nonsingular transformed covariance: here \(X\) already has a deterministic relation, and the map preserves it in the output.

Worked Example: A Singular Input Can Have a Nonsingular Output

Suppose \(X\sim N_3(\mu,\Sigma)\), where \(\mu=(0,3,2)^T\) and \(\Sigma=\operatorname{diag}(4,0,1)\). Take \(A=\begin{pmatrix}1&1&1\\0&1&2\end{pmatrix}\) and \(b=(1,-1)^T\). Find the mean and covariance of \(Y=AX+b\), and decide whether its support is all of \(\mathbb{R}^2\).

The second coordinate of \(X\) is fixed at 3 almost surely because its variance is zero. Nevertheless, the two varying input coordinates can generate two output directions. The mean and covariance are

\eta=A\mu+b= \begin{pmatrix}6\\6\end{pmatrix}, \qquad C=A\Sigma A^T= \begin{pmatrix}5&2\\2&4\end{pmatrix}.

The determinant of \(C\) is \(5\cdot4-2\cdot2=16>0\), and its first diagonal entry is positive, so \(C\) is positive definite and has rank two. Thus the support is \(\mathbb{R}^2\), despite the singular covariance of the input. Degeneracy of the input does not by itself imply degeneracy of every transformed output.

Common Mistakes and Why the Distinction Matters

Checking only the dimensions of \(A\). A matrix with full row rank can still produce a singular covariance if the input has no variation in some directions used by the map. Check \(A\Sigma A^T\), not just \(A\).
Assuming every written coordinate varies. A vector in \(\mathbb{R}^m\) may have only \(r<m\) degrees of random variation. The null space of its covariance identifies the exact constraints.
Using an ordinary multivariate density on a lower-dimensional support. When the covariance is singular, the distribution assigns probability one to a set of \(m\)-dimensional Lebesgue measure zero. Describe the law on its support instead.

These distinctions matter whenever a Gaussian model includes duplicated measurements, exact linear identities, or a covariance matrix with zero eigenvalues. The rank tells how many directions vary; the range locates those directions; and the null space states the deterministic relations among coordinates. Together they provide a precise description of the transformed distribution, including when an ordinary full-dimensional density does not exist.

Check Your Understanding

Use the rank, range, and null-space relationships to answer the following.

If \(C\) is a \(5\times5\) covariance matrix of rank 3, what is the dimension of the Gaussian vector’s support?
For a centered Gaussian vector \(Y\), why does \(Cc=0\) imply \(c^TY=0\) almost surely?
Can a full-row-rank transformation of a singular Gaussian input have a nonsingular covariance? Explain what determines the answer.
What is the support of a Gaussian vector with mean \(\eta\) and zero covariance?
Why does a degenerate Gaussian distribution not have a density with respect to ambient Lebesgue measure?

Rank, Support, and Degeneracy Under Gaussian Linear Maps

What You'll Learn