When a Multivariate Law Has No Ambient Density
A random vector can have continuously varying coordinates and still fail to possess a density on the full space. The reason may be geometric: its possible values can be confined to a curve or a hyperplane, even though that set occupies zero volume in the ambient space. This matters whenever a calculation assumes a joint density with respect to ordinary area or volume.
The key distinction is between the dimension of the space in which a vector is recorded and the dimension of the set on which it actually lies. A vector in \(\mathbb{R}^2\) may be determined by one random quantity and therefore lie on a curve. A vector in \(\mathbb{R}^3\) may satisfy an exact linear equation and lie on a plane. In either case, the ambient Lebesgue measure of the support can be zero, despite the vector assigning that support probability one.
A simple consequence of these definitions is that a law concentrated on a null set cannot have an ambient density. If \(P(X\in A)=1\) and \(A\) has Lebesgue measure zero, a density \(f\) would instead give \(P(X\in A)=\int_A f(x)\,dx=0\). The contradiction rules out such a density. This conclusion concerns density with respect to ambient volume; it does not say the random vector cannot be described by a density for a lower-dimensional parameter.
Worked Examples of Lower-Dimensional Support
Worked Example: A Random Vector on a Line
Let \(Z\sim N(0,1)\) and define \(X=(X_1,X_2)=(Z,2Z+1)\). Find its support constraint and covariance matrix, and determine whether \(X\) has a density on \(\mathbb{R}^2\).
Every possible value satisfies \(X_2=2X_1+1\), so \(X\) is concentrated on the line \(A=\{(x_1,x_2):x_2=2x_1+1\}\). This line has two-dimensional Lebesgue measure zero: for each fixed \(x_1\), its vertical slice contains just one value of \(x_2\), and hence has one-dimensional measure zero. Fubini's theorem then gives zero area for the line.
The means are \(E[X_1]=0\) and \(E[X_2]=1\). Since adding a constant does not change variance or covariance, \(\operatorname{Var}(X_1)=1\), \(\operatorname{Cov}(X_1,X_2)=\operatorname{Cov}(Z,2Z+1)=2\), and \(\operatorname{Var}(X_2)=4\). Thus
The law assigns probability one to a zero-area line, so it has no density with respect to area on \(\mathbb{R}^2\). Its covariance is also singular, in agreement with the exact linear constraint.
Worked Example: A Vector on a Hyperplane
Let \(U\) and \(V\) be independent \(U(0,1)\) random variables and set \(X=(U,V,U+V)\). Show that this vector has no density on \(\mathbb{R}^3\), and calculate its covariance matrix.
The coordinates always obey \(X_3=X_1+X_2\), so the vector is concentrated on a plane in \(\mathbb{R}^3\). A plane is a three-dimensional Lebesgue-null set: after solving its equation for one coordinate, each vertical slice in that coordinate is a singleton, so Fubini's theorem gives zero volume. Because the plane has probability one, \(X\) cannot have a density with respect to three-dimensional volume.
A \(U(0,1)\) variable has mean \(1/2\) and variance \(1/12\). Independence gives \(\operatorname{Cov}(U,V)=0\). The covariance entries involving \(U+V\) are \(\operatorname{Cov}(U,U+V)=1/12\), \(\operatorname{Cov}(V,U+V)=1/12\), and \(\operatorname{Var}(U+V)=1/12+1/12=1/6\). Therefore
The third row is the sum of the first two rows, so this matrix is singular. Equivalently, the coefficient vector \((-1,-1,1)^\mathsf{T}\) gives the zero-variance combination \(-X_1-X_2+X_3=0\).
Worked Example: A Curved Support with Nonsingular Covariance
Let \(U\sim U(0,1)\) and set \(X=(U,U^2)\). Show that \(X\) has no ambient density, but its covariance matrix is nonsingular.
The vector lies on the parabola \(A=\{(u,u^2):0\leq u\leq1\}\). This graph has area zero: each vertical slice at a fixed first coordinate contains at most one point, so Fubini's theorem gives two-dimensional Lebesgue measure zero. Since \(P(X\in A)=1\), the law has no density with respect to area.
Using \(E[U^k]=1/(k+1)\) for \(k\geq0\), we obtain \(\operatorname{Var}(U)=1/3-1/4=1/12\), \(\operatorname{Cov}(U,U^2)=1/4-(1/2)(1/3)=1/12\), and \(\operatorname{Var}(U^2)=1/5-1/9=4/45\). Hence
Thus the covariance is nonsingular even though the law is concentrated on a curve. The curve restriction is nonlinear; there is no nonzero linear combination of the two coordinates that is constant almost surely.
Linear Constraints and Singular Covariance
The line and plane examples illustrate a general equivalence. It applies to any random vector with finite second moments and characterizes precisely when its covariance matrix is singular. It does not characterize every law without an ambient density: the parabola example already shows why.
Proof. A covariance matrix is symmetric and positive semidefinite. If \(\Sigma\) is singular, finite-dimensional linear algebra gives a nonzero vector \(a\) with \(\Sigma a=0\). The variance of the scalar random variable \(a^\mathsf{T}X\) is
A finite-variance scalar random variable with variance zero equals its expectation almost surely: indeed, \(E[(a^\mathsf{T}X-a^\mathsf{T}\mu)^2]=0\), and a nonnegative random variable with expectation zero is zero almost surely. Therefore \(a^\mathsf{T}X=a^\mathsf{T}\mu\) almost surely.
Conversely, suppose a nonzero \(a\) satisfies that relation almost surely. Then \(a^\mathsf{T}(X-\mu)=0\) almost surely, and
Thus \(\Sigma\) has a nonzero null vector and is singular. \(\square\)
The equation \(a^\mathsf{T}X=a^\mathsf{T}\mu\) describes an affine hyperplane when \(a\ne0\). The theorem therefore says that singular covariance is exactly the presence of an almost-sure affine constraint. In particular, such a vector lies in a Lebesgue-null hyperplane in \(\mathbb{R}^d\) and cannot have an ambient density. The reverse implication does not hold: concentration on a nonlinear curve need not create a zero-variance linear combination.
Why the Distinction Matters
A full-dimensional density model assumes that probabilities are calculated by integrating over \(d\)-dimensional volume. For a law concentrated on a lower-dimensional set, that approach cannot describe the law: every such set has zero ambient volume even when it has probability one. One must instead describe the random quantities that generate the vector, or use a measure defined on the lower-dimensional support.
The covariance matrix provides a useful diagnostic for linear degeneracy. A nonzero vector \(a\) with \(a^\mathsf{T}\Sigma a=0\) signals an exact linear relation, not merely a strong association or a visually narrow cloud. But a nonsingular covariance matrix does not guarantee a full-dimensional density, as the parabola example demonstrates. Covariance records second-order linear structure; it does not determine the geometry of the entire probability law.
Check Your Understanding
Use the definitions and results above to answer each question.
- Why can a law that assigns probability one to a Lebesgue-null set not have a density with respect to ambient Lebesgue measure?
- If \(X\in\mathbb{R}^d\) has finite second moments and \(a^\mathsf{T}X=c\) almost surely for a nonzero \(a\), what does this imply about \(\operatorname{Cov}(X)\)?
- For the vector \((U,V,U+V)\), state an explicit nonzero vector that gives a zero-variance linear combination of its coordinates.
- Why does the parabola example show that singular covariance is sufficient but not necessary for lack of an ambient density?
- What additional geometric information, beyond whether the covariance matrix is singular, would be needed to assess whether a law is concentrated on a lower-dimensional set?