Multivariate Normality Through Linear Combinations

Why Define Normality Through Linear Combinations?

For a single random variable, normality describes its whole distribution. For a vector, checking that each coordinate is normal is not enough: the coordinates may depend on one another in ways that make a linear combination nonnormal. A useful definition therefore asks a stronger question: is every scalar linear combination of the coordinates normal?

This definition covers both ordinary Gaussian vectors and vectors whose values are confined to a lower-dimensional set. In the latter case, some nonzero linear combinations can be constant. Those constant projections must count as normal with variance zero; excluding them would rule out important degenerate cases.

Definition by Scalar Linear Combinations

Definition: Let \(X=(X_1,\ldots,X_p)^T\) be a random vector with values in \(\mathbb{R}^p\). It is multivariate normal (or a Gaussian vector) if, for every \(a=(a_1,\ldots,a_p)^T\in\mathbb{R}^p\), the scalar random variable \(a^T X=\sum_{j=1}^p a_jX_j\) has a one-dimensional normal distribution. A normal distribution may be degenerate: \(N(m,0)\) means the point mass at \(m\).

The coefficient vector \(a\) is arbitrary: its entries may be positive, negative, or zero. The choice \(a=0\) gives the constant zero and is automatically allowed. More importantly, a nonzero \(a\) can also produce a constant projection. By the definition of \(N(m,0)\), a random variable with that distribution equals \(m\) with probability one.

This definition is about the joint behavior of the vector, not just the distributions of \(X_1,\ldots,X_p\) separately. To establish multivariate normality directly from the definition, one must check the distribution of every possible projection. A useful way to do this in practice is to construct the vector from independent normal variables and derive the distribution of an arbitrary linear combination.

Worked Example: A Standard Normal Vector

Let \(Z_1\) and \(Z_2\) be independent \(N(0,1)\) random variables, and set \(X=(Z_1,Z_2)^T\). To check the definition, take any \(a=(a_1,a_2)^T\). Then

$$ a^T X=a_1Z_1+a_2Z_2. $$

Because the variables are independent normals, this sum is normal with mean zero and variance \(a_1^2+a_2^2\). Thus

a^T X\sim N(0,a_1^2+a_2^2).

If \(a\ne 0\), then \(a_1^2+a_2^2>0\). If \(a=0\), the projection is the constant zero, which has distribution \(N(0,0)\). Every scalar linear combination is therefore normal, including the degenerate one, so \(X\) is multivariate normal.

Independent Normal Components: A Useful Construction

The calculation in the preceding example relies on a general fact: a sum of finitely many independent normal random variables is normal, even when some summands are constants. Here is a derivation that includes the degenerate case.

Theorem: Suppose \(X_1,\ldots,X_p\) are independent, with \(X_j\sim N(\mu_j,\tau_j^2)\) and \(\tau_j^2\ge 0\). For any fixed real coefficients \(a_1,\ldots,a_p\),

\sum_{j=1}^p a_jX_j\sim N\left(\sum_{j=1}^p a_j\mu_j,\ \sum_{j=1}^p a_j^2\tau_j^2\right).

Proof. If \(\tau_j^2=0\), then \(X_j=\mu_j\) with probability one, so \(a_jX_j=a_j\mu_j\) is a constant. If \(\tau_j^2>0\), scaling the normal density shows that \(a_jX_j\) is normal with mean \(a_j\mu_j\) and variance \(a_j^2\tau_j^2\); when \(a_j=0\), this is again a constant. The nonconstant terms remain independent because they are functions of distinct independent variables.

It remains to verify that adding independent nonconstant normal variables adds their variances. For independent centered normals \(U\sim N(0,v)\) and \(V\sim N(0,w)\), where \(v,w>0\), the density of \(U+V\) is the convolution

\begin{aligned} f_{U+V}(x) &=\frac{1}{2\pi\sqrt{vw}}\int_{-\infty}^{\infty} \exp\left\{-\frac{(x-y)^2}{2v}-\frac{y^2}{2w}\right\}\,dy,\\ \frac{(x-y)^2}{v}+\frac{y^2}{w} &=\frac{v+w}{vw}\left(y-\frac{wx}{v+w}\right)^2+\frac{x^2}{v+w}. \end{aligned}

Substituting the second identity into the integral and using the integral of a normal kernel gives

f_{U+V}(x)=\frac{1}{\sqrt{2\pi(v+w)}}\exp\left\{-\frac{x^2}{2(v+w)}\right\}.

So \(U+V\sim N(0,v+w)\). Adding the means shifts this distribution by their sum. Repeating this two-variable argument proves the formula for any finite number of nonconstant terms; constant terms add to the mean and contribute zero to the variance. This proves the theorem. \(\square\)

The theorem immediately gives a construction of multivariate normal vectors: if the coordinates are independent normals, every scalar linear combination is normal. Independence is a convenient sufficient condition for this construction, not part of the definition itself.

Worked Example: A Projection That Is Constant

Let \(Z\sim N(0,1)\) and define \(X_1=2+Z\) and \(X_2=-1+3Z\). For arbitrary real \(a,b\),

$$ aX_1+bX_2=(2a-b)+(a+3b)Z. $$

If \(a+3b\ne 0\), this is normal with mean \(2a-b\) and variance \((a+3b)^2\). If \(a+3b=0\), the random part disappears, and the projection is the constant \(2a-b\). In that case its distribution is \(N(2a-b,0)\).

For instance, choose \(a=3\) and \(b=-1\). Then \(a+3b=3-3=0\), and

$$ 3X_1-X_2=3(2+Z)-(-1+3Z)=7. $$

Every projection is therefore normal, including the nonzero coefficient choices that yield constants. By the definition, \((X_1,X_2)^T\) is multivariate normal.

Why Normal Marginals Are Not Enough

The definition requires every linear combination to be normal, not merely each coordinate. The distinction matters because dependence can change the distribution of a sum even when both summands have normal marginals.

Worked Example: Normal Coordinates, Non-Normal Vector

Let \(Z\sim N(0,1)\), and let \(S\) be independent of \(Z\), with \(P(S=1)=P(S=-1)=1/2\). Set \(X_1=Z\) and \(X_2=SZ\). Since \(S\) only changes the sign of \(Z\), and the standard normal distribution is symmetric about zero, \(X_2\sim N(0,1)\). Also, \(X_1\sim N(0,1)\).

Now consider the linear combination \(X_1+X_2\). When \(S=-1\), it is zero. When \(S=1\), it is \(2Z\), which equals zero with probability zero. Thus

P(X_1+X_2=0)=P(S=-1)+P(S=1)P(2Z=0)=\frac12+\frac12\cdot 0=\frac12.

A nondegenerate one-dimensional normal distribution assigns probability zero to every single point, while a degenerate normal distribution puts probability one at one point. This sum has probability \(1/2\) at zero, so it is neither type of normal distribution. Hence \((X_1,X_2)^T\) is not multivariate normal, despite both coordinates being standard normal.

What the Definition Guarantees

Two consequences follow directly from the definition. First, every coordinate of a multivariate normal vector is normal: choose \(a\) to have a 1 in the coordinate's position and zeros elsewhere, giving \(a^TX=X_j\). Second, for any fixed coefficients, the resulting scalar projection is normal, with a variance that is permitted to be zero. These statements are consequences of the definition; they do not say that normal coordinate marginals alone are sufficient.

The requirement ranges over an uncountable set of coefficient vectors, so it is not generally a practical checklist for data. Its value is conceptual and mathematical: it specifies exactly what joint normality means, including cases where the vector has deterministic linear relationships among its coordinates. In applications, one often establishes the requirement using a construction such as independent normal components, or relies on additional characterizations once those have been introduced.

Common Mistake

Wrong reasoning: “Each coordinate is normal, so the vector is multivariate normal.” The example \((Z,SZ)^T\) disproves this: both coordinates are standard normal, but their sum is not normal.
Correction: Check the joint requirement. Every scalar linear combination must have a normal distribution, including combinations with negative coefficients and combinations that are constant.
Wrong reasoning: “A constant projection cannot be normal.” Under the convention \(N(m,0)\) is a point mass at \(m\), a constant projection is a degenerate normal variable.

Key takeaway: A random vector is multivariate normal precisely when every scalar linear combination of its coordinates is one-dimensional normal, allowing variance zero. Normal coordinate marginals alone do not establish this property.

Check Your Understanding

Answer each question using the linear-combination definition.

What does \(N(m,0)\) mean, and why must it be allowed in the definition?
If \(X_1,\ldots,X_p\) are independent normals, what are the mean and variance of \(\sum_{j=1}^p a_jX_j\)?
For \(X_1=2+Z\) and \(X_2=-1+3Z\), which condition on \(a,b\) makes \(aX_1+bX_2\) constant?
Why does the vector \((Z,SZ)^T\) fail to be multivariate normal even though both coordinates are standard normal?
Does verifying that each coordinate of a vector is normal suffice to prove multivariate normality? Explain.