Why Several Definitions Describe the Same Class
A multivariate normal distribution can be recognized in three different ways: by looking at every scalar linear combination of its coordinates, by constructing the vector as a linear image of a standard normal vector, or by checking its characteristic function. Each viewpoint emphasizes something different. Linear combinations make the distribution's one-dimensional projections explicit; the linear-image description gives a concrete construction; and the characteristic function provides a compact formula that also covers degenerate cases.
The equivalence depends on allowing zero variances. A vector can be multivariate normal even when its coordinates satisfy an exact linear relation, so its covariance matrix need not be invertible. In what follows, a normal distribution with variance zero means a point mass at its mean.
Three Characterizations
Let \(X\) be a random vector in \(\mathbb{R}^d\). Vectors such as \(a\) and \(t\) are column vectors, and a prime denotes transpose. The characteristic function of \(X\) is \(\phi_X(t)=E[\exp(i t'X)]\), for \(t\in\mathbb{R}^d\).
Linear-combination characterization: \(X\) is multivariate normal if \(a'X\) has a univariate normal distribution for every \(a\in\mathbb{R}^d\), allowing variance zero. In this case, set \(\mu=E[X]\) and \(\Sigma=\operatorname{Cov}(X)\).
Linear-image characterization: There are a standard normal vector \(Z\sim N_d(0,I_d)\) and a matrix \(A\in\mathbb{R}^{d\times d}\) such that \(X\) has the same distribution as \(\mu+AZ\), where \(AA'=\Sigma\). A factor with fewer columns can be used when \(\Sigma\) is singular.
Characteristic-function characterization: For every \(t\in\mathbb{R}^d\), $$ \phi_X(t)=\exp\!\left(i t'\mu-\frac{1}{2}t'\Sigma t\right). $$
In the scalar notation \(N(m,v)\) used below, the second argument \(v\) is the variance. If \(X\) satisfies the linear-combination characterization, each coordinate is normal by taking \(a\) to be a coordinate vector. Thus each coordinate has a finite second moment, so \(E[X]\) and \(\operatorname{Cov}(X)\) are defined. For every \(a\), \(a'X\) then has mean \(a'\mu\) and variance \(a'\Sigma a\). In particular, \(\Sigma\) is positive semidefinite because \(a'\Sigma a=\operatorname{Var}(a'X)\geq 0\).
The standard normal vector \(Z\sim N_d(0,I_d)\) has independent standard normal coordinates. The characteristic function of a univariate \(N(m,v)\) variable is \(\exp(i s m-s^2v/2)\). These standard facts will be used in the proof.
Worked Example: Checking a Linear-Image Construction
Let \(Z\sim N_2(0,I_2)\), let \(\mu=(1,-1)'\), and take $$ A=\begin{pmatrix}1&0\\1&2\end{pmatrix}. $$ For \(X=\mu+AZ\), find \(AA'\) and the distribution of the projection with \(a=(2,-1)'\).
First multiply the rows of \(A\) to obtain $$ AA'=\begin{pmatrix} 1\cdot1+0\cdot0 & 1\cdot1+0\cdot2\\ 1\cdot1+2\cdot0 & 1\cdot1+2\cdot2 \end{pmatrix} =\begin{pmatrix}1&1\\1&5\end{pmatrix}. $$ The off-diagonal entry is \(1\cdot1+0\cdot2=1\); the other off-diagonal entry is also \(1\). Thus the covariance parameter in the linear-image characterization is \(\Sigma=\begin{pmatrix}1&1\\1&5\end{pmatrix}\).
The projection has mean \(a'\mu=2(1)+(-1)(-1)=3\). Also, $$ a'A=(2,-1)\begin{pmatrix}1&0\\1&2\end{pmatrix}=(1,-2), \qquad a'AA'a=(1,-2)(1,-2)'=1^2+(-2)^2=5. $$ Therefore \(a'X\sim N(3,5)\). The same variance follows by multiplying \(\Sigma a=(1,-3)'\), giving \(a'\Sigma a=2(1)+(-1)(-3)=5\).
Equivalence Theorem and Proof
We first record the matrix fact needed for the construction. If \(\Sigma\) is a real symmetric positive semidefinite \(d\times d\) matrix, the spectral theorem gives an orthogonal matrix \(Q\) and nonnegative eigenvalues \(\lambda_1,\ldots,\lambda_d\) such that $$ \Sigma=Q\operatorname{diag}(\lambda_1,\ldots,\lambda_d)Q'. $$ Consequently, \(A=Q\operatorname{diag}(\sqrt{\lambda_1},\ldots,\sqrt{\lambda_d})\) satisfies \(AA'=\Sigma\). This construction does not require any eigenvalue to be strictly positive. If \(\Sigma\) has rank \(r\), retaining just the \(r\) positive eigenvalues gives a \(d\times r\) factor instead.
Proof. We prove the implications in turn.
Linear combinations imply the characteristic-function formula. Under the linear-combination characterization, for each fixed \(t\), the scalar \(t'X\) is normal with mean \(t'\mu\) and variance \(t'\Sigma t\). Applying the univariate normal characteristic function at argument \(1\) gives $$ \phi_X(t)=E[\exp(i t'X)] =\exp\!\left(i t'\mu-\frac{1}{2}t'\Sigma t\right). $$ If \(t'\Sigma t=0\), then \(t'X=t'\mu\) almost surely, and the same formula is \(\exp(i t'\mu)\), as required.
The characteristic-function formula implies the linear-image characterization. Since \(\Sigma\) is positive semidefinite, construct \(A\) with \(AA'=\Sigma\) as above, and let \(Z\sim N_d(0,I_d)\). Set \(Y=\mu+AZ\). Independence of the coordinates of \(Z\), together with the univariate normal characteristic function, gives $$ \phi_Y(t) =\exp(i t'\mu)\prod_{j=1}^d \exp\!\left(-\frac{1}{2}\big((A't)_j\big)^2\right) =\exp\!\left(i t'\mu-\frac{1}{2}t'AA't\right) =\exp\!\left(i t'\mu-\frac{1}{2}t'\Sigma t\right). $$ Thus \(X\) and \(Y\) have the same characteristic function. The uniqueness theorem for characteristic functions states that two probability distributions on \(\mathbb{R}^d\) with the same characteristic function at every \(t\in\mathbb{R}^d\) are equal. It follows that \(X\) and \(Y\) have the same distribution, which is the linear-image characterization.
The linear-image characterization implies linear combinations are normal. If \(X\) has the same distribution as \(\mu+AZ\), then, for any \(a\in\mathbb{R}^d\), $$ a'X\ \overset{d}{=}\ a'\mu+(A'a)'Z. $$ The second term on the right is a linear combination of independent standard normal coordinates, so it is normal with mean zero and variance \((A'a)'(A'a)=a'AA'a\). This statement includes \(A'a=0\), when the term is the constant zero. Hence every \(a'X\) is normal, with mean \(a'\mu\) and variance \(a'AA'a\), as required. Together, these implications prove all three characterizations equivalent. \(\square\)
Worked Example: A Singular Gaussian Vector
Let \(Z\sim N(0,1)\) and define \(X=(Z,1+2Z)'\). Express \(X\) as a linear image, calculate its covariance parameter, and check a projection whose variance is zero.
The representation is \(X=\mu+AZ\), where $$ \mu=\begin{pmatrix}0\\1\end{pmatrix}, \qquad A=\begin{pmatrix}1\\2\end{pmatrix}. $$ Direct multiplication gives $$ AA'=\begin{pmatrix}1\cdot1&1\cdot2\\2\cdot1&2\cdot2\end{pmatrix} =\begin{pmatrix}1&2\\2&4\end{pmatrix}. $$ This matrix is singular: its second row is twice its first row. For \(a=(2,-1)'\), the projection is $$ a'X=2Z-(1+2Z)=-1. $$ Its mean is \(a'\mu=2(0)-1(1)=-1\), and its variance is $$ a'AA'a=(a'A)(a'A)'=(2-2)^2=0. $$ Thus the projection is a degenerate normal variable, exactly as the linear-combination characterization permits. In the characteristic-function formula, the same zero variance makes the quadratic term vanish for this direction.
Worked Example: Recovering a Linear Image From a Characteristic Function
Suppose a random vector \(X\in\mathbb{R}^2\) has characteristic function $$ \phi_X(t)=\exp\!\left(i(t_1-2t_2)-\frac{1}{2}(4t_1^2-2t_1t_2+t_2^2)\right). $$ Identify its parameters and construct a linear-image representation.
The linear term is \(i t'\mu\) for \(\mu=(1,-2)'\). The quadratic term equals \(t'\Sigma t\) for $$ \Sigma=\begin{pmatrix}4&-1\\-1&1\end{pmatrix}, $$ since \(t'\Sigma t=4t_1^2-2t_1t_2+t_2^2\). A factor is $$ A=\begin{pmatrix}2&0\\-\frac{1}{2}&\frac{\sqrt{3}}{2}\end{pmatrix}, \qquad AA'=\begin{pmatrix}4&-1\\-1&\frac14+\frac34\end{pmatrix} =\begin{pmatrix}4&-1\\-1&1\end{pmatrix}. $$ Therefore \(X\) has the same distribution as \(\mu+AZ\) for \(Z\sim N_2(0,I_2)\). As a check, at \(t=(1,2)'\) the quadratic form is \(4-4+4=4\) and \(t'\mu=1-4=-3\), so \(\phi_X(1,2)=\exp(-2-3i)\).
What the Equivalence Does—and Does Not—Require
The equivalence is useful because a problem may present only one of the three descriptions. A projection calculation can establish normality by the linear-combination definition; a construction from independent standard normal variables can establish it through a linear image; and a specified characteristic function can be compared directly with the Gaussian formula. The theorem allows a reader to move between these descriptions without changing the distribution being studied.
A common mistake is to assume that the covariance matrix must be invertible. That condition is needed for some later formulas, such as a density with an ordinary inverse covariance matrix, but it is not needed for any characterization proved here. For example, in the preceding singular example, \(X_2-2X_1=1\) with probability one, while every linear combination of the coordinates is still normal, possibly with variance zero. In the matrix construction, zero eigenvalues simply produce zero entries in the square-root factor.
Another error is to use only the fact that each coordinate is normally distributed. Normal coordinates alone do not establish multivariate normality: the definition requires every linear combination to be normal. The characteristic-function formula and the linear-image construction encode the joint distribution, not just the separate coordinate distributions.
Check Your Understanding
Use the three characterizations to answer the following questions.
- Why does the linear-combination definition imply that \(E[X]\) and \(\operatorname{Cov}(X)\) exist?
- If \(X\) has the same distribution as \(\mu+AZ\), what are the mean and variance of \(a'X\) in terms of \(a\), \(\mu\), and \(A\)?
- What does a zero value of \(t'\Sigma t\) imply for the projection \(t'X\) under the characteristic-function characterization?
- Explain why a singular positive semidefinite matrix can still be written as \(AA'\).
- Why is knowing that every coordinate of a vector is normal insufficient, by itself, to invoke the linear-combination characterization?