The Cramér–Wold Proof of the Multivariate CLT

From Scalar Limits to a Joint Limit

A vector-valued sample mean contains several coordinates, but its fluctuations are not determined by the coordinates in isolation. Dependence between coordinates matters: the uncertainty of a difference, for example, depends on their covariance. The Cramér–Wold method handles this by examining every fixed linear combination of the coordinates. Each such combination is a scalar sum, so the scalar iid central limit theorem can be applied directly.

The argument has two parts. First, show that each projection of the centered and scaled sample mean has the appropriate scalar normal limit. Then use the Cramér–Wold characterization to turn convergence of all those projections into convergence of the vector itself.

Definition: For a random vector \(X\in\mathbb R^d\) with finite second moments, its mean is \(\mu=E[X]\), and its covariance matrix is \(\Sigma=E[(X-\mu)(X-\mu)^{\mathsf T}]\). A multivariate normal vector \(Z\sim N_d(0,\Sigma)\) is characterized by the characteristic function \(E[\exp(i t^{\mathsf T}Z)]=\exp(-\tfrac12 t^{\mathsf T}\Sigma t)\) for \(t\in\mathbb R^d\).

A fixed vector \(t\in\mathbb R^d\) defines a scalar projection \(t^{\mathsf T}X\). Its expectation and variance follow from the definitions:

E[t^{\mathsf T}X]=t^{\mathsf T}\mu,\qquad \operatorname{Var}(t^{\mathsf T}X) =E\!\left[\{t^{\mathsf T}(X-\mu)\}^2\right] =t^{\mathsf T}\Sigma t.

Thus the covariance matrix records the variance in every direction. In particular, the positive-definiteness condition \(t^{\mathsf T}\Sigma t>0\) for every nonzero \(t\) ensures that each nontrivial projection has positive variance.

The Cramér–Wold Characterization

The key bridge between scalar and vector convergence is the Cramér–Wold characterization. It concerns a sequence of random vectors \(Y_n\) and a proposed limiting random vector \(Y\), all taking values in \(\mathbb R^d\).

Theorem (Cramér–Wold characterization): \(Y_n\xrightarrow{d}Y\) if and only if \(t^{\mathsf T}Y_n\xrightarrow{d}t^{\mathsf T}Y\) for every fixed \(t\in\mathbb R^d\).

For the forward direction, if \(Y_n\xrightarrow{d}Y\), then for any bounded continuous function \(g:\mathbb R\to\mathbb R\), the function \(y\mapsto g(t^{\mathsf T}y)\) is bounded and continuous on \(\mathbb R^d\). The definition of convergence in distribution therefore gives convergence of the projected variables.

For the reverse direction, suppose every fixed projection converges in distribution. The characteristic function of \(Y_n\) at any \(t\) is the characteristic function of \(t^{\mathsf T}Y_n\) evaluated at \(1\). Hence it converges to the characteristic function of \(t^{\mathsf T}Y\) at \(1\), which is the characteristic function of \(Y\) at \(t\). The limiting characteristic function is continuous at the origin, so the multivariate Lévy continuity theorem gives \(Y_n\xrightarrow{d}Y\). This establishes the characterization.

The Multivariate IID Central Limit Theorem

Theorem (Multivariate iid central limit theorem): Let \(X_1,X_2,\ldots\) be iid random vectors in \(\mathbb R^d\), with \(E[X_i]=\mu\) and covariance matrix \(\Sigma\). Assume \(E[\|X_i\|^2]<\infty\) and that \(\Sigma\) is positive definite. Then

\frac{1}{\sqrt n}\sum_{i=1}^n(X_i-\mu) =\sqrt n(\bar X_n-\mu) \xrightarrow{d}N_d(0,\Sigma), \qquad \bar X_n=\frac1n\sum_{i=1}^n X_i.

The positive-definiteness assumption is used here to ensure that every nonzero projection has positive variance, so the ordinary nondegenerate scalar iid central limit theorem applies to it. The proof is a sequence of scalar applications followed by Cramér–Wold.

Fix a direction.
Choose any fixed \(t\in\mathbb R^d\). If \(t=0\), the projected variables are identically zero and already have the required limit.

Form scalar observations.
For nonzero \(t\), set \(W_i=t^{\mathsf T}(X_i-\mu)\). These variables are iid, have mean zero, and have variance \(t^{\mathsf T}\Sigma t>0\).

Apply the scalar CLT.
The Lindeberg–Levy theorem gives a normal limit for \(n^{-1/2}\sum_{i=1}^n W_i\).

Identify the vector limit.
The resulting scalar normal law is the law of the same projection of \(N_d(0,\Sigma)\). Cramér–Wold then gives the claimed vector convergence.

Here are the calculations underlying those steps. Since each \(X_i\) has finite second moments, \(W_i\) also has a finite second moment. Linearity of expectation and the covariance identity give

E[W_i]=t^{\mathsf T}E[X_i-\mu]=0,\qquad \operatorname{Var}(W_i) =E[(t^{\mathsf T}(X_i-\mu))^2] =t^{\mathsf T}\Sigma t.

Independence and identical distribution of the vectors imply the same properties for the scalar variables \(W_i\). The scalar iid central limit theorem therefore yields

t^{\mathsf T}\left\{\frac{1}{\sqrt n}\sum_{i=1}^n(X_i-\mu)\right\} =\frac{1}{\sqrt n}\sum_{i=1}^n W_i \xrightarrow{d}N(0,t^{\mathsf T}\Sigma t).

By the definition of \(N_d(0,\Sigma)\), the projection \(t^{\mathsf T}Z\) for \(Z\sim N_d(0,\Sigma)\) has precisely the distribution \(N(0,t^{\mathsf T}\Sigma t)\). Thus every fixed projection of the vector on the left converges to the corresponding projection of \(Z\). The Cramér–Wold characterization completes the proof. \(\square\)

Worked Projections

Worked Example: A Contrast of Two Measurements

Suppose each subject contributes a two-dimensional measurement \(X_i=(A_i,B_i)^{\mathsf T}\), with mean \(\mu=(10,20)^{\mathsf T}\) and covariance matrix

\Sigma= \begin{pmatrix} 4&1\\ 1&9 \end{pmatrix}.

The matrix is positive definite: its first diagonal entry is \(4>0\), and its determinant is \(4(9)-1(1)=35>0\). For the contrast \(A_i-B_i\), take \(t=(1,-1)^{\mathsf T}\). Its mean is \(t^{\mathsf T}\mu=10-20=-10\), and its variance is

t^{\mathsf T}\Sigma t =4+9-2(1)=11.

Consequently, the scalar CLT gives

\sqrt n\{(\bar A_n-\bar B_n)-(-10)\} \xrightarrow{d}N(0,11).

The covariance term matters: adding the marginal variances alone would give \(4+9=13\), not \(11\). The projection method automatically includes the covariance with the correct sign for the contrast.

Worked Example: Mutually Exclusive Categories

For each trial, let \(X_i=(I_{Ai},I_{Bi})^{\mathsf T}\), where \(I_{Ai}\) and \(I_{Bi}\) indicate two mutually exclusive outcomes. Suppose \(P(A)=0.3\), \(P(B)=0.2\), and the remaining outcome has probability \(0.5\). Then

\mu= \begin{pmatrix}0.3\\0.2\end{pmatrix}, \qquad \Sigma= \begin{pmatrix} 0.3(0.7)&-0.3(0.2)\\ -0.3(0.2)&0.2(0.8) \end{pmatrix} = \begin{pmatrix} 0.21&-0.06\\ -0.06&0.16 \end{pmatrix}.

The off-diagonal entry is negative because \(I_{Ai}I_{Bi}=0\) on every trial, so \(E[I_{Ai}I_{Bi}]-E[I_{Ai}]E[I_{Bi}]=0-(0.3)(0.2)=-0.06\). The determinant is \(0.21(0.16)-0.06^2=0.03>0\), so this covariance matrix is positive definite.

For any fixed \(t=(a,b)^{\mathsf T}\), the projected observation is \(aI_{Ai}+bI_{Bi}\), with mean \(0.3a+0.2b\) and variance

t^{\mathsf T}\Sigma t=0.21a^2-0.12ab+0.16b^2.

The scalar CLT applies to the centered projected sum for every \(a,b\). Cramér–Wold therefore yields the joint normal limit for the two centered, scaled category proportions, with exactly the covariance matrix shown above.

Worked Example: Dependent Coordinates Built From Independent Variables

Let \(U_i\) and \(V_i\) be independent random signs, each taking \(1\) and \(-1\) with probability \(1/2\). Assume the pairs \((U_i,V_i)\) are iid, and define \(X_i=(U_i,U_i+V_i)^{\mathsf T}\). Then \(E[X_i]=0\). Since \(E[U_i^2]=E[V_i^2]=1\) and \(E[U_iV_i]=0\),

\Sigma= \begin{pmatrix} E[U_i^2]&E[U_i(U_i+V_i)]\\ E[U_i(U_i+V_i)]&E[(U_i+V_i)^2] \end{pmatrix} = \begin{pmatrix}1&1\\1&2\end{pmatrix}.

Its determinant is \(1\), so the covariance matrix is positive definite. For \(t=(a,b)^{\mathsf T}\), the scalar projection is

t^{\mathsf T}X_i=aU_i+b(U_i+V_i)=(a+b)U_i+bV_i,

and its variance is \((a+b)^2+b^2=a^2+2ab+2b^2=t^{\mathsf T}\Sigma t\). These projected variables are iid and bounded, hence have finite variance; the scalar CLT applies in every direction. The vector result is

\frac1{\sqrt n}\sum_{i=1}^n X_i \xrightarrow{d}N_2\!\left(0, \begin{pmatrix}1&1\\1&2\end{pmatrix}\right).

The coordinates are dependent even though \(U_i\) and \(V_i\) were independent: both coordinates of \(X_i\) contain \(U_i\). The covariance matrix captures that dependence in the limiting vector.

Why Every Direction Matters

It is not enough to prove a scalar CLT separately for each coordinate. Coordinatewise convergence checks only the directions \(t=(1,0,\ldots,0)^{\mathsf T}\), \(t=(0,1,\ldots,0)^{\mathsf T}\), and so on. Those checks do not by themselves establish that the joint distribution converges to the claimed one. The Cramér–Wold step requires convergence for every fixed direction, including contrasts and other combinations that reveal dependence.

A common mistake is to calculate the limiting marginal distribution of each coordinate and conclude that the limit vector has independent coordinates. That conclusion is not justified: marginal limits do not determine a joint distribution, and a multivariate normal limit can have nonzero off-diagonal covariances. The correction is to compute \(t^{\mathsf T}\Sigma t\) for a general \(t\), apply the scalar CLT to that projection, and identify its limit with the projection of the proposed multivariate normal vector. Doing this for every \(t\) is precisely what makes Cramér–Wold applicable.

Key takeaway: The multivariate iid CLT follows by projecting the vector sum in an arbitrary fixed direction, applying the scalar iid CLT to that projection, and using Cramér–Wold to recover joint convergence. The full covariance matrix enters through the projected variance \(t^{\mathsf T}\Sigma t\).

Check Your Understanding

Use the projection argument and the stated covariance identities to answer these questions.

For a fixed \(t\), what are the mean and variance of \(t^{\mathsf T}(X_i-\mu)\)?
Why does positive definiteness of \(\Sigma\) ensure that the scalar CLT has positive variance for every nonzero projection?
If \(t=(1,-1)^{\mathsf T}\), write the projected variance in terms of the two marginal variances and their covariance.
What additional step turns convergence of every scalar projection into convergence of the vector?
Why does convergence of each coordinate separately not, by itself, establish the claimed joint normal limit?