Multivariate CLTs for Independent Arrays

From Scalar Limits to Vector Limits

A vector sum can have several coordinates that fluctuate together, so a central limit theorem must describe both the marginal fluctuations and their dependence. For independent triangular arrays, the strategy is to examine every fixed linear combination of the vector summands. Each such combination is a scalar triangular array, where the Lindeberg–Feller theorem applies; the Cramér–Wold theorem then translates those scalar limits into a joint limit.

The covariance matrix records how the coordinates fluctuate together, but it does not by itself prevent a small number of summands from carrying substantial variance. The Lindeberg conditions control those large contributions in every direction.

Setup and Theorem

Definition: For each $n$, let $X_{n1},\ldots,X_{nk_n}$ be independent random vectors in $\mathbb{R}^d$, with $E[X_{nk}]=0$ and finite second moments. Define the row sum and its covariance matrix by $$ S_n=\sum_{k=1}^{k_n}X_{nk}, \qquad \Sigma_n=\sum_{k=1}^{k_n}\operatorname{Cov}(X_{nk}). $$ For a fixed vector $a\in\mathbb{R}^d$, the scalar projection of a summand is $a^\mathsf{T}X_{nk}$. The projected Lindeberg condition is that, for every $a\in\mathbb{R}^d$ and every $\varepsilon>0$, $$ \sum_{k=1}^{k_n}E\!\left[(a^\mathsf{T}X_{nk})^2 \mathbf{1}\{|a^\mathsf{T}X_{nk}|>\varepsilon\}\right]\longrightarrow 0. $$

Here $\mathbf{1}\{A\}$ is 1 when the event $A$ occurs and 0 otherwise. The threshold $\varepsilon$ is fixed as $n$ grows. The condition says that, in each fixed direction, the total second-moment contribution from summands exceeding that threshold vanishes.

Theorem: Suppose the vectors in each row are independent and centered, with finite second moments. Suppose $\Sigma_n$ converges entrywise to a finite matrix $\Sigma$, and the projected Lindeberg condition holds for every $a\in\mathbb{R}^d$ and $\varepsilon>0$. Then $$ S_n\xrightarrow{d}N_d(0,\Sigma). $$ The matrix $\Sigma$ is the limiting covariance matrix. It is symmetric and positive semidefinite, since each $\Sigma_n$ is.

The notation $N_d(0,\Sigma)$ denotes the $d$-dimensional normal distribution with mean zero and covariance matrix $\Sigma$. To apply the theorem, one must check both the limiting covariance and the projected Lindeberg conditions. In particular, the condition is about the summands in each row, not just the distribution of the final sum.

Why the Projection Argument Works

Fix $a\in\mathbb{R}^d$. The variables $a^\mathsf{T}X_{n1},\ldots,a^\mathsf{T}X_{nk_n}$ are independent and centered. Their total variance is $$ \sum_{k=1}^{k_n}\operatorname{Var}(a^\mathsf{T}X_{nk}) =a^\mathsf{T}\Sigma_n a \longrightarrow a^\mathsf{T}\Sigma a. $$ The assumed condition is precisely the scalar Lindeberg condition for this projected array.

If $a^\mathsf{T}\Sigma a>0$, the scalar Lindeberg–Feller theorem gives $$ a^\mathsf{T}S_n\xrightarrow{d}N(0,a^\mathsf{T}\Sigma a). $$ If $a^\mathsf{T}\Sigma a=0$, the variance of $a^\mathsf{T}S_n$ tends to zero. For every $\delta>0$, Chebyshev’s inequality gives $$ P(|a^\mathsf{T}S_n|>\delta) \leq \frac{a^\mathsf{T}\Sigma_n a}{\delta^2}\longrightarrow 0. $$ Thus $a^\mathsf{T}S_n$ converges in probability, and hence in distribution, to zero, which is the normal limit with variance zero.

For every fixed $a$, the resulting scalar limit is the distribution of the corresponding projection of $N_d(0,\Sigma)$. The Cramér–Wold theorem states that convergence in distribution of all fixed linear projections to the projections of a given vector distribution implies convergence in distribution to that vector distribution. Applying it here proves the theorem.

Key calculation: For any fixed $a$, the variance of the projected sum is $a^\mathsf{T}\Sigma_n a$. Therefore the limiting covariance matrix is found by summing the within-row covariance matrices and taking their limit, not by treating the coordinates as independent.

Worked Examples

Worked Example: Independent Coordinate Fluctuations

For each $n$, let $R_{n1},\ldots,R_{nn}$ and $T_{n1},\ldots,T_{nn}$ be mutually independent random variables, each taking the values $1$ and $-1$ with probability $1/2$. Define $$ X_{nk}=\frac{1}{\sqrt n}(R_{nk},T_{nk}),\qquad 1\leq k\leq n. $$ Find the limiting covariance and verify the projected Lindeberg condition.

Each vector has mean zero. Since $E[R_{nk}^2]=E[T_{nk}^2]=1$ and $E[R_{nk}T_{nk}]=0$, $$ \operatorname{Cov}(X_{nk}) =\frac{1}{n} \begin{pmatrix} 1&0\\ 0&1 \end{pmatrix}. $$ Adding the $n$ covariance matrices gives $\Sigma_n=I_2$ for every $n$, so the limiting covariance is $I_2$.

For a fixed $a=(a_1,a_2)^\mathsf{T}$, $$ |a^\mathsf{T}X_{nk}| =\frac{|a_1R_{nk}+a_2T_{nk}|}{\sqrt n} \leq\frac{|a_1|+|a_2|}{\sqrt n}. $$ For any $\varepsilon>0$, this upper bound is less than $\varepsilon$ for all sufficiently large $n$. Every indicator in the projected Lindeberg sum is then zero, so that sum tends to zero. The theorem yields $$ \sum_{k=1}^n X_{nk}\xrightarrow{d}N_2(0,I_2). $$

Worked Example: A Nonzero Limiting Correlation

Let $\rho=3/5$. For each row, let $A_{nk}$ and $B_{nk}$ be independent random signs, each equally likely to be $1$ or $-1$, with independence across all indices. Set $$ X_{nk}=\frac{1}{\sqrt n} \left(A_{nk},\frac{3}{5}A_{nk}+\frac{4}{5}B_{nk}\right), \qquad 1\leq k\leq n. $$

Both coordinates have mean zero. The variance of the second coordinate before division by $\sqrt n$ is $$ E\!\left[\left(\frac{3}{5}A_{nk}+\frac{4}{5}B_{nk}\right)^2\right] =\frac{9}{25}+\frac{16}{25} +\frac{24}{25}E[A_{nk}B_{nk}] =1, $$ because $E[A_{nk}B_{nk}]=0$. The covariance between the unscaled coordinates is $$ E\!\left[A_{nk}\left(\frac{3}{5}A_{nk}+\frac{4}{5}B_{nk}\right)\right] =\frac{3}{5}. $$ Consequently, $$ \operatorname{Cov}(X_{nk}) =\frac{1}{n} \begin{pmatrix} 1&3/5\\ 3/5&1 \end{pmatrix}, \qquad \Sigma_n= \begin{pmatrix} 1&3/5\\ 3/5&1 \end{pmatrix}. $$

For any fixed $a=(a_1,a_2)^\mathsf{T}$, the absolute value of $a^\mathsf{T}X_{nk}$ is bounded above by $$ \frac{|a_1|+(3/5)|a_2|+(4/5)|a_2|}{\sqrt n} =\frac{|a_1|+(7/5)|a_2|}{\sqrt n}. $$ For every positive $\varepsilon$, this bound is eventually less than $\varepsilon$, so the projected Lindeberg sum is eventually zero. The limiting vector is therefore normal with variances 1 and correlation $3/5$, as recorded by the off-diagonal entries of its covariance matrix.

Worked Example: Covariance Convergence Without a CLT

This example shows why covariance convergence cannot replace Lindeberg’s condition. Let $J_n$ equal $\sqrt n$ or $-\sqrt n$, each with probability $1/(2n)$, and equal zero with probability $1-1/n$. Let $R_{n1},\ldots,R_{nn}$ be independent random signs, independent of $J_n$. Define $$ X_{n1}=(J_n,R_{n1}/\sqrt n),\qquad X_{nk}=(0,R_{nk}/\sqrt n),\quad 2\leq k\leq n. $$ These vectors are independent and centered within each row.

The first coordinate of $X_{n1}$ has variance $$ E[J_n^2]=n\left(\frac{1}{2n}+\frac{1}{2n}\right)=1. $$ The total variance of the second coordinate is $n(1/n)=1$, and all cross-covariances are zero by independence and centering. Hence $\Sigma_n=I_2$ for every $n$.

However, take $a=(1,0)^\mathsf{T}$ and any fixed $\varepsilon>0$. For sufficiently large $n$, $\sqrt n>\varepsilon$, and the projected Lindeberg sum is $$ E[J_n^2\mathbf{1}\{|J_n|>\varepsilon\}] =n\left(\frac{1}{2n}+\frac{1}{2n}\right)=1. $$ It does not tend to zero. Indeed, the first coordinate of $S_n$ is $J_n$, which is zero with probability $1-1/n$; it tends to zero in probability despite having variance 1 for every $n$. Thus covariance convergence alone does not ensure the claimed normal limit.

How to Apply the Theorem and Avoid a Pitfall

A reliable verification separates the calculation into two tasks. First sum the covariance matrices and identify the candidate limit. Then fix an arbitrary vector $a$, project every summand, and control the total squared contribution from projected values larger than a fixed threshold. A bound on the largest possible projected summand, as in the first two examples, is often enough: if that bound tends to zero, the Lindeberg sum is eventually zero for each fixed threshold.

Check row assumptions.
Confirm independence within each row, zero means, and finite second moments.

Compute the covariance limit.
Find $\Sigma_n=\sum_k\operatorname{Cov}(X_{nk})$ and check entrywise convergence.

Check every fixed projection.
For arbitrary fixed $a$ and $\varepsilon>0$, verify that the projected Lindeberg sum tends to zero.

Conclude joint convergence.
Apply scalar Lindeberg–Feller to the projections and then use Cramér–Wold.

A common mistake is to calculate the limiting covariance correctly and conclude that the sum must be approximately normal. The third example shows the gap: a rare, large summand can preserve a nonzero variance while disappearing with high probability. The covariance describes second moments; Lindeberg’s condition rules out this kind of unbalanced contribution in each fixed direction.

Key takeaway: For independent vector arrays, a multivariate normal limit follows when the summed covariance matrices converge and every fixed scalar projection satisfies Lindeberg’s condition. Covariance identifies the candidate limit; the projection conditions justify the normal approximation.

Check Your Understanding

Use the definitions and theorem above to answer the following questions.

For a fixed $a$, what is the total variance of $a^\mathsf{T}S_n$ in terms of $a$ and $\Sigma_n$?
Why does the projected Lindeberg condition have to be checked for every fixed direction $a$, rather than inferred from covariance convergence?
In the first worked example, why does a uniform bound on $|a^\mathsf{T}X_{nk}|$ establish Lindeberg’s condition?
In the second worked example, which covariance-matrix entries encode the correlation between the two limiting coordinates?
In the third worked example, explain how the first-coordinate variance remains 1 even though that coordinate tends to zero in probability.