When CLT Conditions Fail: Dependence and Dominant Terms

When a Normal Approximation Needs More Than a Large Sample

A large number of terms does not, by itself, guarantee that a sum has an approximately normal distribution. The familiar central limit theorem (CLT) for independent, identically distributed terms requires a finite, positive variance. Extensions to independent terms with different distributions also need conditions that prevent any one term or extreme tail from dominating the sum. Dependence can change the variance and the limiting behavior as well.

The practical question is therefore not simply, “Are there many observations?” It is, “Do the assumptions that make the normal approximation work hold for these observations?” This tutorial focuses on three warning signs: strong dependence, infinite variance, and a summand that remains substantial relative to the total variance.

Definition: If $X_1,\ldots,X_n$ are independent and identically distributed with $E[X_i]=\mu$ and $0<\operatorname{Var}(X_i)=\sigma^2<\infty$, the IID central limit theorem states that $$ \frac{\sum_{i=1}^n X_i-n\mu}{\sigma\sqrt{n}} \ \xrightarrow{d}\ N(0,1) $$ as $n$ increases. The finite-variance condition is essential to this version of the theorem. A large $n$ does not repair a failure of independence or an infinite variance.

Dependence Changes the Variance Calculation

For any random variables with finite variances, the variance of a sum includes covariance terms:

\operatorname{Var}\!\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n\operatorname{Var}(X_i) + 2\sum_{1\le i<j\le n}\operatorname{Cov}(X_i,X_j).

Independence makes every covariance in the second sum zero, which is why independent terms with common variance $\sigma^2$ have sum variance $n\sigma^2$. Positive dependence can make the variance much larger; negative dependence can make it smaller. If the covariances are ignored, the standard deviation used to standardize the sum may be wrong, even before considering whether a normal shape is appropriate.

Worked Example: A Shared Random Effect

Suppose 25 hypothetical measurements have the form $X_i=Z$ for every $i$, where the shared variable $Z\sim N(0,1)$. Find the distribution and variance of their sum, then compare the correct probability $P(S>5)$ with the result from incorrectly treating the measurements as independent.

Method. Write the sum in terms of the shared variable, then use the covariance-aware variance formula. The mistake to avoid is using the independent-sum variance $25$ when all measurements are identical copies of one random variable.

Let $S=\sum_{i=1}^{25}X_i$. Since each $X_i=Z$, $$ S=25Z,\qquad E[S]=25E[Z]=0,\qquad \operatorname{Var}(S)=25^2\operatorname{Var}(Z)=625. $$ The correct standard deviation is therefore $25$, and $S/25=Z\sim N(0,1)$. Thus $$ P(S>5)=P\!\left(Z>\frac{5}{25}\right)=P(Z>0.2)\approx 0.4207. $$ If independence were incorrectly assumed, the calculation would use variance $25\operatorname{Var}(Z)=25$, standard deviation $5$, and give $$ P(S>5)\ \text{(incorrectly)}\approx P(Z>1)=0.1587. $$ The correct result is consistent with $S=25Z$: the event $S>5$ is exactly $Z>0.2$. The independent calculation fails because it ignores the perfect dependence.

This example does not establish that every dependent sum fails to have a normal limit. Some forms of weak dependence can permit CLT results, but they require their own conditions. The IID theorem cannot be applied just by counting the terms, and dependence must be addressed in both the variance calculation and the choice of an appropriate limit theorem.

Infinite Variance Removes the Usual Standardization

The IID CLT uses the finite standard deviation $\sigma$ to scale the centered sum by $\sigma\sqrt n$. If the variance is infinite, that scale is not defined. This does not mean the variables have no useful properties: their means may exist, and other limit results may be available under additional assumptions. It does mean that the ordinary finite-variance IID CLT does not justify a normal approximation.

Worked Example: A Pareto Loss With Infinite Variance

A hypothetical individual loss $X$ has a Pareto distribution with minimum value $x_m=2$ and shape parameter $\alpha=1.5$. Its density is $f_X(x)=\alpha x_m^\alpha x^{-(\alpha+1)}$ for $x\ge x_m$. Determine its mean, assess whether its variance is finite, and calculate $P(X>20)$.

Method. Use the Pareto moment conditions and survival function. The mistake to avoid is applying the usual CLT merely because the mean exists; that theorem also requires a finite variance.

For a Pareto variable with this parameterization, $E[X]=\alpha x_m/(\alpha-1)$ when $\alpha>1$. Substitution gives $$ E[X]=\frac{1.5(2)}{1.5-1}=\frac{3}{0.5}=6. $$ The second moment is finite only when $\alpha>2$. Here $1.5<2$, so $E[X^2]=\infty$ and $\operatorname{Var}(X)=\infty$.

The survival function is $P(X>x)=(x_m/x)^\alpha$ for $x\ge x_m$. Hence $$ P(X>20)=\left(\frac{2}{20}\right)^{1.5} =(0.1)^{1.5}\approx 0.0316. $$ The finite mean is $6$, but there is no finite $\sigma$ with which to form the usual IID CLT statistic $(\sum X_i-6n)/(\sigma\sqrt n)$. The calculation therefore rules out that standard justification for a normal approximation; it does not, by itself, establish what alternative limiting distribution applies.

Independent Terms: Check Whether One Term Dominates

For independent, nonidentically distributed terms, a useful framework is a triangular array: the collection of terms may change as the number $n$ of terms increases. Let $X_{n1},\ldots,X_{nn}$ be independent within each row, with finite means and variances. Define the centered terms $Y_{nk}=X_{nk}-E[X_{nk}]$ and the total variance $$ s_n^2=\sum_{k=1}^n\operatorname{Var}(X_{nk}). $$ A normal approximation concerns the centered and standardized sum $\sum_kY_{nk}/s_n$, not an uncentered sum with an arbitrary scale.

The Lindeberg condition is a standard sufficient condition for a CLT for such independent arrays. It requires that, for every fixed $\varepsilon>0$, the total contribution to variance from centered terms larger than $\varepsilon s_n$, relative to the total variance, tends to zero:

Theorem: For an independent triangular array of centered terms with $0<s_n^2<\infty$, the Lindeberg condition is $$ \frac{1}{s_n^2}\sum_{k=1}^n E\!\left[Y_{nk}^2\,\mathbf{1}_{\{|Y_{nk}|>\varepsilon s_n\}}\right] \longrightarrow 0 \quad\text{for every }\varepsilon>0. $$ Under this condition, the standardized row sum converges in distribution to $N(0,1)$. The condition limits the effect of large terms; it is a sufficient condition, not a claim that every array failing it must have a nonnormal limit.

A related diagnostic is whether the largest individual variance becomes negligible relative to $s_n^2$. If one term accounts for a fixed positive fraction of total variance, that term may remain visible after standardization. A small largest-variance fraction alone does not check every possible tail problem; Lindeberg directly checks the contribution from large centered values.

Worked Example: One Summand Retains Half the Variance

For each $n\ge2$, let $R$ equal $1$ or $-1$, each with probability $1/2$, and let $G_2,\ldots,G_n$ be independent $N(0,1)$ variables, independent of $R$. Define $$ X_{n1}=\frac{R}{\sqrt{2}},\qquad X_{nk}=\frac{G_k}{\sqrt{2(n-1)}}\quad (2\le k\le n). $$ Assess whether a CLT is justified by the Lindeberg condition, and calculate $P(T_n>1)$, where $T_n=\sum_{k=1}^nX_{nk}$.

Method. Compute each variance and the total variance, test whether the first term’s large-value contribution vanishes, and then use the exact sum distribution. The mistake to avoid is declaring a CLT valid solely because the number of terms grows.

All terms are centered and independent. The first term has variance $1/2$. Each of the remaining $n-1$ terms has variance $1/[2(n-1)]$, so $$ s_n^2=\frac12+(n-1)\frac{1}{2(n-1)}=1. $$ Thus the first term contributes one half of the total variance for every $n$; its variance share does not approach zero. More directly, choose $\varepsilon=1/2$. Since $|X_{n1}|=1/\sqrt2>1/2=\varepsilon s_n$, its contribution to the Lindeberg expression is $$ E\!\left[X_{n1}^2\mathbf{1}_{\{|X_{n1}|>1/2\}}\right]=\frac12. $$ The expression is therefore at least $1/2$ for every $n$, so it does not tend to zero. The Lindeberg condition fails.

The remaining terms sum to a normal variable with mean zero and variance $1/2$. In distribution, their sum is $G/\sqrt2$ for $G\sim N(0,1)$, independent of $R$. Consequently $T_n$ has, for every $n$, the distribution of $(R+G)/\sqrt2$, not a standard normal distribution. Conditioning on the two values of $R$, $$ P(T_n>1) =\frac12P(G>\sqrt2-1)+\frac12P(G>\sqrt2+1) \approx 0.17362 \approx 0.1736. $$ The final probability is rounded to four decimal places; the component tail probabilities should not be rounded before taking their average. This exact mixture calculation confirms that the standardized sum has not approached a standard normal distribution in this construction.

Why These Diagnostics Matter

A normal approximation is useful only when the distributional assumptions and scale are credible. In applications, the three failure modes call for different checks:

Dependence: determine whether covariance terms change the variance and whether an appropriate dependent-data limit theorem is available.
Infinite variance: do not use the finite-variance IID CLT or its $\sigma\sqrt n$ scaling. Verify the moment assumptions rather than inferring them from a finite sample.
Dominant terms: for independent nonidentical terms, compare individual contributions with total variance and assess a condition such as Lindeberg when applicable.

Failure of a sufficient condition is not automatically proof that a normal approximation is inaccurate in every finite sample. It is a warning that the stated theorem does not justify the approximation. A different theorem, a model-specific analysis, or direct computation may still be appropriate. The essential exam skill is to identify which assumption is at issue and show its consequence for the proposed standardization.

Key takeaway: Before using a CLT for a sum, verify independence or an applicable dependence result, check that the required variance exists, and ensure no term or tail retains a nonnegligible share of total variance. A large number of terms is not a substitute for these checks.

Check Your Understanding

For each problem, identify the relevant CLT diagnostic and show the key variance or tail calculation.

Let $X_i=Z+\varepsilon_i$ for $i=1,\ldots,10$, where $Z\sim N(0,4)$, the $\varepsilon_i$ are independent $N(0,1)$ variables, and all variables are mutually independent except for the shared $Z$ in the $X_i$. Find $\operatorname{Var}(\sum_iX_i)$ and compare it with the variance obtained by incorrectly treating the $X_i$ as independent. (Answer: Correct variance $410$; incorrect independent variance $50$.)
A hypothetical loss has Pareto minimum $x_m=3$ and shape $\alpha=2$. Does it have a finite mean and finite variance? State whether the ordinary finite-variance IID CLT applies. (Answer: The mean is infinite because $\alpha\le1$, and the variance is infinite; the ordinary finite-variance IID CLT does not apply.)
An independent array has $n$ centered terms, with one term of variance $4$ and $n-1$ terms each of variance $4/(n-1)$. Find the total variance and the largest term’s variance share. What does that share indicate? (Answer: Total variance $8$; largest share $4/8=1/2$, so the largest-variance contribution does not vanish.)
For each $n$, let $X_1,\ldots,X_n$ have variance $1$ and common pairwise covariance $0.2$. Find the variance of their average. Does it tend to zero as $n$ increases? (Answer: $\operatorname{Var}(\bar X)=0.2+0.8/n$, which tends to $0.2$, not zero.)
Let $R$ be equally likely to be $1$ or $-1$, let $G\sim N(0,1)$ independently, and set $T=(R+G)/\sqrt2$. Express $P(T>1)$ as a mixture of standard-normal tail probabilities and give its value to four decimals. (Answer: $\frac12P(G>\sqrt2-1)+\frac12P(G>\sqrt2+1)\approx0.1736$.)

What You'll Learn