Tutorials › Statistical Theory I - Probability and Distribution Theory › When the Classical CLT Fails

Home  /  Tutorials  /  Central Limit Theorems

Central Limit Theorems · Tutorial 869 of 994

When the Classical CLT Fails

Diagnose when the usual square-root-of-sample-size normal approximation is unreliable, and check which assumptions fail.

Advanced 11 min read

What You'll Learn

  • Identify why finite variance matters for the classical iid CLT.
  • Derive the behavior of sums of Cauchy and stable random variables.
  • Check the Lindeberg and Feller conditions for independent arrays.
  • See how a rare, high-variance summand can change the limiting distribution.
  • Distinguish heavy tails with finite variance from infinite-variance failures.

What “Failure” Means

The classical iid central limit theorem gives a standard normal limit for a sum of independent, identically distributed random variables with a finite, positive variance. In its usual form, if \(X_1,X_2,\ldots\) have mean \(\mu\) and variance \(0<\sigma^2<\infty\), then

$$ \frac{\sum_{i=1}^n X_i-n\mu}{\sigma\sqrt{n}} \xrightarrow{d} N(0,1). $$

A failure means that this usual centering, square-root-of-\(n\) scaling, and normal limit do not apply. It does not mean that every normal approximation is useless, nor that every heavy-tailed model fails the theorem. The key questions are whether the variance exists and, for independent arrays, whether the variance is spread across many small contributions rather than concentrated in a few.

Definition: For independent, mean-zero variables \(X_{n1},\ldots,X_{nk_n}\) in row \(n\), write \(s_n^2=\sum_{j=1}^{k_n}\operatorname{Var}(X_{nj})\), with \(0<s_n^2<\infty\). The Lindeberg condition is that, for every \(\varepsilon>0\),
$$ \frac{1}{s_n^2}\sum_{j=1}^{k_n} E\!\left[X_{nj}^2\mathbf{1}\{|X_{nj}|>\varepsilon s_n\}\right] \longrightarrow 0. $$
This condition is sufficient for the Lindeberg–Feller central limit theorem: the row sum divided by \(s_n\) converges in distribution to \(N(0,1)\). The Feller condition is \(\max_j\operatorname{Var}(X_{nj})/s_n^2\to0\); it is implied by Lindeberg, but its failure alone does not rule out a normal limit.

These array conditions help diagnose where the approximation breaks down. Infinite variance prevents using the classical iid theorem with its variance normalization. In arrays with finite row variances, a few summands may instead carry a nonvanishing share of the total variance. The examples below show both mechanisms and also clarify why “heavy-tailed” is not, by itself, a diagnosis.

Infinite Variance: The Cauchy Example

Worked Example: A Sample Mean That Does Not Settle Down

Let \(X_1,\ldots,X_n\) be independent standard Cauchy variables, whose characteristic function is \(\varphi_X(t)=e^{-|t|}\). This distribution has no finite mean or variance, so there is no \(\mu\) or \(\sigma^2\) to insert into the classical iid CLT.

Independence gives the characteristic function of their sum \(S_n=\sum_{i=1}^n X_i\):

$$ \varphi_{S_n}(t)=\prod_{i=1}^n\varphi_X(t) =e^{-n|t|}. $$

The characteristic function of \(S_n/n\) is therefore

$$ \varphi_{S_n/n}(t)=\varphi_{S_n}(t/n) =e^{-|t|}. $$

Thus the sample mean has the same standard Cauchy distribution for every \(n\); it does not concentrate around a finite population mean. Moreover, \(\sqrt n\,S_n/n\) has a Cauchy distribution with scale \(\sqrt n\). For any fixed \(M>0\),

$$ P\!\left(\left|\frac{S_n}{\sqrt n}\right|\le M\right) =\frac{2}{\pi}\arctan\!\left(\frac{M}{\sqrt n}\right) \longrightarrow 0. $$

So the square-root-scaled sum is not even tight: its probability of lying in any fixed bounded interval tends to zero. A standard normal limit is impossible.

Heavy Tails and Alternative Scaling

Infinite-variance distributions can have a different scaling and a different limiting law. A symmetric stable distribution gives a particularly transparent calculation. For \(1<\alpha<2\), let \(X\) have characteristic function \(\varphi_X(t)=e^{-|t|^\alpha}\). Such a variable has finite mean \(E[X]=0\) but infinite variance; its absolute moment of order \(p\) is finite exactly when \(0<p<\alpha\).

Worked Example: A Finite-Mean Stable Sum

Take \(\alpha=3/2\) and let \(X_1,\ldots,X_n\) be iid with this symmetric stable distribution. Independence gives

$$ \varphi_{S_n}(t)=e^{-n|t|^{3/2}} =\varphi_X\!\left(n^{2/3}t\right), \qquad S_n=\sum_{i=1}^nX_i. $$

Equality of characteristic functions shows that \(S_n\) has the same distribution as \(n^{2/3}X\). Consequently, the sample mean has the distribution of \(n^{-1/3}X\), while the usual square-root-scaled sum has the distribution of \(n^{1/6}X\):

$$ \frac{S_n}{n}\ \overset{d}{=}\ n^{-1/3}X, \qquad \frac{S_n}{\sqrt n}\ \overset{d}{=}\ n^{1/6}X. $$

The first identity shows concentration around the finite mean zero at a rate different from \(n^{-1/2}\). For any fixed \(M>0\), \(P(|S_n/\sqrt n|\le M)=P(|X|\le M n^{-1/6})\to0\), because this stable distribution is continuous and has no point mass at zero. Thus the usual normalization does not yield a normal limit. The calculation does not say that all heavy-tailed sums behave this way; it exhibits one precise infinite-variance alternative.

The contrast with finite variance is essential. A distribution can have a long, heavy tail and still have a finite second moment. In that case, the iid CLT still applies, although a normal approximation at a particular sample size can be inaccurate. Whether convergence is fast enough for a practical approximation is a separate question from whether the theorem applies.

Worked Example: A Heavy-Tailed Model That Still Satisfies the CLT

Let \(X_1,\ldots,X_n\) be iid Student \(t\) variables with three degrees of freedom. This distribution has mean zero and variance \(3\); its tails are heavier than those of a normal distribution, but its second moment is finite. The Lindeberg expression for the standardized sum is

$$ \frac{1}{3n}\sum_{i=1}^n E\!\left[X_i^2\mathbf{1}\{|X_i|>\varepsilon\sqrt{3n}\}\right] = \frac{1}{3}E\!\left[X_1^2 \mathbf{1}\{|X_1|>\varepsilon\sqrt{3n}\}\right]. $$

As \(n\to\infty\), the threshold tends to infinity. The integrand tends pointwise to zero and is bounded by \(X_1^2\), which has expectation \(3<\infty\). Dominated convergence makes the displayed expression tend to zero. Hence the Lindeberg condition holds, and the standardized sum converges to \(N(0,1)\). Heavy tails can slow practical convergence, but here they do not invalidate the asymptotic CLT.

When One Summand Carries Too Much Variance

For independent arrays, checking that every row has finite total variance is not enough. The Lindeberg condition controls large summands on the scale of the row’s total standard deviation. The Feller condition provides a simpler warning sign: if it fails, some summand retains a nonvanishing share of total variance. That does not automatically force a nonnormal limit; for example, a sum of independent normal variables is normal even if one contributes a large fraction of the variance. But a dominant, rarely nonzero summand can make the variance normalization misleading.

Worked Example: A Rare Jump Prevents a Standard Normal Limit

In row \(n\), take \(n\) independent variables \(G_{n1},\ldots,G_{nn}\), each distributed as \(N(0,1/n)\). Independently, take \(J_n\) with probabilities

$$ P(J_n=\sqrt n)=\frac{1}{2n},\qquad P(J_n=-\sqrt n)=\frac{1}{2n},\qquad P(J_n=0)=1-\frac{1}{n}. $$

All summands have mean zero. The Gaussian variables have total variance \(n(1/n)=1\), and \(E[J_n^2]=n(1/n)=1\), so the row total variance is \(s_n^2=2\). The Feller ratio is

$$ \frac{\max(1/n,1)}{2}=\frac12, $$

which does not tend to zero. For any fixed \(\varepsilon>0\) and all sufficiently large \(n\), \(\sqrt n>\varepsilon\sqrt2\), so the jump’s contribution to the Lindeberg expression is

$$ \frac{1}{2}E\!\left[J_n^2 \mathbf{1}\{|J_n|>\varepsilon\sqrt2\}\right] =\frac12. $$

Thus Lindeberg fails. To see the actual limit, put \(G_n=\sum_{j=1}^nG_{nj}\); then \(G_n\sim N(0,1)\) for every \(n\). Also \(P(J_n\ne0)=1/n\), so \(J_n\to0\) in probability. The normalized row sum is \((G_n+J_n)/\sqrt2\). For any \(\delta>0\), its distribution function at \(x\) is bounded below by \(P(G_n/\sqrt2\le x-\delta)-P(|J_n|/\sqrt2>\delta)\) and above by \(P(G_n/\sqrt2\le x+\delta)+P(|J_n|/\sqrt2>\delta)\). First let \(n\to\infty\), so the error probabilities vanish; then let \(\delta\downarrow0\), using continuity of the normal distribution function. The normalized sum therefore converges in distribution to \(N(0,1/2)\), not \(N(0,1)\). Its variance is one in every row, but the rare jumps carry half that variance and disappear in probability.

Diagnostic Checks and a Common Pitfall

When a normal approximation looks suspect, separate the distributional claim from the practical accuracy of an approximation. For iid variables, check whether the mean and variance exist before using the classical CLT. For an independent array, identify the total variance, examine whether large summands contribute a vanishing fraction of it, and check Lindeberg when applying the Lindeberg–Feller theorem. If those checks fail, do not infer a particular alternative limit without additional assumptions.

  • Pitfall: “The distribution is heavy-tailed, so the CLT fails.” This is wrong: the \(t_3\) example has heavy tails and finite variance, and its standardized iid sum satisfies the CLT.
  • Correction: Check the second moment, not the tail’s appearance alone. Infinite variance rules out the classical finite-variance theorem, while finite variance lets that theorem apply to iid observations.
  • Pitfall: “Each summand has finite variance, so the array sum is asymptotically standard normal.” Finite row variances alone do not prevent rare summands from carrying a fixed share of total variance.
  • Correction: Check the array conditions at the scale \(s_n\). A failed condition warns that the standard Lindeberg–Feller argument does not apply; it is not, by itself, proof that every possible normal limit is impossible.
Key takeaway: The classical normal approximation depends on more than independence and a large sample size. For iid sums, finite variance is central; for independent arrays, the Lindeberg condition prevents a nonvanishing variance contribution from hiding in rare, large summands. Diagnose the mechanism before choosing a normalization or claiming a limit.

Check Your Understanding

Use the assumptions and calculations in this tutorial to answer the following.

  1. Why does the standard Cauchy sample mean not converge to a finite population mean as the sample size grows?
  2. For the symmetric stable example with \(\alpha=3/2\), what is the scale of \(S_n\), and why does \(S_n/\sqrt n\) fail to have a standard normal limit?
  3. Why does the \(t_3\) example satisfy the Lindeberg condition despite its heavy tails?
  4. In the rare-jump array, what fraction of the total row variance comes from \(J_n\), and what is the limiting distribution after dividing the row sum by \(s_n\)?
  5. Does failure of the Feller condition alone prove that a sum cannot have a normal limit? Explain.

Previous

Next

Back to Tutorials