The Lindeberg–Levy Central Limit Theorem

Why an IID Sum Becomes Approximately Normal

A sum of many independent observations can have a distribution that is much simpler than the distribution of any one observation. The Lindeberg–Levy central limit theorem makes this idea precise: after subtracting the sum’s mean and dividing by its standard deviation, the sum converges in distribution to a standard normal random variable. The theorem applies whether the original observations are discrete or continuous, provided they are identically distributed, independent, and have a finite, positive variance.

The scaling matters. If each observation has variance $\sigma^2$, the variance of a sum of $n$ independent observations is $n\sigma^2$, so its standard deviation is $\sigma\sqrt n$. Dividing by $n\sigma$, or by $\sigma$ alone, would not produce the standardization in the theorem.

Statement of the Lindeberg–Levy Theorem

Definition: A sequence $X_1,X_2,\ldots$ is iid if its random variables are independent and have the same distribution. Write $E[X_i]=\mu$ and $\operatorname{Var}(X_i)=\sigma^2$ when these moments exist. The sample average is $\bar X_n=n^{-1}\sum_{i=1}^n X_i$.

Throughout, convergence in distribution, written $\xrightarrow{d}$, means convergence of distribution functions at every continuity point of the limiting distribution function. Let $\Phi$ denote the standard normal distribution function, so $\Phi(x)=P(Z\leq x)$ for $Z\sim N(0,1)$.

Theorem (Lindeberg–Levy central limit theorem): Suppose $X_1,X_2,\ldots$ are iid random variables with $E[X_1]=\mu$ and $\operatorname{Var}(X_1)=\sigma^2$, where $0<\sigma^2<\infty$. Then $$ \frac{\sum_{i=1}^n X_i-n\mu}{\sigma\sqrt n} = \frac{1}{\sigma\sqrt n}\sum_{i=1}^n(X_i-\mu) \xrightarrow{d}N(0,1) \qquad\text{as }n\to\infty. $$

The theorem needs no assumption that the $X_i$ are themselves normally distributed, and it does not require moments of order three or higher. Its conditions do matter: the observations must be iid, and their common variance must be finite and strictly positive. The theorem does not say that the sum is exactly normal for a given finite $n$.

What the Theorem Says About Sample Averages

The sample average is the sum divided by $n$. Rewriting the theorem’s standardized sum gives the corresponding result for averages.

Corollary (central limit theorem for the sample average): Under the hypotheses of the Lindeberg–Levy theorem, $$ \frac{\sqrt n(\bar X_n-\mu)}{\sigma}\xrightarrow{d}N(0,1). $$ Equivalently, the centered average scaled by $\sqrt n$ satisfies $\sqrt n(\bar X_n-\mu)\xrightarrow{d}N(0,\sigma^2)$.

Derivation. By the definition of $\bar X_n$, $$ \frac{\sqrt n(\bar X_n-\mu)}{\sigma} = \frac{\sqrt n}{\sigma}\left(\frac{1}{n}\sum_{i=1}^nX_i-\mu\right) = \frac{1}{\sigma\sqrt n}\sum_{i=1}^n(X_i-\mu). $$ The expression on the right is exactly the standardized sum in the theorem, so it converges in distribution to $N(0,1)$. Multiplying that standardized expression by $\sigma$ gives the equivalent limit $N(0,\sigma^2)$. $\square$

For large $n$, this result is often expressed informally by saying that $\bar X_n$ is approximately $N(\mu,\sigma^2/n)$. This is a useful description of its approximate center and spread, not a claim that $\bar X_n$ has an exact normal distribution for finite $n$.

The theorem also yields probability approximations for intervals. For any fixed real numbers $a<b$, $$ P\left(a\leq\frac{\sqrt n(\bar X_n-\mu)}{\sigma}\leq b\right) \longrightarrow \Phi(b)-\Phi(a). $$ To justify the interval statement, convergence in distribution gives convergence of the distribution functions at $a$ and $b$, because the standard normal distribution function is continuous everywhere. In particular, the probabilities of the endpoints converge to zero: for any $\varepsilon>0$, the probability of a single endpoint is bounded by the probability of an interval of width $2\varepsilon$ around it, whose limiting probability tends to zero as $\varepsilon\downarrow0$. Thus using either open or closed interval endpoints gives the same limit. $\square$

In an application, translate the event about $\bar X_n$ into an event about the standardized variable, then use $\Phi$. This calculation uses the population values $\mu$ and $\sigma$ in the theorem; it is not automatically a statement about replacing them by estimates.

Worked Applications

Worked Example: A Sample Proportion From Bernoulli Observations

Suppose $X_1,\ldots,X_{100}$ are iid Bernoulli random variables with $P(X_i=1)=0.5$. Let $\bar X_{100}$ be the observed proportion of ones. Approximate $P(0.40\leq\bar X_{100}\leq0.60)$.

For a Bernoulli variable with success probability $p$, $E[X_i]=p$ and $\operatorname{Var}(X_i)=p(1-p)$. Here $\mu=0.5$, $\sigma^2=0.25$, and $\sigma=0.5$. The standardized average is therefore $$ \frac{\sqrt{100}(\bar X_{100}-0.5)}{0.5}. $$ At the lower endpoint $0.40$, this expression is $10(-0.10)/0.5=-2$. At the upper endpoint $0.60$, it is $10(0.10)/0.5=2$. The CLT approximation is $$ P(-2\leq Z\leq2)=\Phi(2)-\Phi(-2)=2\Phi(2)-1\approx 0.9545, $$ where $Z\sim N(0,1)$ and $\Phi(2)\approx0.97725$. Thus the approximate probability is $0.9545$. This is an approximation, not the exact binomial probability for $n=100$.

Worked Example: An Average of Exponential Observations

Let $X_1,\ldots,X_{64}$ be iid exponential random variables with rate $1/2$, so each has mean $2$ and variance $4$. Approximate $P(\bar X_{64}>2.5)$.

The standard deviation of one observation is $\sigma=2$. The standardized threshold for the average is $$ \frac{\sqrt{64}(2.5-2)}{2} = \frac{8(0.5)}{2} = 2. $$ The CLT gives $$ P(\bar X_{64}>2.5) \approx P(Z>2) =1-\Phi(2) \approx 1-0.97725 =0.02275. $$ The approximation is about $0.0228$. The exponential distribution is skewed, but the theorem still applies because its mean and variance are finite and the observations are iid. The theorem guarantees convergence as the sample size grows; it does not give a universal accuracy guarantee at $n=64$.

Worked Example: When the Normal Approximation Is Exact

Suppose $X_1,\ldots,X_{25}$ are iid $N(10,4)$, where the second parameter is the variance. Find $P(9.2\leq\bar X_{25}\leq10.8)$.

Here $\mu=10$, $\sigma=2$, and $\sqrt n=5$. Standardizing both endpoints gives $$ \frac{5(9.2-10)}{2}=-2 \qquad\text{and}\qquad \frac{5(10.8-10)}{2}=2. $$ Because independent normal random variables have a normal sum, the sample average is exactly normal in this example. Consequently, $$ P(9.2\leq\bar X_{25}\leq10.8) = P(-2\leq Z\leq2) = 2\Phi(2)-1 \approx0.9545. $$ This is an exact calculation for the specified normal model, unlike the approximations in the first two examples.

Common Mistakes and Practical Interpretation

Using the wrong scale. The standard deviation of a sum is $\sigma\sqrt n$; the standard deviation of an average is $\sigma/\sqrt n$. For example, $\sqrt n(\bar X_n-\mu)/\sigma$ is dimensionless and has the standard normal limit.
Assuming the observations must be normal. Normality of the observations is sufficient for an exact normal distribution of the average, but it is not required for the iid central limit theorem.
Ignoring the assumptions. A finite positive variance and the iid sampling structure are part of this theorem. A similar-looking expression for dependent or non-identically distributed data requires a result whose assumptions cover that situation.
Treating the approximation as exact. For nonnormal observations, the theorem describes a limiting distribution. At a particular sample size, approximation quality depends on the distribution and the event being considered.
Rounding intermediate normal probabilities too early. Carry sufficient precision through the subtraction. For instance, $2\Phi(1.25)-1\approx2(0.8943502)-1=0.7887004$, which rounds to $0.7887$, not $0.7888$.

The central limit theorem is useful because it turns a probability question about a sum or average into a standard-normal calculation. Its practical role is distinct from a law of large numbers: a law of large numbers describes convergence of the average toward its mean, while the CLT describes the distribution of the centered error on the $\sqrt n$ scale.

Key takeaway: For iid observations with common mean $\mu$ and finite positive variance $\sigma^2$, center the sum at $n\mu$ and divide by $\sigma\sqrt n$, or center the average at $\mu$ and multiply by $\sqrt n/\sigma$. The resulting quantity converges in distribution to $N(0,1)$.

Check Your Understanding

For each question, identify the relevant assumptions and show how the quantity should be standardized.

If iid observations have mean $3$ and variance $9$, what expression involving their sum converges in distribution to $N(0,1)$?
If the same observations have sample average $\bar X_n$, what is the corresponding standardized expression for the average?
For iid Bernoulli observations with success probability $0.2$, state the mean and variance needed to apply the theorem.
Does the Lindeberg–Levy theorem assert that an average of finitely many nonnormal observations is exactly normal? Explain.
Why is dividing a centered sum by $n\sigma$, rather than $\sigma\sqrt n$, not the standardization in the theorem?