A Proof Sketch of the Central Limit Theorem

Why a Proof Sketch Helps

The Central Limit Theorem (CLT) says that, under specific conditions, standardized sums of independent observations become approximately normal as the number of observations grows. The earlier material on the Central Limit Theorem states this result and examines when the approximation is useful. Here we look inside its proof: the characteristic function of one standardized observation has a particular form near zero, and independence raises that function to the \(n\)th power.

The key point is that finite variance is enough to determine the leading terms of that small-argument form. The distribution of an individual observation need not itself be normal, symmetric, or continuous.

The Theorem and Its Assumptions

The conditions matter. The observations must be independent and identically distributed, and their common distribution must have finite variance. The positive-variance condition allows division by \(\sigma\); if the variance is zero, every observation equals \(\mu\) with probability 1, so there is no nondegenerate standardized sum.

For the proof, define a standardized observation \(Y=(X-\mu)/\sigma\). It has \(E(Y)=0\) and \(E(Y^2)=1\). The corresponding standardized sum is \(Z_n=(Y_1+\cdots+Y_n)/\sqrt{n}\), where the \(Y_i\) are independent copies of \(Y\). We will show that the characteristic function of \(Z_n\) approaches the characteristic function of a standard normal variable.

Characteristic Functions and the Key Expansion

Two facts make characteristic functions useful here. First, if \(Y_1,\ldots,Y_n\) are independent, the characteristic function of their sum is the product of their individual characteristic functions. Second, the characteristic function of a standard normal random variable is \(e^{-t^2/2}\).

The central technical step is the expansion \[ \phi_Y(t)=1-\frac{t^2}{2}+o(t^2)\quad\text{as }t\to0. \] Here \(o(t^2)\) denotes a remainder which, divided by \(t^2\), tends to zero as \(t\) tends to zero. This expansion uses only \(E(Y)=0\) and \(E(Y^2)=1\), not a third moment.

Why Finite Variance Gives the Expansion

The expansion of \(e^{iu}\) near zero begins \(1+iu-u^2/2\). To apply that fact when \(Y\) might be unbounded, write the remainder in a form that can be controlled by \(Y^2\): \[ r(u)=\frac{e^{iu}-1-iu+u^2/2}{u^2}\quad (u\ne0),\qquad r(0)=0. \] Taylor expansion near zero shows \(r(u)\to0\) as \(u\to0\). Also, \(r\) is bounded: near zero this follows from the Taylor remainder, and for large \(|u|\) the numerator is bounded by a constant times \(u^2\).

Substitute \(u=tY\), take expectations, and use \(E(Y)=0\) and \(E(Y^2)=1\):

For each fixed value of \(Y\), \(r(tY)\to0\) as \(t\to0\). Since \(r\) is bounded, \(|Y^2r(tY)|\le C Y^2\) for some finite constant \(C\); the bound is integrable because \(E(Y^2)=1\). The dominated convergence theorem therefore gives \(E[Y^2r(tY)]\to0\). This proves the stated expansion. The finite second moment is exactly what lets us control the expected remainder.

Independence Turns the Expansion into a Limit

Because \(Z_n=(Y_1+\cdots+Y_n)/\sqrt{n}\), its characteristic function at \(t\) is the product of the \(n\) characteristic functions evaluated at \(t/\sqrt{n}\). Independence is the reason this product rule applies:

For each fixed \(t\), the argument \(t/\sqrt{n}\) tends to zero. Use the expansion just proved: \[ \phi_Y\!\left(\frac{t}{\sqrt{n}}\right) =1-\frac{t^2}{2n}+o(1/n). \] The term \(o(1/n)\) here is for fixed \(t\) as \(n\) grows. Raising to the \(n\)th power gives the limit:

To see the mechanism, write the bracket as \(1+a_n/n\), where \(a_n\to-t^2/2\). The standard exponential limit \((1+a_n/n)^n\to e^{-t^2/2}\) then applies (also for these complex-valued terms). The result \(e^{-t^2/2}\) is the characteristic function of the standard normal distribution.

The final step uses the continuity theorem for characteristic functions: if a sequence of characteristic functions converges pointwise to a characteristic function that is continuous at zero, the associated distributions converge in distribution to the distribution represented by that limit. Since \(e^{-t^2/2}\) is continuous at zero and represents the standard normal distribution, \(Z_n\) converges in distribution to the standard normal. This completes the proof sketch.

What the Proof Does—and Does Not—Say

The proof explains why the normal distribution appears: after centering and scaling, the first-order term vanishes because the mean is zero, while the second-order term is set by the variance. At the small argument \(t/\sqrt{n}\), that second-order contribution is of size \(1/n\); accumulating it across \(n\) independent factors produces the exponential form \(e^{-t^2/2}\).

This is a statement about convergence in distribution as \(n\) grows. It does not say that a finite sum is exactly normal, that every finite sample size gives a highly accurate approximation, or that the original observations become normal. Accuracy for a particular distribution and sample size depends on more than the limiting argument alone.

The independence assumption also has a precise role: it turns the characteristic function of a sum into a product. If observations are dependent, that product calculation generally fails, and this proof does not establish the same conclusion. Likewise, finite variance is not a minor technical detail. It is used to obtain the second-order expansion; when variance is infinite, this proof cannot be applied.

Common Mistake

• Wrong reasoning: “The CLT says that, once \(n\) is large, the observations themselves have a normal distribution.”
• Correction: The theorem concerns the distribution of a standardized sum or sample mean across repeated samples. Each observation retains its original distribution. The normal limit arises from the characteristic function of the standardized sum.
• Wrong reasoning: “Since the limiting characteristic function is normal, the finite-\(n\) characteristic function must already equal it.”
• Correction: The proof establishes convergence, not exact equality for finite \(n\). The expansion includes a remainder, and the CLT by itself does not state how large that remainder's effect is for a particular sample size.