Tutorials › Statistical Theory I - Probability and Distribution Theory › A Characteristic-Function Proof of the IID Central Limit Theorem

Home  /  Tutorials  /  Central Limit Theorems

Central Limit Theorems · Tutorial 848 of 994

A Characteristic-Function Proof of the IID Central Limit Theorem

Learn how a finite second moment controls the characteristic function near zero and forces normalized iid sums to have a standard normal limit.

Advanced 10 min read

What You'll Learn

  • Expand the characteristic function of a centered, variance-one random variable near zero using only a finite second moment.
  • Raise the small-argument expansion to the sample-size power and justify the resulting exponential limit.
  • Apply Lévy’s continuity theorem to conclude convergence to the standard normal distribution.
  • Follow the proof for Bernoulli, exponential, and Rademacher observations.
  • Identify the scaling and centering errors that commonly spoil characteristic-function proofs.

Why the Characteristic Function Gives a Proof

The iid central limit theorem says that averages of independent observations become approximately normal after centering and scaling. A characteristic-function proof reveals the mechanism: finite variance determines the first two terms in the characteristic function near zero, and independence turns those terms into a power that converges to the normal characteristic function.

The theorem itself is the Lindeberg–Levy central limit theorem. Here the emphasis is on proving it through characteristic functions, with particular care about the second-order expansion and the changing argument \(t/\sqrt{n}\). Those details are what allow the proof to work under a finite-variance assumption without requiring a third moment.

Definition: The characteristic function of a real-valued random variable \(X\) is \(\varphi_X(t)=E[e^{itX}]\), for \(t\in\mathbb{R}\), where \(i^2=-1\). It always exists, since \(|e^{itX}|=1\). If \(X\) and \(Y\) are independent, then \(\varphi_{X+Y}(t)=\varphi_X(t)\varphi_Y(t)\).

For a sequence of random variables, we will use the Lévy continuity theorem: if their characteristic functions converge pointwise to a function that is itself a characteristic function and is continuous at zero, then the random variables converge in distribution to the law with that limiting characteristic function. The characteristic function \(e^{-t^2/2}\) is that of \(N(0,1)\).

The Small-Argument Expansion

First standardize one observation. Suppose \(X_1,X_2,\ldots\) are iid with \(E[X_1]=\mu\) and \(0<\operatorname{Var}(X_1)=\sigma^2<\infty\), and define \(Y_j=(X_j-\mu)/\sigma\). Then \(E[Y_j]=0\) and \(E[Y_j^2]=1\). The crucial expansion is

$$ \varphi_Y(u)=1-\frac{u^2}{2}+o(u^2),\qquad u\to 0. $$

The notation \(o(u^2)\) means a remainder whose ratio to \(u^2\) tends to zero as \(u\to0\). To establish the expansion under only a second-moment condition, it is not enough to apply a formal Taylor series inside an expectation. The remainder must be controlled by an integrable random variable.

Lemma: If \(E[Y]=0\) and \(E[Y^2]=1\), then \(\varphi_Y(u)=1-u^2/2+o(u^2)\) as \(u\to0\).

Proof. For real \(v\), set \(r(v)=e^{iv}-1-iv+v^2/2\). Taylor’s formula gives \(r(v)/v^2\to0\) as \(v\to0\). Also, there is a finite constant \(C\) such that \(|r(v)|\le C v^2\) for all real \(v\): near zero this follows from the Taylor remainder, and for \(|v|\ge1\) it follows by bounding each term in the definition of \(r\) by a constant times \(v^2\). For \(u\ne0\),

$$ \frac{\varphi_Y(u)-1-iuE[Y]+(u^2/2)E[Y^2]}{u^2} =E\left[\frac{r(uY)}{u^2}\right]. $$

For each fixed value of \(Y\), \(r(uY)/u^2\to0\) as \(u\to0\). Moreover, \(|r(uY)/u^2|\le C Y^2\), and \(E[Y^2]=1<\infty\). The dominated convergence theorem therefore shows that the expectation on the right tends to zero. Substituting \(E[Y]=0\) and \(E[Y^2]=1\) proves the expansion. \(\square\)

From the Expansion to a Normal Limit

Theorem: Let \(X_1,X_2,\ldots\) be iid random variables with finite mean \(\mu\) and variance \(0<\sigma^2<\infty\). Then $$ \frac{\sum_{j=1}^n X_j-n\mu}{\sigma\sqrt n}\xrightarrow{d}N(0,1). $$

Proof. Write \(Y_j=(X_j-\mu)/\sigma\) and \(Z_n=n^{-1/2}\sum_{j=1}^nY_j\). The variables \(Y_j\) are iid with mean zero and variance one. Independence gives the characteristic function of \(Z_n\) as

$$ \varphi_{Z_n}(t) =\left\{\varphi_Y\left(\frac{t}{\sqrt n}\right)\right\}^{n}. $$

For each fixed \(t\), the lemma implies

$$ \varphi_Y\left(\frac{t}{\sqrt n}\right) =1-\frac{t^2}{2n}+o\left(\frac1n\right). $$

When \(t=0\), the characteristic function is exactly one. For fixed nonzero \(t\), write the expression inside braces as \(1+w_n\), where \(w_n=-t^2/(2n)+o(1/n)\). Thus \(nw_n\to-t^2/2\) and \(n|w_n|^2\to0\). For complex \(w\) sufficiently close to zero, the logarithm defined near \(1\) satisfies \(\log(1+w)=w+O(|w|^2)\). Consequently,

$$ n\log(1+w_n)=nw_n+O(n|w_n|^2)\longrightarrow-\frac{t^2}{2}, \qquad (1+w_n)^n\longrightarrow e^{-t^2/2}. $$

This gives pointwise convergence \(\varphi_{Z_n}(t)\to e^{-t^2/2}\) for every real \(t\). To identify the limit, the standard normal density is \((2\pi)^{-1/2}e^{-x^2/2}\). Its characteristic function is \(e^{-t^2/2}\): differentiating its defining integral with respect to \(t\) is justified by the integrability of \(|x|e^{-x^2/2}\); integration by parts then gives \(\varphi'(t)=-t\varphi(t)\), with \(\varphi(0)=1\). Solving this differential equation yields the claimed characteristic function. It is continuous at zero, so the Lévy continuity theorem gives \(Z_n\xrightarrow{d}N(0,1)\). \(\square\)

The proof has two distinct inputs. The finite second moment produces the quadratic expansion, while independence produces the \(n\)th power. The scaling by \(\sqrt n\) makes the quadratic term of each factor order \(1/n\), so its accumulated contribution remains finite rather than disappearing or diverging.

Worked Examples

Worked Example: Bernoulli Observations with \(p=1/4\)

Let \(X_j\) be iid Bernoulli random variables with \(P(X_j=1)=1/4\) and \(P(X_j=0)=3/4\). Their mean and variance are

$$ \mu=\frac14,\qquad \sigma^2=E[X_j^2]-\mu^2=\frac14-\frac1{16}=\frac3{16}. $$

Thus \(\sigma=\sqrt3/4\). The centered, standardized variable \(Y_j=(X_j-1/4)/(\sqrt3/4)\) takes the values \(\sqrt3\) with probability \(1/4\) and \(-1/\sqrt3\) with probability \(3/4\). Direct calculation verifies \(E[Y_j]=(\sqrt3/4)-(3/(4\sqrt3))=0\) and \(E[Y_j^2]=3/4+(3/4)(1/3)=1\).

Its characteristic function is \(\varphi_Y(u)=\tfrac14e^{iu\sqrt3}+\tfrac34e^{-iu/\sqrt3}\). Expanding the two exponentials, the linear term is \(iuE[Y]=0\), and the quadratic term is \(-u^2E[Y^2]/2=-u^2/2\). Since \(Y\) takes only two finite values, the remaining Taylor terms are \(O(|u|^3)\), hence \(o(u^2)\). The theorem therefore yields

$$ \frac{\sum_{j=1}^n X_j-n/4}{(\sqrt3/4)\sqrt n} \xrightarrow{d}N(0,1). $$

Equivalently, if \(B_n=\sum_{j=1}^nX_j\), then \((B_n-n/4)/\sqrt{3n/16}\) converges in distribution to a standard normal random variable.

Worked Example: Exponential Observations with Rate \(2\)

Let \(X_j\) be iid exponential random variables with rate \(2\), so their density is \(2e^{-2x}\) for \(x\ge0\). The mean is \(1/2\) and the variance is \(1/4\), so \(Y_j=(X_j-1/2)/(1/2)=2X_j-1\) has mean zero and variance one. The characteristic function of \(X_j\) is \(2/(2-it)\); therefore

$$ \varphi_Y(u)=e^{-iu}\frac{2}{2-2iu} =\frac{e^{-iu}}{1-iu}. $$

For small \(u\), use \(\log(1-iu)=-iu+u^2/2+O(|u|^3)\). Then

$$ \log\varphi_Y(u) =-iu-\log(1-iu) =-\frac{u^2}{2}+O(|u|^3). $$

At \(u=t/\sqrt n\), multiplying this logarithm by \(n\) gives \(-t^2/2+O(|t|^3/\sqrt n)\), which tends to \(-t^2/2\). Hence the characteristic function of \(n^{-1/2}\sum_{j=1}^nY_j\) tends to \(e^{-t^2/2}\). In the original units, the conclusion is

$$ \frac{\sum_{j=1}^nX_j-n/2}{\sqrt n/2} \xrightarrow{d}N(0,1). $$

This calculation reaches the same limit directly for this particular distribution. The general proof does not require an explicit characteristic function or a third moment.

Worked Example: Rademacher Observations

Let \(X_j\) be iid with \(P(X_j=1)=P(X_j=-1)=1/2\). Then \(\mu=0\), \(\sigma^2=1\), and the characteristic function is \(\varphi_X(u)=\cos u\). The Taylor expansion \(\cos u=1-u^2/2+O(u^4)\) gives

$$ \varphi_{n^{-1/2}\sum_{j=1}^nX_j}(t) =\left\{\cos\left(\frac{t}{\sqrt n}\right)\right\}^{n} =\left\{1-\frac{t^2}{2n}+O\left(\frac1{n^2}\right)\right\}^{n} \longrightarrow e^{-t^2/2}. $$

The limiting characteristic function is that of \(N(0,1)\), so \(n^{-1/2}\sum_{j=1}^nX_j\xrightarrow{d}N(0,1)\). This example also shows that the observations need not themselves have a continuous or approximately normal distribution.

Common Pitfall: Scaling the Wrong Argument

A frequent error is to write the characteristic function of the normalized sum as \(\{\varphi_Y(t)\}^n\). That is the characteristic function of the unscaled sum \(\sum_{j=1}^nY_j\), not of \(n^{-1/2}\sum_{j=1}^nY_j\). Scaling a random variable by \(1/\sqrt n\) changes the argument of its characteristic function, giving \(\{\varphi_Y(t/\sqrt n)\}^n\). With the correct argument, each factor is \(1-t^2/(2n)+o(1/n)\); its \(n\)th power has a finite nontrivial limit. Using \(t\) instead would raise a fixed factor to the \(n\)th power and cannot establish the desired normal limit.

A second related mistake is to forget centering. The expansion has a linear term \(iuE[Y]\), which vanishes only after subtracting the mean. If the observations are not centered, that linear term changes the behavior of the \(n\)th power. Similarly, standardizing by \(\sigma\) is essential: it makes the quadratic coefficient \(-u^2/2\), matching the characteristic function of \(N(0,1)\).

Key takeaway: For iid observations with finite, positive variance, centering and scaling reduce the problem to mean-zero, variance-one variables. Their characteristic function is \(1-u^2/2+o(u^2)\); independence and the argument \(t/\sqrt n\) turn this into the limiting characteristic function \(e^{-t^2/2}\).

Check Your Understanding

Use the expansion and the scaling argument to check each conclusion.

  1. Why does finite variance, rather than a finite third moment, suffice for the small-argument expansion used in the proof?
  2. Write the characteristic function of \(n^{-1/2}\sum_{j=1}^nY_j\) in terms of \(\varphi_Y\), assuming the \(Y_j\) are iid.
  3. For Bernoulli observations with success probability \(p\), what are the mean and variance, and for which values of \(p\) is the theorem’s positive-variance assumption satisfied?
  4. In the exponential example, explain why the linear terms in \(-iu-\log(1-iu)\) cancel.
  5. What goes wrong if one uses \(\{\varphi_Y(t)\}^n\) in place of \(\{\varphi_Y(t/\sqrt n)\}^n\)?

Previous

Next

Back to Tutorials