Why the MGF Derivation Works
The central limit theorem says that sums of many independent, identically distributed random variables, after centering and scaling, approach a normal distribution. The moment generating function (MGF) gives a compact route to this result: independence turns the MGF of a sum into a power, and the local behavior of a single summand’s MGF determines the limiting power.
This derivation focuses on the normalized sum and its MGF, rather than on using a normal approximation for a particular probability. Its key feature is the expansion near zero of a centered variable with variance one. The linear term vanishes, the quadratic term remains, and the chosen square-root scaling makes that quadratic term persist in the limit.
The MGF assumption is important. A finite mean and variance do not, by themselves, guarantee that an MGF exists near zero. The ordinary IID central limit theorem requires only a finite, positive variance, but the proof developed here uses the stronger MGF condition. When the MGF exists near zero, its derivatives at zero give the moments needed for the expansion.
Centering, Scaling, and the Target Result
Let \(X_1,X_2,\ldots\) be independent and identically distributed random variables with \(E[X_i]=\mu\) and \(\operatorname{Var}(X_i)=\sigma^2\), where \(0<\sigma^2<\infty\). Define \(Z_i=(X_i-\mu)/\sigma\). Then the \(Z_i\) are also independent and identically distributed, \(E[Z_i]=0\), and \(\operatorname{Var}(Z_i)=1\). The normalized sum is \(W_n=(Z_1+\cdots+Z_n)/\sqrt n\).
The IID CLT conclusion is \(W_n\) converges in distribution to \(N(0,1)\). Equivalently, the centered sum of the original variables, divided by its standard deviation \(\sigma\sqrt n\), has that limiting distribution. We will establish the result by showing that the MGF of \(W_n\) tends to the MGF of \(N(0,1)\).
The Single-Summand Expansion
Write \(M_Z(u)=E[e^{uZ}]\) for the MGF of one centered, variance-one summand. In a neighborhood of zero, Taylor expansion gives \(e^{uZ}=1+uZ+u^2Z^2/2+\cdots\). Taking expectations yields \(M_Z(u)=1+uE[Z]+u^2E[Z^2]/2+o(u^2)\). Since \(E[Z]=0\) and \(E[Z^2]=1\), the first nonconstant term is \(u^2/2\).
Here \(o(u^2)\) denotes a remainder whose ratio to \(u^2\) tends to zero as \(u\) tends to zero. The MGF’s existence in a neighborhood of zero justifies the local expansion and the use of its derivatives at zero. The mean and variance assumptions determine the displayed coefficients; higher moments, when available, determine further terms.
The next step is to take a logarithm. If \(q\) tends to zero, then \(\log(1+q)=q+o(q)\). Applying this with \(q=u^2/2+o(u^2)\) gives the equivalent logarithmic expansion. Using the logarithm is useful because the MGF of the normalized sum will be a power.
From One MGF to the Limiting MGF
For any fixed real \(t\), independence makes the MGF of \(W_n\) a product of \(n\) identical MGFs. Each factor evaluates the single-summand MGF at \(t/\sqrt n\). Taking logarithms turns that product into \(n\) times the log MGF, where the expansion above can be applied.
Because the \(Z_i\) are independent, the MGF of their sum is the product of their MGFs. Thus \(M_{W_n}(t)=[M_Z(t/\sqrt n)]^n\).
\(\log M_{W_n}(t)=n\log M_Z(t/\sqrt n)\). For fixed \(t\), the argument \(t/\sqrt n\) tends to zero as \(n\) increases.
Use \(\log M_Z(u)=u^2/2+o(u^2)\) with \(u=t/\sqrt n\). Multiplying by \(n\) gives \(t^2/2+o(1)\).
Exponentiating gives \(M_{W_n}(t)\to e^{t^2/2}\), the MGF of \(N(0,1)\). The MGF convergence theorem then gives convergence in distribution.
The small-remainder step can be seen explicitly. For fixed \(t\), write the remainder as \(r(u)\), where \(r(u)/u^2\to0\). Then \(n r(t/\sqrt n)=t^2[r(t/\sqrt n)/(t/\sqrt n)^2]\to0\). This verifies that the remainder disappears after multiplication by \(n\), rather than merely assuming it does.
A standard MGF convergence theorem states that if a sequence of MGFs converges on an open interval around zero to an MGF that is finite there, then the corresponding distributions converge to the distribution with that limiting MGF. In this setting, for any fixed \(t\), \(t/\sqrt n\) eventually lies in the interval where \(M_Z\) exists. The limiting function \(e^{t^2/2}\) is the MGF of a standard normal random variable and is finite for every real \(t\). Therefore \(W_n\) converges in distribution to \(N(0,1)\).
Worked Expansions
Worked Example: A Symmetric Two-Point Summand
A standardized claim indicator takes the values \(Z=-1\) and \(Z=1\), each with probability \(1/2\). Find its MGF and the first correction to the limiting log MGF for \(W_{25}\) at \(t=0.5\), using the expansion through the fourth power.
Method. Compute the exact single-summand MGF, expand its logarithm near zero, and then substitute \(u=t/\sqrt n\). This tracks the finite-\(n\) correction while avoiding the mistake of treating the limiting MGF as exact at \(n=25\).
The MGF is \(M_Z(u)=(e^{-u}+e^u)/2=\cosh(u)\). Its log expansion is \(\log\cosh(u)=u^2/2-u^4/12+O(u^6)\). For \(W_{25}\), the argument is \(u=0.5/\sqrt{25}=0.1\), so the log MGF through the fourth power is \(25[0.1^2/2-0.1^4/12]\).
The quadratic contribution is \(25(0.005)=0.125\). The fourth-power contribution is \(25(0.0001/12)=0.0002083333\). Subtracting gives the approximated log MGF \(0.1247916667\). Exponentiating gives approximately \(1.1329\) to four decimals. The limiting standard normal MGF at \(t=0.5\) is \(e^{0.5^2/2}=e^{0.125}\approx1.1331\).
As a check, the exact log MGF is \(25\log(\cosh(0.1))\), approximately \(0.1247922\), close to the fourth-power expansion. The exact MGF is consequently approximately \(1.1329\) to four decimals. The small difference from the limiting MGF is consistent with a finite sample size.
Worked Example: A Centered Exponential Summand
Let \(Y\) have an exponential distribution with rate \(1\), so \(E[Y]=1\) and \(\operatorname{Var}(Y)=1\). Set \(Z=Y-1\). For \(W_n\) at \(t=1\), find the log-MGF approximation that retains terms through order \(n^{-1/2}\), and evaluate it for \(n=10{,}000\).
Method. Derive the exact MGF and expand its logarithm. The common mistake to avoid is using \(M_Y\) directly without centering: the CLT derivation requires a zero-mean summand.
For \(u<1\), \(M_Y(u)=1/(1-u)\). Therefore \(M_Z(u)=E[e^{u(Y-1)}]=e^{-u}/(1-u)\), and \(\log M_Z(u)=-u-\log(1-u)\). Using \(-\log(1-u)=u+u^2/2+u^3/3+u^4/4+\cdots\), the linear terms cancel: \(\log M_Z(u)=u^2/2+u^3/3+u^4/4+\cdots\).
For \(W_n\) with \(t=1\), substitute \(u=1/\sqrt n\) and multiply by \(n\). Retaining terms through order \(n^{-1/2}\) gives \(\log M_{W_n}(1)\approx1/2+1/(3\sqrt n)\). At \(n=10{,}000\), \(\sqrt n=100\), so this equals \(0.5+1/300=0.5033333\). The corresponding approximate MGF is \(e^{0.5033333}\approx1.6542\), rounded to four decimals.
The next log-MGF term is \(1/(4n)=0.000025\), so omitting it has a small effect at this sample size. The limiting value is \(e^{1/2}\approx1.6487\); the finite-\(n\) approximation is slightly larger, as expected from the positive cubic term.
Worked Example: An Asymmetric Bernoulli Summand
Let \(B\) be Bernoulli with \(P(B=1)=0.2\), and set \(Z=(B-0.2)/0.4\). Find the third standardized moment and use it to approximate \(\log M_{W_{100}}(1)\) through order \(n^{-1/2}\).
Method. Standardize the two outcomes, calculate the third moment directly, and substitute the resulting cubic coefficient in the log-MGF expansion. This avoids assuming that asymmetry has no effect before the limit.
When \(B=1\), \(Z=(1-0.2)/0.4=2\), with probability \(0.2\). When \(B=0\), \(Z=(0-0.2)/0.4=-0.5\), with probability \(0.8\). The mean is \(0.2(2)+0.8(-0.5)=0\), and the variance is \(0.2(4)+0.8(0.25)=1\).
The third moment is \(E[Z^3]=0.2(2^3)+0.8((-0.5)^3)=1.6-0.1=1.5\). Since the quadratic MGF coefficient is \(1/2\), the cubic coefficient is \(E[Z^3]/6=1.5/6=0.25\). Thus \(\log M_Z(u)=u^2/2+u^3/4+o(u^3)\).
For \(t=1\) and \(n=100\), \(u=1/\sqrt{100}=0.1\). Multiplying the expansion by \(100\) gives \(\log M_{W_{100}}(1)\approx100[0.1^2/2+0.1^3/4]=100(0.005+0.00025)=0.525\). The corresponding approximate MGF is \(e^{0.525}\approx1.6905\), rounded to four decimals. The limiting MGF is \(e^{0.5}\approx1.6487\); the positive skewness accounts for a positive first correction. The approximation omits terms of smaller order and is not an exact finite-\(n\) value.
Common Mistakes and the Role of the Assumptions
A frequent algebra error is to retain a linear term in the single-summand MGF after centering. For a centered variable, \(M_Z(u)\) does not have the form \(1+u+u^2/2+\cdots\). Its linear coefficient is \(E[Z]=0\). If one incorrectly used that expression, then \(n\log M_Z(t/\sqrt n)\) would contain a term approximately \(t\sqrt n\), which does not have the required finite limit.
The correction is to calculate the coefficients from the assumptions: \(M_Z(0)=1\), \(M_Z'(0)=E[Z]=0\), and \(M_Z''(0)=E[Z^2]=1\). Hence \(M_Z(u)=1+u^2/2+o(u^2)\), and the logarithm and square-root scaling produce the finite limit \(t^2/2\).
Another trap is to claim that finite variance always makes the MGF proof valid. For example, a Student \(t\) random variable with five degrees of freedom has mean zero and variance \(5/3\), but its MGF is not finite on any open interval containing zero. The standard finite-variance IID CLT still applies to independent copies after centering and scaling; this particular MGF convergence argument does not. Distinguish the assumptions of a theorem from the assumptions of one proof.
| Feature | Role in the derivation |
|---|---|
| Independence | Turns the MGF of the sum into a product. |
| Identical distribution | Gives the same single-summand MGF in every factor. |
| Mean zero | Eliminates the linear term in the local MGF expansion. |
| Variance one | Sets the quadratic coefficient to \(1/2\). |
| MGF finite near zero | Allows this proof to use the local MGF expansion and convergence theorem. |
Check Your Understanding
Use the local MGF expansion and its assumptions to work through these problems.
- A Bernoulli variable has success probability \(0.4\). Standardize it to have mean zero and variance one. What is its third moment, and what is the coefficient of \(u^3\) in its MGF expansion? (Answer: \(E[Z^3]=1/\sqrt{6}\); the coefficient is \(1/(6\sqrt{6})\).)
- For symmetric \(Z\) taking values \(-1\) and \(1\) equally likely, use \(\log\cosh(u)=u^2/2-u^4/12+O(u^6)\). What is the fourth-order approximation to \(\log M_{W_{400}}(2)\)? (Answer: \(2-16/(12\cdot400)=1.9967\), to four decimals.)
- An IID summand has mean zero, variance one, and third moment \(\gamma=2\). What is the log-MGF approximation for the normalized sum at fixed \(t=1.5\), retaining the cubic correction? (Answer: \(1.5^2/2+2(1.5)^3/(6\sqrt n)=1.125+1.125/\sqrt n\).)
- For an exponential variable with rate \(2\), set \(Z=2Y-1\). Verify the mean and variance of \(Z\), then give \(\log M_Z(u)\) for values near zero. (Answer: \(E[Z]=0\), \(\operatorname{Var}(Z)=1\), and \(\log M_Z(u)=-u-\log(1-u)=u^2/2+u^3/3+\cdots\), for \(u<1\).)
- An IID population has a Student \(t\) distribution with five degrees of freedom. Can the MGF-based derivation here be applied directly, and does the ordinary finite-variance IID CLT apply to its standardized sample sums? (Answer: The MGF proof cannot be applied directly because the MGF does not exist near zero; the ordinary IID CLT applies because the variance is finite and positive.)