Standardized Sum Error Beyond the Central Limit Theorem

Why the Normal Approximation Has Structured Error

The central limit theorem says that a suitably standardized sum approaches a standard normal distribution. It does not, by itself, describe how quickly the approximation improves as the sample size grows, or how the shape of the summand distribution affects the error. Edgeworth expansions address these questions by adding polynomial corrections to the normal density or distribution function.

The leading correction is governed by skewness and is typically of order $n^{-1/2}$. At the next order, excess kurtosis and squared skewness appear, each with a factor of $n^{-1}$. These are asymptotic statements under regularity conditions: they are not automatic error bounds for every distribution or every sample size.

Setup and the Edgeworth Approximation

Definition: Let $X_1,X_2,\ldots$ be independent and identically distributed random variables with $E[X_i]=0$, $\operatorname{Var}(X_i)=1$, and finite moment generating function in a neighborhood of zero. Define the standardized sum $$ Z_n=\frac{X_1+\cdots+X_n}{\sqrt n}. $$ Write $\gamma_1=E[X_1^3]$ for the standardized skewness and $\gamma_2=E[X_1^4]-3$ for the excess kurtosis.

The following classical form of the Edgeworth expansion requires an additional condition on the characteristic function $\chi(t)=E[e^{itX_1}]$: assume the Cramér condition $\limsup_{|t|\to\infty}|\chi(t)|<1$. Under these assumptions, the two-term expansion below holds uniformly in $x$, with remainder $o(n^{-1})$. This condition excludes lattice distributions, for which additional corrections may be needed.

Theorem (two-term Edgeworth expansion): Under the assumptions above, as $n\to\infty$, $$ P(Z_n\le x) = \Phi(x) -\phi(x)\left[ \frac{\gamma_1}{6\sqrt n}(x^2-1) +\frac{1}{n}\left\{ \frac{\gamma_2}{24}(x^3-3x) +\frac{\gamma_1^2}{72}(x^5-10x^3+15x) \right\} \right] +o(n^{-1}), $$ uniformly for $x\in\mathbb R$. Here $\Phi$ and $\phi$ are the standard normal CDF and density.

The display separates the effects by order. Skewness first contributes at $n^{-1/2}$. At order $n^{-1}$, the excess-kurtosis term and the squared-skewness term both matter. If the summand distribution is symmetric, then $\gamma_1=0$, so the $n^{-1/2}$ term disappears; the kurtosis term can still contribute at order $n^{-1}$.

Why the Sample-Size Orders Appear

The scaling can be seen from cumulants. For a random variable with a moment generating function near zero, its $r$th cumulant is the $r$th derivative at zero of the logarithm of its moment generating function. The first two cumulants are the mean and variance; for a centered, variance-one variable, the third and fourth are $\gamma_1$ and $\gamma_2$.

Derived result: If $X_i$ are iid with the moments above, the $r$th cumulant of $Z_n$ is $n^{1-r/2}$ times the $r$th cumulant of $X_1$. In particular, the third cumulant of $Z_n$ is $\gamma_1/\sqrt n$, and the fourth is $\gamma_2/n$.

To verify this scaling, let $M(t)=E[e^{tX_1}]$. Independence gives the moment generating function of $Z_n$ as $M(t/\sqrt n)^n$. Its log is $n\log M(t/\sqrt n)$. Differentiating $r$ times at zero brings a factor $n$ from the outer multiplier and $n^{-r/2}$ from the rescaled argument. Thus its $r$th derivative is $n^{1-r/2}$ times the corresponding derivative of $\log M$. Taking $r=3$ and $r=4$ gives the stated results. $\square$

This scaling explains why a skewness correction has size $n^{-1/2}$, while the direct fourth-cumulant correction has size $n^{-1}$. The squared-skewness contribution also has that order because $(n^{-1/2})^2=n^{-1}$.

How the Density Correction Produces the CDF Correction

The polynomial forms in the theorem are not arbitrary. To see how they enter the CDF, write the corresponding density correction through order $n^{-1}$ as $$ \phi(x)\left[ 1+\frac{\gamma_1}{6\sqrt n}(x^3-3x) +\frac{1}{n}\left\{ \frac{\gamma_2}{24}(x^4-6x^2+3) +\frac{\gamma_1^2}{72}(x^6-15x^4+45x^2-15) \right\} \right]. $$

The following integration identity verifies the CDF polynomial terms: for each integer $k\ge 1$, $\frac{d}{dx}\{\phi(x)Q_{k-1}(x)\}=-\phi(x)Q_k(x)$, where the polynomials needed here are $Q_2=x^2-1$, $Q_3=x^3-3x$, $Q_4=x^4-6x^2+3$, $Q_5=x^5-10x^3+15x$, and $Q_6=x^6-15x^4+45x^2-15$. This follows by differentiating and using $\phi'(x)=-x\phi(x)$. Since $\phi(x)Q_{k-1}(x)\to0$ as $x\to-\infty$, integration from $-\infty$ to $x$ gives $\int_{-\infty}^x\phi(u)Q_k(u)\,du=-\phi(x)Q_{k-1}(x)$. Applying this identity to each correction term yields the CDF signs and polynomials in the theorem. $\square$

This calculation explains the form of the correction once the Edgeworth density approximation is available; it is not, by itself, a proof of the theorem’s uniform remainder. Establishing that remainder requires the regularity assumptions and the asymptotic analysis behind the Edgeworth theorem.

Worked Applications

Worked Example: Normal Summands Give No Correction

Let $X_i\sim N(0,1)$. Then $\gamma_1=E[X_i^3]=0$, and $E[X_i^4]=3$, so $\gamma_2=3-3=0$. Also, $Z_n\sim N(0,1)$ for every $n$, not merely in the limit. Both Edgeworth correction terms are zero, and $P(Z_n\le x)=\Phi(x)$ exactly.

This case is a useful check on the formula: the normal distribution has no skewness or excess kurtosis, and standardization of a sum of normal variables introduces no approximation error.

Worked Example: First-Order Correction for Exponential Summands

Let $Y_i$ be iid exponential random variables with rate 1, and set $X_i=Y_i-1$. Then $E[X_i]=0$, $\operatorname{Var}(X_i)=1$, $\gamma_1=2$, and $\gamma_2=6$. For $n=25$ and $x=0$, the first-order approximation is $$ P(Z_{25}\le 0) \approx \Phi(0)-\phi(0)\frac{2}{6\sqrt{25}}(0^2-1). $$

Since $\Phi(0)=0.5$, $\phi(0)=1/\sqrt{2\pi}\approx0.39894$, and $2/(6\sqrt{25})=1/15$, this gives $$ P(Z_{25}\le 0)\approx0.5+0.39894\left(\frac{1}{15}\right)\approx0.52660. $$

The positive correction is consistent with the right-skewed exponential summands: after centering, the sum’s distribution is not symmetric about zero. At $x=0$, both second-order polynomials in the theorem vanish, so the displayed two-term formula has the same explicit correction there. The asymptotic remainder is not a guarantee of a particular numerical error at $n=25$.

Worked Example: Kurtosis Correction for Uniform Summands

Let $X_i$ be uniform on $[-\sqrt3,\sqrt3]$. The distribution is symmetric, has mean zero and variance one, and $E[X_i^4]=9/5$. Therefore $\gamma_1=0$ and $\gamma_2=9/5-3=-6/5$. At $n=16$ and $x=1$, the expansion reduces to $$ P(Z_{16}\le1) \approx \Phi(1)-\phi(1)\frac{1}{16}\frac{-6/5}{24}(1^3-3\cdot1). $$

Here $(-6/5)/24=-1/20$, and $1-3=-2$, so the product inside the correction is $1/10$. With $\Phi(1)\approx0.8413447$ and $\phi(1)\approx0.2419707$, $$ P(Z_{16}\le1)\approx0.8413447-\frac{0.2419707}{160}\approx0.8398324. $$

There is no skewness correction because the summands are symmetric. The nonzero correction comes from their negative excess kurtosis.

Order, Interpretation, and a Common Pitfall

An Edgeworth approximation is a refinement organized by powers of $n^{-1/2}$, not a promise that every correction improves every numerical answer. Its terms describe systematic departures from normality under the theorem’s assumptions. They also show how the shape of the summands affects the standardized sum: skewness matters first, and kurtosis and squared skewness enter at the next order.

If $\gamma_1\ne0$, the leading correction is generally of order $n^{-1/2}$.
If $\gamma_1=0$, the first correction shown is of order $n^{-1}$, and depends on $\gamma_2$.
If $\gamma_1=\gamma_2=0$, the displayed terms vanish, but that alone does not imply the sum is exactly normal.

Common pitfall: A small sample size does not make the normal approximation automatically accurate just because an Edgeworth term has been added. The expansion is asymptotic, and the correction can be large enough to signal that the approximation is unreliable. It can even produce a value outside the interval $[0,1]$, which cannot be a probability. Lattice distributions also need special care: the Cramér condition used here fails for them, and the stated expansion need not apply without additional terms.

Key takeaway: For a regular iid standardized sum, skewness supplies the leading $n^{-1/2}$ correction; excess kurtosis and squared skewness supply $n^{-1}$ corrections. The terms refine the central limit approximation, but their validity and usefulness depend on the distribution and sample size.

Check Your Understanding

Use the expansion and its assumptions to answer the following.

For centered, variance-one iid summands with skewness $-1$, what is the order in $n$ of the leading correction?
Why does symmetry remove the $n^{-1/2}$ term, and which shape feature can still affect the CDF at order $n^{-1}$?
For uniform $[-\sqrt3,\sqrt3]$ summands, verify the excess kurtosis and explain why the first-order skewness term is zero.
What does the Cramér condition rule out in the theorem stated here?
Why is it incorrect to interpret the expansion as a guarantee that the corrected approximation is accurate for every finite $n$?