Berry–Esseen Bounds on Normal Approximation Error

Why Bound the Normal Approximation Error?

The Central Limit Theorem explains why a standardized sum can be approximated by a standard normal distribution when there are many independent contributions. It is an asymptotic result: it describes what happens as the number of contributions grows, but it does not by itself give a numerical error bound at a particular sample size. The Berry–Esseen theorem adds a quantitative guarantee. It bounds the largest possible difference between the true cumulative distribution function of a standardized sum and the standard normal cumulative distribution function.

The bound depends on the summands’ third absolute centered moments. These moments reflect how much probability lies far from each summand’s mean, and they can make a normal approximation less accurate even when the total variance is known. The bound is often conservative, so it is best read as a guaranteed upper limit rather than a prediction of the actual error.

Definition: For a random variable $X_i$ with mean $\mu_i$, variance $\sigma_i^2$, and finite third absolute centered moment, write $\beta_i=E|X_i-\mu_i|^3$. The third absolute centered moment measures the average cubed distance from the mean, without cancellation between positive and negative deviations.

The Berry–Esseen Bound

Let $X_1,\ldots,X_n$ be independent random variables with means $\mu_i$, variances $\sigma_i^2$, and finite third absolute centered moments $\beta_i$. Define the centered sum $S=\sum_{i=1}^n(X_i-\mu_i)$, and let $V=\sum_{i=1}^n\sigma_i^2$ be its variance. The theorem requires $V>0$, so that standardization is possible.

Theorem: Under these assumptions, for every real number $x$, $$ \left|P\left(\frac{S}{\sqrt{V}}\le x\right)-\Phi(x)\right| \le 0.56\,\frac{\sum_{i=1}^n\beta_i}{V^{3/2}}, $$ where $\Phi(x)=P(Z\le x)$ is the CDF of a standard normal random variable $Z\sim N(0,1)$. The same inequality holds simultaneously for all real $x$; equivalently, the supremum of the absolute CDF difference over all cutoffs is no larger than the stated bound.

This version permits independent summands that do not have identical distributions. The constant $0.56$ is a valid universal constant for this setting. The theorem does not require the summands to be continuous. In particular, the true CDF of the sum may have jumps, as happens for discrete summands; the comparison is still made directly with the continuous normal CDF at every cutoff.

The bound is dimensionless. If each summand is measured in some unit, the third-moment sum has units cubed, as does $V^{3/2}$, so their ratio has no units. A bound of $0.04$, for example, guarantees that at any chosen cutoff the true CDF probability differs from the corresponding normal CDF probability by at most 0.04, or four percentage points.

How the Bound Behaves for Identically Distributed Summands

A useful consequence appears when $X_1,\ldots,X_n$ are independent and identically distributed, each with mean $\mu$, variance $\sigma^2>0$, and third absolute centered moment $\beta=E|X_1-\mu|^3$. The total variance is $V=n\sigma^2$, and the sum of the third absolute centered moments is $n\beta$. Substituting these expressions into the theorem gives

0.56\,\frac{n\beta}{(n\sigma^2)^{3/2}} =0.56\,\frac{\beta}{\sigma^3\sqrt{n}}.

This algebra shows that, for a fixed summand distribution, the guaranteed error bound decreases at the rate $1/\sqrt{n}$. It also shows why sample size alone does not determine the guarantee: a larger value of $\beta/\sigma^3$ produces a larger bound. The ratio is a standardized third absolute moment, so it measures tail size relative to the summand’s standard deviation.

This is a derivation of the iid form from the general theorem, not a claim that every actual approximation error equals the bound. The theorem says only that the error cannot exceed it. The actual difference can be much smaller.

Worked Example: A Sum of Symmetric Two-Point Variables

Suppose $X_1,\ldots,X_{100}$ are independent, with each taking the values $-1$ and $1$ with probability $1/2$. Use Berry–Esseen to bound the error in approximating the CDF of $S/\sqrt{V}$ by $\Phi$, where $S=\sum_{i=1}^{100}X_i$.

Check the assumptions. The variables are independent and identically distributed. Each has mean $0$, variance $1$, and a finite third absolute centered moment. Thus $V=100$, which is positive.

Since $|X_i|=1$ with probability one, $\beta_i=E|X_i|^3=1$. Therefore the Berry–Esseen bound is

0.56\,\frac{\sum_{i=1}^{100}\beta_i}{V^{3/2}} =0.56\,\frac{100}{100^{3/2}} =0.56\,\frac{100}{1000} =0.0560.

So for every real $x$, the difference between $P(S/10\le x)$ and $\Phi(x)$ is at most 0.0560. For example, since $\Phi(1)$ is approximately 0.8413, the true probability $P(S/10\le 1)$ must lie between 0.7853 and 0.8973. This is a valid guarantee, though it may be wider than the actual discrepancy.

Calculating a Third Absolute Moment

The third absolute moment is centered at the summand’s mean and uses an absolute value before cubing. The absolute value matters: without it, positive and negative deviations could cancel, giving a different quantity that would not serve as the numerator in this bound. For many distributions, the moment can be found directly from its probability mass function or density.

Worked Example: Centered Exponential Summands

Let $Y_1,\ldots,Y_{20}$ be independent exponential random variables with rate 1. Each has mean 1 and variance 1. For $X_i=Y_i-1$, find a Berry–Esseen bound for the normal approximation to $\sum_{i=1}^{20}X_i/\sqrt{20}$.

Check the assumptions. The shifted variables $X_i$ are independent and identically distributed, their variance is 1, and their third absolute centered moment is finite. Thus the total variance is $V=20>0$. The density of each $Y_i$ is $e^{-y}$ for $y\ge0$, so

\begin{aligned} \beta_i &=E|Y_i-1|^3\\ &=\int_0^1(1-y)^3e^{-y}\,dy+\int_1^\infty(y-1)^3e^{-y}\,dy\\ &=\left(\frac{6}{e}-2\right)+\frac{6}{e}\\ &=\frac{12}{e}-2 \approx 2.4146. \end{aligned}

For the second integral, substituting $u=y-1$ gives $e^{-1}\int_0^\infty u^3e^{-u}\,du=6/e$. For the first, substituting $u=1-y$ gives $e^{-1}\int_0^1u^3e^u\,du=6/e-2$. Hence the total third-moment sum is $20(2.4146)$, while $V^{3/2}=20^{3/2}$. The bound is

0.56\,\frac{20(2.4146)}{20^{3/2}} =0.56\,\frac{2.4146}{\sqrt{20}} \approx 0.3024.

At any cutoff $x$, the normal CDF approximation to $P(\sum X_i/\sqrt{20}\le x)$ is therefore guaranteed to be within about 0.3024. This is a weak guarantee: it does not establish that the approximation is poor, but it cannot certify a small error. The relatively large third absolute moment helps explain why this theorem gives a looser bound than it did for the symmetric two-point variables.

Unequal Weights and Interval Probabilities

The general theorem also handles weighted sums. If $Y_i$ are independent and the quantity of interest is $W=\sum_{i=1}^n a_i(Y_i-EY_i)$, then the summands are $X_i=a_i(Y_i-EY_i)$. Their variances add to $V=\sum_i a_i^2\operatorname{Var}(Y_i)$, and their third absolute centered moments add to $\sum_i |a_i|^3E|Y_i-EY_i|^3$. The absolute value on each weight ensures the third-moment contribution is nonnegative, including when a weight is negative.

The CDF bound can also be used to control probabilities between two cutoffs. If the CDF discrepancy is at most $\Delta$ everywhere, then for $a<b$, the difference between the true and normal probabilities of the interval $(a,b]$ is at most $2\Delta$. Indeed, each interval probability is a difference of two CDF values, and each of those two values differs by at most $\Delta$. This does not generally give a $\Delta$ bound for an interval, because errors at its two endpoints can contribute.

Worked Example: Three Unequally Weighted Summands

Let $U_1,U_2,U_3$ be independent, with each taking $-1$ and $1$ with equal probability. Consider $W=2U_1+U_2+U_3$. Find the Berry–Esseen bound for the CDF of $W/\sqrt{\operatorname{Var}(W)}$.

Check the assumptions. The three summands are independent, each has mean 0 and variance 1, and all have finite third absolute moments. Each $|U_i|^3=1$, so the third absolute centered moment of a weighted summand $a_iU_i$ is $|a_i|^3$. The total variance is positive.

The variance and third-moment sum are

V=2^2+1^2+1^2=6, \qquad \sum_{i=1}^3 E|a_iU_i|^3=|2|^3+|1|^3+|1|^3=10.

Consequently,

\sup_x\left|P\left(\frac{W}{\sqrt{6}}\le x\right)-\Phi(x)\right| \le 0.56\,\frac{10}{6^{3/2}} \approx 0.3810.

This bound applies even though the standardized weighted sum is discrete and its CDF has jumps. It limits the CDF difference at every cutoff; it does not make the discrete CDF continuous. For an interval probability $(a,b]$, the corresponding guaranteed discrepancy is at most $2(0.3810)=0.7620$, which is mathematically valid but not informative. The example illustrates that a small number of summands and a relatively large weight can yield a loose bound.

What the Bound Tells You—and What It Does Not

Berry–Esseen is useful when a normal approximation needs an explicit error guarantee rather than a qualitative justification. It makes two influences visible: the standardized third absolute moments of the summands and how those contributions compare with the total variance. In the iid case, the bound shrinks as the sample size grows; with unequal weights, a few large weights can dominate the third-moment sum.

A bound is a worst-case guarantee across all cutoffs. It need not resemble the actual error at any particular cutoff. If the calculated bound exceeds 1, it remains a valid inequality but is usually uninformative, since probabilities themselves cannot differ by more than 1. Even a smaller bound can be conservative. The earlier material on CLT accuracy in central and extreme tails addresses how approximation quality may vary across regions; Berry–Esseen instead supplies a single uniform bound over the whole CDF.

Common pitfall: “The sum is discrete, so a CDF bound against a normal distribution cannot apply.” This is incorrect. The true CDF may have jumps, but the normal CDF is continuous, and the theorem compares their values directly at each real cutoff. Discontinuity of the true CDF does not invalidate the inequality.

Another common error is to use a signed third central moment in place of $E|X_i-\mu_i|^3$. The theorem requires the third absolute centered moment. Check that the absolute value is inside the expectation, verify independence and finite moments, calculate the total variance, and only then substitute into the bound.

Check Your Understanding

Use the theorem and the definitions in this tutorial to answer the following.

State the assumptions required for the Berry–Esseen bound given here, including the condition on total variance.
Why does the theorem still apply when the standardized sum has a discrete distribution?
If iid summands have variance $\sigma^2$ and third absolute centered moment $\beta$, write the Berry–Esseen error bound in terms of $n$, $\sigma$, and $\beta$.
For independent centered summands with weights $a_i$, how do the weights enter the total variance and the sum of third absolute moments?
If a CDF approximation error is at most $\Delta$ at every cutoff, what upper bound follows for the error in approximating a probability of the form $P(a<Z\le b)$?