A Normal-Approximation Workshop for Independent Sums

From an Independent Sum to a Normal Approximation

A normal approximation is useful only when the sum, the centering, and the scale have all been chosen correctly. In a practical problem, the work is not just to substitute a mean and variance into a normal CDF: first identify the independent summands, then check that a central limit theorem applies, and finally translate the event without changing its meaning. Discrete sums add one more decision: whether to use a continuity correction.

This tutorial uses the Lyapunov central limit theorem as a sufficient condition for independent sums. It is not the only available theorem, but it gives a clear checklist. The earlier material on the central limit theorem and its variants supplies other criteria; here the emphasis is on matching a valid result to a probability question and carrying out the approximation carefully.

Definition: For independent random variables $X_{n,1},\ldots,X_{n,k_n}$ in row $n$, suppose each has a finite mean $\mu_{n,i}=E[X_{n,i}]$ and finite variance, and define $$ S_n=\sum_{i=1}^{k_n}X_{n,i},\qquad \mu_n=\sum_{i=1}^{k_n}\mu_{n,i},\qquad s_n^2=\sum_{i=1}^{k_n}\operatorname{Var}(X_{n,i}). $$ If $s_n^2>0$ and, for some $\delta>0$, $$ \frac{\sum_{i=1}^{k_n}E\!\left[|X_{n,i}-\mu_{n,i}|^{2+\delta}\right]}{s_n^{2+\delta}}\longrightarrow 0, $$ then the Lyapunov central limit theorem gives $$ \frac{S_n-\mu_n}{s_n}\xrightarrow{d}N(0,1). $$

For a fixed $x$, this convergence implies $P((S_n-\mu_n)/s_n\le x)\to\Phi(x)$, where $\Phi$ is the standard normal CDF. It also gives convergence of probabilities of intervals whose standardized endpoints converge to finite values, because the normal distribution assigns probability zero to every single endpoint. Lyapunov's condition is sufficient, not necessary: failure to verify it does not by itself prove that a normal approximation is invalid.

A Reliable Calculation Workflow

For a probability such as $P(a\le S_n\le b)$, the mean and variance of the sum are obtained from independence: $$ E[S_n]=\mu_n,\qquad \operatorname{Var}(S_n)=s_n^2. $$ Thus the standard deviation is $s_n$, not $s_n^2$. Since $s_n>0$, subtracting $\mu_n$ and dividing by $s_n$ preserves the order of endpoints: $$ P(a\le S_n\le b) = P\!\left(\frac{a-\mu_n}{s_n}\le \frac{S_n-\mu_n}{s_n}\le \frac{b-\mu_n}{s_n}\right). $$ The CLT then suggests replacing the standardized sum by $Z\sim N(0,1)$. The approximation is $$ \Phi\!\left(\frac{b-\mu_n}{s_n}\right) - \Phi\!\left(\frac{a-\mu_n}{s_n}\right). $$ For a one-sided upper-tail probability, the corresponding expression is $1-\Phi((a-\mu_n)/s_n)$; for a lower tail, it is $\Phi((b-\mu_n)/s_n)$. These formulas follow from the event transformation and the standard normal CDF, not from a change in the original sum's distribution.

Write the sum as independent terms.
Record each summand's mean and variance, and verify that the terms are independent.

Choose a CLT and check its conditions.
For Lyapunov's theorem, check finite $2+\delta$ moments, positive total variance, and the stated Lyapunov ratio as the row size grows.

Center and scale.
Use the sum of the means and the square root of the sum of variances.

Translate the event and evaluate.
For an integer-valued sum, consider a continuity correction. Evaluate the appropriate standard normal CDF difference and describe it as an approximation.

Why a Continuity Correction Works

A discrete sum can take only integer values, whereas a normal random variable spreads probability continuously across intervals. If $S$ is integer-valued, then the event $a\le S\le b$, for integers $a\le b$, consists of the integer points $a,a+1,\ldots,b$. The continuous interval centered around those same points is bounded halfway between adjacent integers.

Derived result: If $S$ is integer-valued and $a,b$ are integers with $a\le b$, then $$ \{a\le S\le b\}=\{a-\tfrac12<S<b+\tfrac12\}. $$ Consequently, if a normal approximation is appropriate for $S$, approximate this probability using the normal interval from $a-\tfrac12$ to $b+\tfrac12$, not from $a$ to $b$.

To prove the event identity, an integer value $s$ satisfies $a\le s\le b$ exactly when $a\le s$ and $s\le b$. Since $s$ is an integer, the first inequality is equivalent to $a-\tfrac12<s$, and the second is equivalent to $s<b+\tfrac12$. Conversely, if the integer $s$ lies strictly between those half-integers, it must be at least $a$ and at most $b$. Thus the events are equal. Replacing the distribution of $S$ by a normal distribution over the enlarged interval is an approximation; only the event identity is exact. $\square$

For example, the event $S\ge a$ for integer $a$ is matched to the interval above $a-\tfrac12$, and $S\le b$ is matched to the interval below $b+\tfrac12$. No such correction is needed for a genuinely continuous sum: its probability over a closed interval is unchanged if the endpoints are included or excluded.

Worked Example: A Binomial Count With a Continuity Correction

Suppose $X\sim\operatorname{Binomial}(100,0.5)$. Approximate $P(40\le X\le60)$.

Write $X=\sum_{i=1}^{100}B_i$, where the $B_i$ are independent Bernoulli random variables with $P(B_i=1)=0.5$. Each has mean $0.5$ and variance $0.5(0.5)=0.25$. Therefore $$ E[X]=100(0.5)=50,\qquad \operatorname{Var}(X)=100(0.25)=25,\qquad \operatorname{SD}(X)=5. $$

The summands are bounded, have finite moments of every order, and their Lyapunov ratio tends to zero as the number of summands grows. The continuity-corrected event is $39.5<X<60.5$. Its standardized endpoints are $$ z_{\mathrm{low}}=\frac{39.5-50}{5}=-2.1,\qquad z_{\mathrm{high}}=\frac{60.5-50}{5}=2.1. $$

The normal approximation is therefore $$ \Phi(2.1)-\Phi(-2.1)=2\Phi(2.1)-1\approx 2(0.9821)-1=0.9642, $$ or about $0.9643$ using a more precise CDF value. This is an approximation to the binomial probability, not an identity. The correction is especially natural here because the requested event includes every integer from 40 through 60.

Independent, Nonidentically Distributed Summands

Equal distributions are not required. What matters is that the summands are independent and that the chosen theorem's condition controls the aggregate contribution of unusually large deviations. In a Poisson-binomial sum, each Bernoulli variable can have a different success probability. Its mean is the sum of those probabilities, and its variance is the sum of $p_i(1-p_i)$; using a single common probability when the $p_i$'s differ gives the wrong centering and scale.

Worked Example: A Poisson-Binomial Sum

For each positive integer $m$, let $S_m$ be the sum of $20m$ independent Bernoulli variables with success probability $0.2$ and $30m$ independent Bernoulli variables with success probability $0.6$. Approximate $P(100\le S_5\le120)$.

For general $m$, the mean and variance are $$ \mu_m=20m(0.2)+30m(0.6)=22m, $$ $$ s_m^2=20m(0.2)(0.8)+30m(0.6)(0.4)=3.2m+7.2m=10.4m. $$ The summands are independent and bounded. For a Bernoulli variable $B$ with success probability $p$, $$ E|B-p|^3=p(1-p)\{p^2+(1-p)^2\}\le p(1-p). $$ Thus the sum of the third absolute centered moments is at most $s_m^2$. With $\delta=1$, the Lyapunov ratio is at most $s_m^2/s_m^3=1/s_m$, which tends to zero because $s_m^2=10.4m\to\infty$. Lyapunov's theorem applies.

At $m=5$, there are 100 variables with probability $0.2$ and 150 with probability $0.6$. Hence $$ \mu_5=110,\qquad s_5^2=52,\qquad s_5=\sqrt{52}\approx7.2111. $$ For the integer interval, use endpoints $99.5$ and $120.5$. Standardizing gives $$ z_{\mathrm{low}}=\frac{99.5-110}{\sqrt{52}}\approx-1.4561,\qquad z_{\mathrm{high}}=\frac{120.5-110}{\sqrt{52}}\approx1.4561. $$

By symmetry of the standard normal distribution, $$ P(100\le S_5\le120)\approx\Phi(1.4561)-\Phi(-1.4561)\approx0.8546. $$ The mean $110$ lies at the center of the requested interval, and the variance $52$ is the sum of the two groups' variances, $16+36$. Both checks are consistent with the calculation.

Continuous Sums: No Lattice Adjustment

For a continuous sum, standardization uses the same mean-and-variance calculation, but there is no half-unit correction. One must still check the asymptotic conditions and remember that a CLT is a limiting statement: it does not promise a precise error bound at a particular sample size.

Worked Example: A Sum of Exponential Measurements

Let $X_1,\ldots,X_{40}$ be independent exponential random variables, each with mean $2$, and let $T=\sum_{i=1}^{40}X_i$. Approximate $P(70\le T\le90)$.

An exponential variable with mean $2$ has variance $4$, so $$ E[T]=40(2)=80,\qquad \operatorname{Var}(T)=40(4)=160,\qquad \operatorname{SD}(T)=\sqrt{160}\approx12.6491. $$ The variables have finite third absolute centered moments, so the iid Lyapunov condition with $\delta=1$ holds: the numerator of the ratio is $40E|X_1-2|^3$, while the denominator is $(160)^{3/2}$. More generally, for $n$ such variables the ratio is a finite constant divided by $\sqrt n$, and tends to zero.

The standardized endpoints are $$ \frac{70-80}{\sqrt{160}}\approx-0.7906,\qquad \frac{90-80}{\sqrt{160}}\approx0.7906. $$ Because the sum is continuous, no continuity correction is used. Thus $$ P(70\le T\le90)\approx\Phi(0.7906)-\Phi(-0.7906) =2\Phi(0.7906)-1\approx0.571. $$ The skewness of an exponential variable means that its finite-sample sum need not be exactly normal. The CLT justifies this as an asymptotic approximation, not as an exact probability for $n=40$.

Common Mistakes and How to Avoid Them

Dividing by the variance. Standardization uses $s_n=\sqrt{\operatorname{Var}(S_n)}$, not $\operatorname{Var}(S_n)$.
Adding variances without independence. In general, the variance of a sum also involves covariances. The simple sum-of-variances formula used here depends on independence.
Using one success probability for a Poisson-binomial sum. Add the individual means and variances; do not replace varying probabilities by an unverified common value.
Applying a correction in the wrong direction. For an inclusive integer interval $a\le S\le b$, use $a-\tfrac12$ and $b+\tfrac12$. The correction makes the interval wider because it includes the full probability mass at both boundary integers.
Calling the approximation exact. A CLT gives a limit. A finite-sample normal probability is an estimate whose quality depends on the distribution and sample size; the theorem alone does not certify a numerical error bound.
Treating a failed sufficient condition as proof of failure. If a Lyapunov check does not work, another CLT may still apply. The criterion is sufficient, not necessary.

Key takeaway: A defensible normal approximation identifies the independent summands, verifies a suitable CLT condition, uses the sum's actual mean and standard deviation, and translates the event correctly. For integer-valued sums, apply a continuity correction before evaluating the standard normal probability.

Check Your Understanding

For each question, identify the relevant check or calculation before deciding on an approximation.

For independent Bernoulli summands with unequal success probabilities, how do you calculate the mean and variance of their sum?
A sum has mean $30$ and variance $25$. What standard deviation should be used to standardize it, and why?
For an integer-valued sum, write the continuity-corrected interval corresponding to $12\le S\le18$.
What does the Lyapunov condition establish, and what does its failure to hold establish?
Why should the normal approximation for a sum of continuous variables not use a half-unit continuity correction?