Normal Approximations for Sums and Sample Means

From a Central Limit Theorem to a Probability

The central limit theorem describes the distribution of a standardized sum when the number of independent observations becomes large. A practical use of that result is to estimate probabilities that would otherwise require a complicated sum or convolution. The calculation has two parts: express the event in standard-deviation units, then evaluate the standard normal cumulative distribution function.

Let \(X_1,X_2,\ldots\) be iid random variables on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with mean \(\mu\) and finite, strictly positive variance \(\sigma^2\). Define \(S_n=\sum_{i=1}^n X_i\) and \(\bar X_n=S_n/n\). Independence gives \(\mathbb{E}S_n=n\mu\) and \(\operatorname{Var}(S_n)=n\sigma^2\). Consequently, \(\mathbb{E}\bar X_n=\mu\) and \(\operatorname{Var}(\bar X_n)=\sigma^2/n\).

Standardizing means subtracting the expectation and dividing by the standard deviation. The exact identities are

Thus a threshold \(c\) for the sum corresponds to the standardized threshold \((c-n\mu)/(\sigma\sqrt n)\). A threshold \(a\) for the sample mean corresponds to \((a-\mu)/(\sigma/\sqrt n)\). These identities are algebra, not approximations; the approximation enters when the standardized variable is treated as standard normal.

Proof. By the central limit theorem for standardized sums, the standardized variables \(W_n=(S_n-n\mu)/(\sigma\sqrt n)\) converge in distribution to a standard normal variable \(Z\). Convergence in distribution implies convergence of CDFs at every continuity point of the limiting CDF. Since \(\Phi\) is continuous at every real number, \(\mathbb{P}(W_n\le z)\to\Phi(z)\) for each fixed \(z\). For the interval statement, write \(\mathbb{P}(a\le W_n\le b)=\mathbb{P}(W_n\le b)-\mathbb{P}(W_n<a)\). For any \(\epsilon>0\), \(\mathbb{P}(W_n\le a-\epsilon)\le\mathbb{P}(W_n<a)\le\mathbb{P}(W_n\le a)\). Taking lower and upper limits in \(n\) bounds the lower and upper limits of \(\mathbb{P}(W_n<a)\) between \(\Phi(a-\epsilon)\) and \(\Phi(a)\). Letting \(\epsilon\) decrease to zero and using continuity gives \(\mathbb{P}(W_n<a)\to\Phi(a)\). The two CDF limits give the stated interval limit. \(\square\)

The theorem is an asymptotic statement: it does not give an error bound for a chosen sample size. It also requires a nonzero finite variance. If \(\sigma^2=0\), each \(X_i=\mu\) almost surely, so the standardized expression divides by zero and is not defined; the sum and mean are instead constant. If the variance is infinite, this version of the central limit theorem does not apply.

Three Standardization Calculations

Reading the Approximation Correctly

For a lower-tail probability, standardize the event's cutoff and read \(\Phi(z)\). For an upper-tail probability, use the complement \(1-\Phi(z)\), taking care to express the original event with the correct inequality. For an interval, subtract the lower-end CDF value from the upper-end CDF value. These operations approximate probabilities under the standardized distribution; they do not change the original event or make its finite-sample distribution continuous.

For a binomial variable, the event \(X\le k\) has the exact standardized form \((X-np)/\sqrt{np(1-p)}\le(k-np)/\sqrt{np(1-p)}\). The theorem motivates evaluating \(\Phi\) at that standardized cutoff for large \(n\). A discrete distribution can have jumps, so a normal CDF does not reproduce its finite-sample probabilities exactly. How to adjust cutoffs for discreteness, and how to assess approximation quality, are separate questions.

Common mistake: using the standard deviation of one observation for the sample mean. In the sample-mean example, an incorrect calculation might divide the endpoint differences by \(10\), obtaining \((48-50)/10=-0.2\) and \((53-50)/10=0.3\). This would give \(\Phi(0.3)-\Phi(-0.2)\approx0.6179-0.4207=0.1972\). The error is that \(10\) is the standard deviation of one observation, not of the mean of \(25\) observations. The correct standard deviation is \(10/\sqrt{25}=2\), giving endpoints \(-1\) and \(3/2\) and the approximation \(0.7745\). Dividing by the wrong scale changes the event's location in standard-deviation units.

Finally, the theorem does not guarantee that every finite-sample approximation is accurate. The assumptions explain when the asymptotic result applies; accuracy at a particular \(n\) can depend on the underlying distribution and the event being estimated. Treat the displayed normal probability as an approximation, and retain enough digits in the standardized endpoint to avoid unnecessary rounding error before consulting a CDF table.