Normal Limits for Sample Means and Sums

From an iid sample to a normal approximation

When many independent observations contribute to a total, the central limit theorem explains why a normal distribution can approximate that total—even when the individual observations are not normally distributed. In applications, the theorem is useful because it translates a probability about a sum or an average into a standard normal probability. The key is to center using the population mean and scale using the population variance.

Let \(X_1,\ldots,X_n\) be iid random variables with \(E[X_i]=\mu\) and \(0<\operatorname{Var}(X_i)=\sigma^2<\infty\). Write \(S_n=\sum_{i=1}^n X_i\) and \(\bar X_n=S_n/n\). The standard deviation of the sum is \(\sigma\sqrt n\); the standard deviation of the sample mean is \(\sigma/\sqrt n\). These scales are different, but the corresponding standardized quantities are exactly equal.

The central limit theorem for iid observations states that \(Z_n\xrightarrow{d}N(0,1)\). This is the classical iid central limit theorem; the finite-variance assumptions above are essential for this version. The result does not say that the original observations are normal, nor that the finite-sample sum is exactly normal. It describes the limiting distribution after centering and scaling.

Translating events into standard normal probabilities

Standardization is an algebraic change of scale, not an approximation. The approximation enters when the distribution of \(Z_n\) is replaced by the standard normal distribution. For example, a threshold for the sample mean becomes

For the sum, the same event written as \(S_n\le s\) becomes

For finite \(n\), these are normal approximations, not identities. The exact standardized probability is on the left; the standard normal probability is the proposed approximation on the right.

The convergence of \(Z_n\) is the iid central limit theorem. To justify the interval statement, first note that convergence in distribution to a continuous CDF gives \(P(Z_n\le t)\to\Phi(t)\) for every real \(t\). Thus

The requested interval includes its lower endpoint, so one more step is needed: \(P(a\le Z_n\le b)\) need not equal \(P(a<Z_n\le b)\) for a finite discrete distribution. For any \(\varepsilon>0\),

Taking lower and upper limits as \(n\to\infty\), the first probability tends to \(\Phi(b)-\Phi(a)\), and the last tends to \(\Phi(b)-\Phi(a-\varepsilon)\). Letting \(\varepsilon\downarrow0\) and using continuity of \(\Phi\) makes the two bounds equal. This proves the stated interval limit. The same reasoning supports normal approximations to intervals for sums and sample means after their endpoints are standardized.

Worked probability calculations

For a sample-mean interval \(l\le\bar X_n\le u\), standardize both endpoints using the standard error \(\sigma/\sqrt n\). For a sum interval, use the standard deviation \(\sigma\sqrt n\). The following examples show both forms and include a discrete sum, where the normal approximation is not an exact description of the possible values.

Choosing and interpreting an approximation

The population mean and variance determine the centering and scaling. In practice, one should first identify the observation-level distribution and calculate \(\mu\) and \(\sigma^2\), then transform the event, and only then evaluate \(\Phi\). This order helps prevent confusion between the standard deviation of one observation, \(\sigma\), and the standard deviation of the sum or mean.

The theorem guarantees convergence at fixed standardized thresholds, and its interval consequence follows because the limiting normal CDF is continuous. It does not give a universal sample-size rule or an error bound for every distribution. Strong skewness, heavy tails (even when the variance is finite), or a small sample can make an approximation inaccurate. Nor should one assume that a discrete sum’s normal approximation automatically accounts for its lattice of possible values; endpoint adjustments are a separate refinement.

Common mistake

A frequent error is to standardize a sample mean using \(\sigma\), as though it were a single observation. For example, if \(\sigma=2\) and \(n=25\), the standard deviation of \(\bar X_n\) is not \(2\); it is \(2/\sqrt{25}=0.4\). Using \(2\) would divide each deviation by a number five times too large and produce incorrect normal probabilities. For sums, the reverse scaling mistake is also common: their standard deviation is \(\sigma\sqrt n\), not \(\sigma/\sqrt n\). Check the scale against the exact identities \(S_n=\sum_i X_i\) and \(\bar X_n=S_n/n\).