The Central Limit Theorem for Independent Identically Distributed Sums

Why the Central Limit Theorem Matters

A sum of many independent random variables can have a distribution that is difficult to calculate exactly. Even when each individual variable is not normally distributed, the sum, after centering and scaling, is often well approximated by a standard normal distribution. This makes it possible to estimate probabilities without deriving the exact distribution of the sum.

The central limit theorem (CLT) is not a claim that every sum is exactly normal. It is a limiting result: as the number of terms grows, the distribution of the standardized sum approaches the standard normal distribution under specified conditions. The method is aligned with the SOA Exam P (Probability) syllabus.

Definition: Let $X_1,X_2,\ldots$ be independent and identically distributed (iid) random variables with common mean $\mu=E[X_i]$ and common variance $\sigma^2=\operatorname{Var}(X_i)$, where $0<\sigma^2<\infty$. Define $S_n=X_1+\cdots+X_n$. The iid central limit theorem states that $$ \frac{S_n-n\mu}{\sigma\sqrt{n}}\ \xrightarrow{d}\ N(0,1) $$ as $n$ tends to infinity. Here $\xrightarrow{d}$ means convergence in distribution.

Standardizing a Sum

For a fixed, sufficiently large $n$, the theorem motivates the approximation $$ S_n\ \approx\ N(n\mu,n\sigma^2). $$ The second parameter in $N(\mu,\sigma^2)$ is the variance. Thus the approximate mean of the sum is $n\mu$, its variance is $n\sigma^2$, and its standard deviation is $\sigma\sqrt{n}$. For a threshold $c$, standardize using $$ z=\frac{c-n\mu}{\sigma\sqrt{n}}. $$ If $\Phi(z)$ denotes the standard normal cumulative distribution function, then, for example, $P(S_n\leq c)\approx\Phi(z)$.

For an interval, standardize both endpoints separately. If $a<b$, then $$ P(a\leq S_n\leq b)\ \approx\ \Phi\left(\frac{b-n\mu}{\sigma\sqrt{n}}\right) -\Phi\left(\frac{a-n\mu}{\sigma\sqrt{n}}\right). $$ The same approach applies to a strict upper-tail event: $P(S_n>c)\approx 1-\Phi((c-n\mu)/(\sigma\sqrt n))$. Be careful to use the common mean and variance of one summand, not the mean and variance of the sum, when identifying the model parameters.

Identify the common distribution.
Confirm that the summands are independent and identically distributed, and find the mean $\mu$ and variance $\sigma^2$ of one summand.

Find the sum’s center and scale.
Use $E[S_n]=n\mu$ and $\operatorname{Var}(S_n)=n\sigma^2$, so the standard deviation is $\sigma\sqrt n$.

Translate the event into z-scores.
For each boundary $c$, calculate $(c-n\mu)/(\sigma\sqrt n)$. Keep the direction of each inequality in the original event.

Use the standard normal distribution.
Evaluate the resulting lower-tail, upper-tail, or interval probability. Round the final probability, not intermediate z-scores unnecessarily.

Why the Standardized Sum Approaches Normality

The theorem’s scaling is not arbitrary. Adding $n$ independent summands adds their means and variances, so the sum is centered by subtracting $n\mu$ and scaled by its standard deviation $\sigma\sqrt n$. A short characteristic-function argument explains why the resulting distribution approaches the standard normal.

Let $Y_i=(X_i-\mu)/\sigma$. Each $Y_i$ has mean 0 and variance 1, and the standardized sum is $(Y_1+\cdots+Y_n)/\sqrt n$. For a random variable with mean 0 and finite variance 1, its characteristic function has the small-$u$ expansion $\varphi_Y(u)=1-u^2/2+o(u^2)$. Independence means characteristic functions multiply. Therefore, for fixed real $t$, the characteristic function of the standardized sum is $$ \left[\varphi_Y\left(\frac{t}{\sqrt n}\right)\right]^n = \left[1-\frac{t^2}{2n}+o\left(\frac1n\right)\right]^n \ \longrightarrow\ e^{-t^2/2}. $$

The function $e^{-t^2/2}$ is the characteristic function of $N(0,1)$. The continuity theorem for characteristic functions then gives convergence in distribution to that normal law. The expansion uses only a finite variance; it does not require the summands themselves to have normal distributions or even to have moment generating functions.

Worked Example: A Sum of Uniform Measurements

A hypothetical calibration process produces 108 independent measurements, each uniformly distributed on $[0,4]$. Let $S_{108}$ be their total. Approximate the probability that the total is between 204 and 240, inclusive. Use the CLT and round the final probability to four decimal places.

Method and model. The measurements are iid uniform variables on $[0,4]$, so their common mean and variance are $$ \mu=\frac{0+4}{2}=2,\qquad \sigma^2=\frac{(4-0)^2}{12}=\frac{16}{12}=\frac43. $$

The sum’s mean is $108(2)=216$. Its variance is $108(4/3)=144$, giving standard deviation $\sqrt{144}=12$.

Standardize the lower boundary: $$ z_{\mathrm{low}}=\frac{204-216}{12}=-1. $$ Standardize the upper boundary: $$ z_{\mathrm{high}}=\frac{240-216}{12}=2. $$

Thus the CLT approximation is $$ P(204\leq S_{108}\leq240)\approx\Phi(2)-\Phi(-1) =0.9772499\ldots-0.1586553\ldots =0.8185946\ldots\approx0.8186. $$

Check and trap avoided. The interval is centered at 216, but its lower and upper distances from the mean are 12 and 24, respectively, so the z-scores are not symmetric. Subtracting the lower-tail probability at $-1$ from the one at $2$ preserves that asymmetry. The reported probability is between 0 and 1, as required.

Worked Example: An Aggregate of Exponential Times

A hypothetical system processes 100 independent tasks. Each task’s processing time is exponentially distributed with rate $1/5$ per minute. Let $S_{100}$ be the total processing time. Approximate the probability that the total exceeds 550 minutes, rounding to four decimal places.

Method and model. The task times are iid exponential variables with rate $\lambda=1/5$. An exponential variable with rate $\lambda$ has mean $1/\lambda$ and variance $1/\lambda^2$. Therefore, $$ \mu=5,\qquad \sigma^2=25. $$

The total has mean $100(5)=500$ minutes and variance $100(25)=2500$ minutes squared. Its standard deviation is $\sqrt{2500}=50$ minutes. The threshold’s standardized value is $$ z=\frac{550-500}{50}=1. $$

The desired upper-tail probability is $$ P(S_{100}>550)\approx1-\Phi(1) =1-0.8413447\ldots =0.1586553\ldots\approx0.1587. $$

Check and trap avoided. The event asks for a total above 550, so the answer is the area to the right of $z=1$, not $\Phi(1)$. Also, the standard deviation of the total is 50, not its variance of 2500. The resulting upper-tail probability is plausible because the threshold is one standard deviation above the mean.

Worked Example: A Sum With Discrete, Non-Normal Summands

A hypothetical quality score $X_i$ for each component takes values 0, 3, or 6, each with probability $1/3$. Scores for 96 components are independent and identically distributed. Approximate the probability that their total $S_{96}$ lies from 240 to 336, inclusive. Round to four decimal places.

Method and model. First calculate the moments of one score: $$ E[X_i]=\frac{0+3+6}{3}=3,\qquad E[X_i^2]=\frac{0^2+3^2+6^2}{3}=\frac{45}{3}=15. $$ Consequently, $$ \operatorname{Var}(X_i)=E[X_i^2]-(E[X_i])^2=15-9=6. $$

For the total, the mean is $96(3)=288$ and the variance is $96(6)=576$. The standard deviation is $\sqrt{576}=24$. The two standardized boundaries are $$ z_{\mathrm{low}}=\frac{240-288}{24}=-2,\qquad z_{\mathrm{high}}=\frac{336-288}{24}=2. $$

Thus $$ P(240\leq S_{96}\leq336)\approx\Phi(2)-\Phi(-2) =0.9772499\ldots-0.0227501\ldots =0.9544997\ldots\approx0.9545. $$

Check and trap avoided. Although a single score is discrete and certainly not normally distributed, the iid CLT applies because it has a finite mean and a positive finite variance. The z-scores are symmetric about zero, so the approximate interval probability is about 0.9545. This is a normal approximation, not an assertion that the discrete total has an exact normal distribution.

Conditions, Limits, and Common Mistakes

Before using the iid CLT, check the assumptions. The terms must be independent, have the same distribution, and have a finite common mean and variance. The variance must be positive for the stated standardization. The theorem describes behavior as $n$ grows; it does not supply a universal cutoff at which an approximation becomes accurate. How large a sample is needed depends in part on the shape of the summand distribution. A highly skewed or heavy-tailed distribution can require more terms before the normal approximation is useful.

The most common computational error is using the variance where a standard deviation belongs. For iid summands, $$ \operatorname{Var}(S_n)=n\sigma^2 \quad\text{but}\quad \operatorname{SD}(S_n)=\sqrt{n\sigma^2}=\sigma\sqrt n. $$

For instance, in the exponential example, an incorrect calculation would divide the 50-minute distance from the mean by the sum’s variance: $$ \frac{550-500}{2500}=0.02. $$ That is not a z-score: it divides minutes by minutes squared. The correction is to divide by the sum’s standard deviation: $$ \frac{550-500}{\sqrt{2500}}=\frac{50}{50}=1. $$

Do not assume exact normality. The theorem gives an approximation for large $n$, not an exact distributional identity for a finite sum.
Do not omit the square root. The sum’s standard deviation grows as $\sqrt n$, not as $n$.
Do not reverse a tail. For a threshold above the mean, $P(S_n>c)$ uses $1-\Phi(z)$, not $\Phi(z)$.
Do not apply the result automatically to nonidentical summands. The theorem stated here is specifically the iid version.

For discrete totals, the CLT can still motivate a normal approximation, but the exact handling of integer boundaries and continuity corrections is a separate issue. The calculations here standardize the stated sum events directly; they do not add a continuity correction.

Key takeaway: For iid summands with finite common mean $\mu$ and positive finite common variance $\sigma^2$, center the total at $n\mu$ and scale it by $\sigma\sqrt n$. Then use the standard normal distribution to approximate probabilities, while remembering that the result is a large-$n$ approximation rather than an exact law.

Check Your Understanding

For each problem, find the one-summand mean and variance, standardize the sum event, and give the CLT approximation. Round probabilities to four decimal places.

One hundred iid measurements are uniform on $[0,6]$. Approximate the probability that their total is between 270 and 330. (Answer: approximately 0.9167)
Sixty-four iid processing times are exponential with rate $1/4$ per minute. Approximate the probability that their total exceeds 288 minutes. (Answer: approximately 0.1587)
Let $X_i$ be iid Bernoulli variables with $P(X_i=1)=P(X_i=0)=1/2$. For 144 trials, approximate the probability that the sum is between 66 and 78, inclusive. (Answer: approximately 0.6827)
Each of 54 independent items receives a score of 0, 2, or 4, each with probability $1/3$. Approximate the probability that the total score is between 84 and 132, inclusive. (Answer: approximately 0.9545)

What You'll Learn