Applying the CLT to Sample Means

Why Approximate the Distribution of a Sample Mean?

A sample mean summarizes many observations in one statistic. Even when individual measurements do not have a normal distribution, the central limit theorem (CLT) often makes the distribution of their mean approximately normal for a sufficiently large sample. This is useful when a question asks about the chance that an average measurement, processing time, or claim amount falls above or below a threshold.

The key is to model the sample mean, not to assume that each observation is normal. The center of the approximation is the population mean, while the variance is smaller than the variance of an individual observation by a factor of the sample size.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed random variables with population mean $E[X_i]=\mu$ and finite, positive population variance $\operatorname{Var}(X_i)=\sigma^2$. The sample mean is $\bar X_n=(X_1+\cdots+X_n)/n$. The CLT says that, as $n$ increases, the distribution of $\bar X_n$ is approximated by a normal distribution with mean $\mu$ and variance $\sigma^2/n$: $$ \bar X_n \ \mathrel{\dot\sim}\ N\left(\mu,\frac{\sigma^2}{n}\right). $$

Here $N(\mu,\sigma^2/n)$ uses the convention that the second parameter is the variance. The symbol $\mathrel{\dot\sim}$ indicates an approximation, not an exact equality in general. The approximation follows under the assumptions that the observations are IID and have a finite positive variance. For a fixed finite sample size, it need not be accurate to any specified degree; the theorem describes behavior as the sample size increases.

A convenient way to use the approximation is to compute the standard error, $\sigma/\sqrt n$, and express a threshold in units of that standard error. Let $\Phi(z)$ denote the cumulative distribution function of a standard normal random variable. For example, an upper-tail probability is approximated by $1-\Phi(z)$, where $z$ is the threshold minus $\mu$, divided by $\sigma/\sqrt n$. The following examples show how to identify the population quantities first, then use them for the sample mean.

From the CLT for Sums to the Sample Mean

The CLT for IID sums states that the sum of $n$ IID observations, after centering and scaling, approaches a standard normal distribution. Applying that result to the mean explains why its variance is $\sigma^2/n$.

\frac{\sum_{i=1}^n X_i-n\mu}{\sigma\sqrt n} = \frac{\sqrt n(\bar X_n-\mu)}{\sigma} \ \xrightarrow{d}\ N(0,1).

Since $\bar X_n=(\sum X_i)/n$, dividing the centered sum by $n$ gives the centered mean. Its scale is $\sigma/\sqrt n$, so the corresponding variance is $(\sigma/\sqrt n)^2=\sigma^2/n$. This is the justification for approximating $\bar X_n$ by $N(\mu,\sigma^2/n)$; it is not a separate assumption about the shape of the original population.

If the population itself is normal, then the sample mean is exactly normal for every positive integer $n$, not merely approximately normal. For nonnormal populations, the normal model for the mean is a CLT approximation. In either case, one must use the population variance and the actual sample size to find the mean’s variance.

Worked Applications

Worked Example: A Gamma-Distributed Measurement Average

A hypothetical measurement has a Gamma distribution with shape $4$ and scale $1$, in the stated measurement units. A random sample of $100$ independent measurements is collected. Use the CLT to approximate the probability that their sample mean exceeds $4.8$.

Method. Find the population mean and variance from the Gamma parameters, obtain the standard error of the sample mean, and evaluate the normal upper tail. The common mistake to avoid is using the variance of one measurement as the variance of the mean.

For a Gamma distribution with shape $\alpha$ and scale $\theta$, the mean is $\alpha\theta$ and the variance is $\alpha\theta^2$. Thus,

\mu=4(1)=4,\qquad \sigma^2=4(1)^2=4,\qquad \sigma=2.

For $n=100$, the approximate variance and standard error of the sample mean are

\operatorname{Var}(\bar X_{100})\approx\frac{4}{100}=0.04, \qquad \operatorname{SE}(\bar X_{100})=\sqrt{0.04}=0.2.

The threshold $4.8$ is $0.8$ units above the center $4$, or $0.8/0.2=4$ standard errors above it. Therefore,

P(\bar X_{100}>4.8) \approx 1-\Phi\left(\frac{4.8-4}{0.2}\right) =1-\Phi(4) \approx 0.0000317.

The approximate probability is about $0.000032$ (approximately $0.0000$ when rounded to four decimal places). The calculation checks out: $4.8$ is four standard errors above the mean, so the upper-tail probability must be very small. Using $\sigma=2$ directly as the standard deviation of the sample mean would give a substantially incorrect answer.

Worked Example: An Interval for a Uniform-Distribution Average

A hypothetical device produces independent readings uniformly distributed between $0$ and $12$. For a sample of $36$ readings, approximate the probability that the sample mean is between $5$ and $7$.

Method. Calculate the mean and variance of one uniform reading, convert these to the approximate mean and standard error of the sample mean, and find the probability between the two standardized endpoints. The common mistake to avoid is using the population standard deviation as the standard error.

For a uniform distribution on $[a,b]$, the mean is $(a+b)/2$ and the variance is $(b-a)^2/12$. Here,

\mu=\frac{0+12}{2}=6,\qquad \sigma^2=\frac{(12-0)^2}{12}=12.

With $n=36$, the approximate variance and standard error of the mean are

\operatorname{Var}(\bar X_{36})\approx\frac{12}{36}=\frac{1}{3}, \qquad \operatorname{SE}(\bar X_{36})=\sqrt{\frac{1}{3}}\approx0.5774.

The lower and upper endpoints are each one unit from the mean. Their standardized values are approximately $(5-6)/0.5774=-1.7321$ and $(7-6)/0.5774=1.7321$. Therefore,

P(5<\bar X_{36}<7) \approx \Phi(1.7321)-\Phi(-1.7321) \approx 0.9584-0.0416 =0.9168.

Using unrounded normal-table values gives approximately $0.9167$; the result to four decimal places is $0.9167$. The interval is symmetric about the population mean, and the two tail probabilities are equal, which checks the calculation.

Worked Example: An Average of Exponential Processing Times

Suppose, hypothetically, that independent processing times have an exponential distribution with mean $10$ minutes. A supervisor samples $64$ processing times. Use the CLT to approximate the probability that their average exceeds $12.5$ minutes.

Method. For an exponential distribution, the variance equals the square of the mean. Use it to calculate the sample-mean standard error, then evaluate the upper tail. The common mistake to avoid is treating $10$ minutes—the standard deviation of an individual time—as the standard error for the average.

The individual processing-time mean is $10$ minutes, and its variance is $10^2=100$ minutes squared. For $n=64$,

\operatorname{Var}(\bar X_{64})\approx\frac{100}{64}=1.5625, \qquad \operatorname{SE}(\bar X_{64})=\sqrt{1.5625}=1.25\text{ minutes}.

The threshold is $12.5-10=2.5$ minutes above the center, which is $2.5/1.25=2$ standard errors. Hence,

P(\bar X_{64}>12.5) \approx 1-\Phi\left(\frac{12.5-10}{1.25}\right) =1-\Phi(2) \approx 0.0228.

The probability is approximately $0.0228$, or $2.28\%$. Since the threshold is two standard errors above the mean, an upper-tail probability of a few percent is consistent with the normal approximation.

Assumptions, Interpretation, and Common Traps

The CLT approximation is useful only when the model and the question match its conditions. Before calculating, check that the observations can be treated as IID and that their population variance is finite. If observations are dependent or come from different distributions, this IID result cannot automatically be applied. The CLT also does not say that every individual observation is approximately normal.

The distinction between population variance and sample-mean variance is central:

Quantity	Mean	Variance
One observation $X_i$	$\mu$	$\sigma^2$
Sample mean $\bar X_n$, by the CLT approximation	$\mu$	$\sigma^2/n$
Standard error of $\bar X_n$	Not applicable	$\sigma/\sqrt n$ is its standard deviation

A larger sample size reduces the variance of the mean by the factor $n$ and its standard deviation by the factor $\sqrt n$. It does not change the population mean. This agrees with the earlier material on variance shrinkage and the weak law: sample means become more concentrated around the population mean as the sample size increases.

A frequent exam error is to report $\sigma^2$ as the variance of $\bar X_n$. For example, if $\sigma^2=25$ and $n=100$, the variance of the mean is $25/100=0.25$, and its standard error is $0.5$, not $5$. A second error is to apply the normal approximation to one observation merely because the sample size is large; the CLT concerns the distribution of a sum or average of many observations.

Key takeaway: Under the IID finite-variance conditions, approximate a sample mean by $N(\mu,\sigma^2/n)$. Keep the population variance, the variance of the mean, and the standard error distinct, and remember that a CLT approximation is not generally an exact distribution.

Check Your Understanding

For each problem, identify the population mean and variance before using the CLT approximation. Answers are rounded to four decimal places where appropriate.

Independent readings are uniform on $[2,8]$. For $n=36$, approximate $P(4.5<\bar X_{36}<5.5)$. (Answer: $0.9167$)
Independent waiting times are exponential with mean $8$ minutes. For $n=64$, approximate the probability that the sample mean exceeds $9$ minutes. (Answer: $0.1587$)
Independent observations are Bernoulli with success probability $0.4$. For $n=100$, approximate the probability that the sample proportion is between $0.35$ and $0.45$. (Answer: $0.6926$)
Independent observations have the exact normal distribution $N(50,100)$. For $n=25$, find the probability that the sample mean exceeds $53$, and state whether the result is exact or a CLT approximation. (Answer: $0.0668$, exact)
Independent measurements have a Gamma distribution with shape $2$ and scale $3$. For $n=72$, approximate the probability that their sample mean is below $5.25$. (Answer: $0.0668$)

Quantity	Mean	Variance
One observation \(X_i\)	\(\mu\)	\(\sigma^2\)
Sample mean \(\bar X_n\), by the CLT approximation	\(\mu\)	\(\sigma^2/n\)
Standard error of \(\bar X_n\)	Not applicable	\(\sigma/\sqrt n\) is its standard deviation

What You'll Learn