The Central Limit Theorem for Sample Means

Why Sample Means Can Have a Familiar Shape

A population can be uneven, lopsided, or concentrated on a few values. Yet if we repeatedly take independent samples from that population and calculate a mean each time, the distribution of those means can become approximately normal as the sample size grows. This result is the Central Limit Theorem (CLT). It explains why the normal distribution appears so often in the study of sample means, even when individual observations are not normally distributed.

The claim concerns a sampling distribution: the distribution of a statistic over repeated samples drawn in the same way. It does not say that the observations in any one sample become normal, or that the population itself becomes normal. The quantity whose distribution is described is the sample mean.

Definition: Let $X_1,X_2,\ldots,X_n$ be independent, identically distributed random variables with population mean $\mu$ and finite, positive population standard deviation $\sigma$. Their sample mean is $\bar X_n=(X_1+\cdots+X_n)/n$. The Central Limit Theorem states that, as $n$ increases, the distribution of the standardized sample mean $$ \frac{\bar X_n-\mu}{\sigma/\sqrt n} $$ converges in distribution to the standard normal distribution $N(0,1)$. Equivalently, for large $n$, the sampling distribution of $\bar X_n$ is approximately $N(\mu,\sigma^2/n)$.

Here, independent means that knowing the value of one observation does not change the probability distribution of another. Identically distributed means that every observation comes from the same population distribution, with the same $\mu$ and $\sigma$. The condition of finite, positive variance means $0<\sigma^2<\infty$. These conditions describe the version of the theorem used here.

The phrase “converges in distribution” has a precise meaning: for every fixed number $z$, the probability that the standardized sample mean is at most $z$ approaches the corresponding probability for a standard normal random variable as $n$ grows. It describes an improving approximation in the limit, not a promise that a particular finite sample size gives an exact normal distribution.

What the Approximate Normal Model Says

For a sufficiently large sample, the CLT gives a useful model for the sample mean: its center is $\mu$, and its variance is $\sigma^2/n$. Its standard deviation is therefore $\sigma/\sqrt n$, the standard error of the sample mean. The standard error has the same units as the observations; the variance has squared units.

The two forms in the definition express the same approximation in different scales. The standardized form measures how far $\bar X_n$ is from $\mu$ in standard-error units. The unstandardized form keeps the sample mean in the original measurement units. In both forms, the theorem concerns a random sample mean over repeated sampling, not a claim that an observed $\bar{x}$ itself is random after it has been calculated.

Worked Example: A Sample Mean From a Skewed Discrete Population

Suppose a hypothetical process produces independent observations $X_i$, each taking the values 2, 4, or 10 units with probabilities 0.50, 0.30, and 0.20, respectively. Consider the sample mean from $n=100$ observations. The population is not normally distributed: most values are near 2, with a smaller chance of a much larger value.

First calculate the population mean: $$ \mu=2(0.50)+4(0.30)+10(0.20)=1+1.2+2=4.2\text{ units}. $$

The population second moment is $$ E(X_i^2)=2^2(0.50)+4^2(0.30)+10^2(0.20)=2+4.8+20=26.8\text{ units}^2. $$ Thus the population variance and standard deviation are $$ \sigma^2=E(X_i^2)-\mu^2=26.8-(4.2)^2=9.16\text{ units}^2, \qquad \sigma=\sqrt{9.16}\approx3.027\text{ units}. $$

The observations are independent and identically distributed by the stated sampling setup, and the population variance is finite and positive. The CLT therefore gives the approximate sampling distribution $$ \bar X_{100}\ \approx\ N\left(4.2,\frac{9.16}{100}\right) =N(4.2,0.0916), $$ where the second parameter is the variance, in squared units. Equivalently, the sample mean has approximate standard error $\sqrt{0.0916}\approx0.3027$ units. The population values remain discrete and skewed; it is the distribution of the mean across repeated samples that the CLT approximates as normal.

A Short Derivation of the Standardized Form

The theorem’s scale comes from centering each observation at its population mean and measuring it in population standard-deviation units. Define $Y_i=(X_i-\mu)/\sigma$. Each $Y_i$ has mean 0 and variance 1. Substituting the definition of $\bar X_n$ gives $$ \frac{\bar X_n-\mu}{\sigma/\sqrt n} = \frac{\frac{1}{n}\sum_{i=1}^n(X_i-\mu)}{\sigma/\sqrt n} = \frac{1}{\sqrt n}\sum_{i=1}^n\frac{X_i-\mu}{\sigma} = \frac{1}{\sqrt n}\sum_{i=1}^nY_i. $$

This identity shows that the standardized sample mean is a sum of centered, unit-variance observations divided by $\sqrt n$. Independence makes the variance of that standardized sum equal to 1. The CLT supplies the further, important conclusion that its distribution approaches $N(0,1)$. The algebra explains the centering and scale; it does not by itself prove the normal limit.

Using the Theorem Carefully

To apply the CLT, first identify the observations and the sampling process. Check that the observations can reasonably be treated as independent, that they share the same population distribution, and that the population variance is finite and positive. Then express the approximation using the population mean and variance, with the sample size in the denominator of the variance.

Specify the observations.
Name what each $X_i$ measures and how the repeated sample is generated.

Check the model conditions.
Confirm independence, a common distribution, and finite positive population variance. If the sampling design or population does not support these assumptions, do not invoke this version of the theorem without qualification.

Describe the sample-mean model.
For a large enough $n$, use the approximate center $\mu$ and variance $\sigma^2/n$, and retain the original units for the mean.

State the approximation honestly.
Say that the sampling distribution is approximately normal, not that it is exactly normal unless some additional condition justifies exact normality.

Worked Example: Checking the Conditions in a Repeated Process

Imagine a hypothetical machine that fills sealed packets. For each packet in a model of the process, let $X_i$ be its fill amount in grams. Suppose the model specifies that each packet’s fill is generated independently from the same distribution, with population mean 50 grams and population standard deviation 4 grams. The variance is finite and positive: $\sigma^2=16\text{ grams}^2$.

For samples of $n=64$ packets, the CLT’s conditions are satisfied by the stated model. It gives the approximate sampling distribution $$ \bar X_{64}\ \approx\ N\left(50,\frac{16}{64}\right) =N(50,0.25). $$ The corresponding standard error is $\sqrt{0.25}=0.5$ grams. The model says that repeated sample means are centered at 50 grams and have a spread of about 0.5 grams under this approximation.

This conclusion depends on the model’s independence assumption. If packets are filled in a way that makes neighboring fill amounts dependent, the stated version of the CLT cannot automatically be applied as written. Merely counting 64 observations does not establish independence.

Worked Example: A Finite but Highly Uneven Population

Consider another hypothetical process in which each independent observation is 0, 1, 2, or 20 units, with probabilities 0.80, 0.10, 0.08, and 0.02. The rare value of 20 makes the population distribution highly uneven, but it still has a finite variance. Calculate its mean: $$ \mu=0(0.80)+1(0.10)+2(0.08)+20(0.02)=0.66\text{ units}. $$

Its second moment is $$ E(X_i^2)=0^2(0.80)+1^2(0.10)+2^2(0.08)+20^2(0.02)=8.42\text{ units}^2, $$ so $$ \sigma^2=8.42-(0.66)^2=7.9844\text{ units}^2. $$ For $n=400$, the CLT gives the approximate model $$ \bar X_{400}\ \approx\ N\left(0.66,\frac{7.9844}{400}\right) =N(0.66,0.019961). $$ The approximate standard error is $\sqrt{0.019961}\approx0.1413$ units.

The finite-variance and independence assumptions permit the CLT’s limiting conclusion. They do not, by themselves, provide a universal sample-size cutoff that guarantees a close approximation at a particular $n$. This calculation states the CLT-based model; it should not be mistaken for a proof that its accuracy is exact at $n=400$.

Why the CLT Matters—and a Common Mistake

The CLT links information about a population to the behavior of sample means. Even when a population’s shape is not normal, the theorem gives a basis for using a normal model for means from sufficiently large independent, identically distributed samples with finite variance. This is one reason that normal approximations are central in statistical inference.

A common mistake is to reason, “The sample size is large, so the observations are normally distributed.” That is not what the theorem says. The observations continue to follow the population distribution. The CLT concerns the sampling distribution of their mean. A related mistake is to treat “large” as a magic number: the theorem is a limiting result, and its assumptions and the degree of approximation matter.

The reverse overstatement is also incorrect: the CLT does not require the population to be normal. It applies under the stated independence, identical-distribution, and finite-variance conditions. Nor does the theorem claim that every population, sampling design, or statistic has this same limiting behavior. Its conclusion here is specifically about the standardized sample mean.

Key takeaway: For independent, identically distributed observations with finite, positive variance, the standardized sample mean approaches $N(0,1)$ as $n$ grows. Consequently, the sampling distribution of $\bar X_n$ is approximately $N(\mu,\sigma^2/n)$ for sufficiently large $n$; this is an approximation, not a claim that the population or individual observations are normal.

Check Your Understanding

Answer each question using the conditions and interpretation of the theorem.

A population has mean 12 and standard deviation 5. Under the CLT conditions, what are the approximate mean and variance of $\bar X_n$ for a sample of size $n$?
In the CLT, which distribution approaches a normal distribution: the population distribution, the distribution of individual observations, or the sampling distribution of the standardized sample mean?
Why does the theorem require a finite population variance in the version stated here?
A researcher has 100 measurements, but each measurement is influenced by the one immediately before it. Why is sample size alone not enough to justify applying this version of the CLT?
Explain why $N(\mu,\sigma^2/n)$ describes an approximate sampling distribution rather than necessarily an exact one.