Normal Populations and Exact Sampling Distributions

Two Routes to a Normal Sampling Distribution

A sample mean can have a normal distribution for two different reasons. If the observations come from a normal population and form an independent random sample, the sample mean is exactly normal for every sample size. If the population is not normal, the Central Limit Theorem (CLT) says that, under its conditions, the distribution of the sample mean becomes approximately normal as the sample size grows. These claims are related, but they are not interchangeable: one is an exact result for a normal population, and the other is an approximation for sufficiently large samples.

In both settings, the sampling distribution is the distribution of the values of \(\bar X\) over repeated samples of the same size from the same population. Its center is the population mean \(\mu\), and its standard error is \(\sigma/\sqrt{n}\) when the observations are independent and have population standard deviation \(\sigma\). The standard error describes variability among sample means, not variability among individual observations.

Definition: For independent, identically distributed observations with population mean \(\mu\) and finite population variance \(\sigma^2\), the sample mean has mean \(\mu\) and variance \(\sigma^2/n\). If the population is normal, its sampling distribution is exactly \(N(\mu,\sigma^2/n)\). If the population is not normal, the CLT gives an approximate \(N(\mu,\sigma^2/n)\) distribution for sufficiently large \(n\), under the theorem’s conditions.

The Exact Result for a Normal Population

Suppose \(X_1,\ldots,X_n\) are an independent random sample from a normal population, so that each observation has distribution \(N(\mu,\sigma^2)\). The sample mean is the scaled sum \(\bar X=(X_1+\cdots+X_n)/n\). Independent normal random variables have a normal sum, and multiplying a normal random variable by a constant preserves normality. Therefore \(\bar X\) is normal for any positive integer \(n\), including \(n=1\).

Its mean is \(\mu\), and its variance is \(\sigma^2/n\). Thus the exact sampling distribution is \(N(\mu,\sigma^2/n)\); its standard deviation, or standard error, is \(\sigma/\sqrt{n}\). In the notation \(N(\mu,\sigma^2)\), the second parameter is a variance, not a standard deviation.

Why the Exact Result Holds

A short derivation makes clear why the normal-population result is exact. The moment-generating function of a normal observation \(X_i\) is \(M_{X_i}(t)=\exp(\mu t+\sigma^2t^2/2)\). Independence means the moment-generating function of a sum is the product of the individual moment-generating functions. The sample mean is that sum multiplied by \(1/n\):

M_{\bar X}(t) = M_{(X_1+\cdots+X_n)/n}(t) = \prod_{i=1}^{n} M_{X_i}(t/n) = \left[\exp\left(\frac{\mu t}{n}+\frac{\sigma^2t^2}{2n^2}\right)\right]^n = \exp\left(\mu t+\frac{\sigma^2t^2}{2n}\right).

This is the moment-generating function of \(N(\mu,\sigma^2/n)\), so it identifies the exact distribution of \(\bar X\). The argument depends on the observations being independent and normally distributed. The mean and variance alone do not establish that a distribution is normal.

Worked Example: An Exact Sample-Mean Distribution

Suppose a hypothetical process produces independent measurements from a normal population with mean \(120\) millilitres and standard deviation \(15\) millilitres. A random sample of \(n=9\) measurements is taken. Find the sampling distribution’s mean, variance, and standard error.

Because the population is normal and the observations are an independent random sample, the exact normal-population result applies. The sample mean has mean \(\mu=120\) millilitres. Its variance is

\operatorname{Var}(\bar X)=\frac{\sigma^2}{n} =\frac{15^2}{9} =\frac{225}{9} =25\ \text{millilitres}^2.

The standard error is the square root of this variance:

SE(\bar X)=\frac{\sigma}{\sqrt n} =\frac{15}{\sqrt 9} =\frac{15}{3} =5\ \text{millilitres}.

Therefore, the exact sampling distribution is \(N(120,25)\): it is normal with mean \(120\) millilitres and standard deviation \(5\) millilitres. The individual measurements have standard deviation \(15\) millilitres; the sample means vary less, with standard error \(5\) millilitres.

What Changes for a Nonnormal Population?

For a nonnormal population, the sample mean is not generally exactly normal at a finite sample size. The CLT supplies a different kind of result: for independent observations from the same population with finite mean \(\mu\) and finite variance \(\sigma^2\), the distribution of the sample mean, after centering and scaling by its standard error, approaches a standard normal distribution as \(n\) increases. In practical terms, the distribution of \(\bar X\) can then be approximated by a normal distribution centered at \(\mu\) with variance \(\sigma^2/n\).

“Approaches” does not mean “becomes exactly normal at a particular sample size.” The accuracy depends on the population’s shape as well as on \(n\); there is no universal sample-size cutoff that guarantees a good approximation. With a small sample, a strongly skewed or discrete population can leave the sampling distribution visibly nonnormal. A normal approximation should therefore be described as an approximation, not as an exact distribution.

Worked Example: A Small Sample with a Discrete Distribution

Consider a hypothetical population in which an observation is \(10\) with probability \(0.25\) and \(0\) with probability \(0.75\). Take an independent random sample of \(n=4\). The population is discrete and strongly right-skewed, so it is not normal. Find the exact distribution of the sample mean and compare it with the CLT’s normal model.

Let \(K\) be the number of sampled observations equal to \(10\). Then \(\bar X=10K/4=2.5K\), and \(K\) has a binomial distribution with \(n=4\) and success probability \(0.25\). Thus the exact possible sample means and their probabilities are:

\begin{array}{c|ccccc} \bar x & 0 & 2.5 & 5.0 & 7.5 & 10.0\\ \hline P(\bar X=\bar x) & 0.3164 & 0.4219 & 0.2109 & 0.04688 & 0.003906 \end{array}

For example, \(P(\bar X=2.5)=P(K=1)=\binom{4}{1}(0.25)(0.75)^3=0.421875\). The exact distribution has only five possible values, and most of its probability is concentrated toward the lower end. It is not a normal distribution.

The population mean and variance are

\mu=(0)(0.75)+(10)(0.25)=2.5, \qquad \sigma^2=(0-2.5)^2(0.75)+(10-2.5)^2(0.25)=18.75.

Consequently, the sample mean has mean \(2.5\) and variance \(18.75/4=4.6875\), giving \(SE(\bar X)=\sqrt{4.6875}\approx2.165\). These are the same center and spread that the CLT would use for a normal approximation, but matching the mean and variance does not make the exact five-point distribution normal. With only four observations, that approximation should not be treated as an exact description.

Worked Example: A Larger Sample from the Same Population

Now take an independent random sample of \(n=100\) observations from the same hypothetical population: each observation is \(10\) with probability \(0.25\), and \(0\) otherwise. The population remains discrete and nonnormal, but the larger sample gives the CLT more opportunity to describe the sampling distribution of the mean.

The mean and variance of each observation remain \(\mu=2.5\) and \(\sigma^2=18.75\). Therefore, the sample mean has mean \(2.5\), variance \(18.75/100=0.1875\), and standard error

SE(\bar X)=\frac{\sqrt{18.75}}{\sqrt{100}} =\frac{4.330}{10} =0.4330.

The CLT-based model is therefore approximately \(N(2.5,0.1875)\). This is an approximation because the population is not normal; the CLT does not make individual observations normal or guarantee that the sample-mean distribution is exactly normal at \(n=100\). The exact sample mean is still discrete. However, compared with the sample of four, it is more reasonable to use a normal model for the distribution of the mean, subject to the approximation being adequate for the purpose at hand.

What the Two Results Have in Common

Both results identify the same center and standard error for a sample mean from independent observations with finite variance: the center is \(\mu\), and the standard error is \(\sigma/\sqrt{n}\). What differs is the claim about shape and the reason for it. A normal population gives an exact normal sampling distribution at every sample size. For a nonnormal population, the CLT supports a normal approximation as the sample size grows, but not exact normality at any stated finite \(n\).

Feature	Normal population	Nonnormal population under the CLT
Shape of the sample-mean distribution	Exactly normal	Approximately normal for sufficiently large \(n\)
Sample-size requirement for normality claim	Any positive integer \(n\)	Large enough for an adequate approximation; no universal cutoff
Mean of \(\bar X\)	\(\mu\)	\(\mu\)
Variance of \(\bar X\)	\(\sigma^2/n\)	\(\sigma^2/n\)
What to call the normal model	Exact distribution	Normal approximation

Common Mistakes and Why the Distinction Matters

Claiming that any sample mean is exactly normal. A mean and variance do not determine a distribution’s shape. Exact normality follows here from a normal population and independent observations; otherwise, the CLT generally provides an approximation.
Using the CLT to describe individual observations. The CLT concerns the sampling distribution of \(\bar X\), not the distribution of the original measurements. In the discrete example, individual measurements remained \(0\) or \(10\).
Treating a large sample size as a proof of exact normality. A larger \(n\) can improve a CLT approximation, but for a nonnormal population the sample mean’s distribution is not thereby declared exactly normal.
Confusing population spread and standard error. The population standard deviation \(\sigma\) describes individual observations; the standard error \(\sigma/\sqrt n\) describes the spread of sample means under the stated sampling assumptions.

The distinction matters whenever a later calculation relies on a normal model for \(\bar X\). Knowing whether that model is exact or approximate helps keep the assumptions and conclusions appropriately qualified. In particular, a reported normal model for a sample mean should be linked either to a normal population or to a justified CLT approximation.

Key takeaway: For an independent random sample from a normal population, \(\bar X\sim N(\mu,\sigma^2/n)\) exactly for every \(n\). For a nonnormal population with finite variance, the same normal model is a CLT-based approximation whose adequacy depends on sample size and population shape.

Check Your Understanding

Answer each question using the distinction between an exact distribution and an approximation.

A population is normal with mean \(30\) grams and standard deviation \(6\) grams. For an independent random sample of \(n=4\), give the exact distribution of \(\bar X\), including its variance and standard error.
A population is strongly skewed but has finite mean and variance. Does the CLT say that individual observations are approximately normal when \(n\) is large? Explain what distribution the CLT addresses instead.
Why does knowing that a sample mean has mean \(\mu\) and variance \(\sigma^2/n\) not establish that its distribution is normal?
A learner says, “With \(n=100\), the sample mean from any population is exactly normal.” Identify the error and give a more accurate statement.
If a population is normal, does the exact normality result require a large sample? State the sampling condition that is still needed.