The Normal Sample Mean

Why the Sample Mean Has an Exact Distribution

A sample mean is itself random: a different sample would generally produce a different value. When the observations are independent and normal, its distribution can be found exactly, not merely approximated for a large sample. The result identifies both its center and its spread, and it provides the basis for probability calculations about averages.

The key mathematical fact is that a sum of independent normal random variables is normal. Averaging is a particular rescaling of such a sum. The independence assumption matters because it lets us factor the moment-generating function of the sum into a product.

Definition: For observations $X_1,\ldots,X_n$, the sample mean is $$ \bar X=\frac{1}{n}\sum_{i=1}^n X_i. $$ Here $n$ is a positive integer. In an independent normal sample, each $X_i$ has a normal distribution, and the observations are mutually independent. If they also share a common mean $\mu$ and variance $\sigma^2$, the sample is called independent and identically distributed (iid).

The Distribution of a Mean of Independent Normals

We first state the result in a form that allows the observations to have different means and variances. Assume $X_1,\ldots,X_n$ are mutually independent and $X_i\sim N(\mu_i,\sigma_i^2)$, with $\sigma_i^2>0$. The mean of these observations is normal, with expectation equal to the average of their means and variance equal to the sum of their variances divided by $n^2$.

Theorem: If $X_1,\ldots,X_n$ are mutually independent with $X_i\sim N(\mu_i,\sigma_i^2)$, then $$ \bar X\sim N\left(\frac{1}{n}\sum_{i=1}^n\mu_i,\ \frac{1}{n^2}\sum_{i=1}^n\sigma_i^2\right). $$ In particular, if they are iid $N(\mu,\sigma^2)$, then $$ \bar X\sim N\left(\mu,\frac{\sigma^2}{n}\right). $$

The second argument of $N$ is a variance. Thus the standard deviation of the iid sample mean is $\sigma/\sqrt n$, not $\sigma/n$. Averaging leaves the center at $\mu$, while increasing $n$ decreases the variance.

Proof by Moment-Generating Functions

For a normal random variable $X_i\sim N(\mu_i,\sigma_i^2)$, its moment-generating function is $$ M_{X_i}(t)=E[e^{tX_i}]=\exp\left(\mu_i t+\frac{\sigma_i^2t^2}{2}\right),\qquad t\in\mathbb R. $$ This formula follows by completing the square in the normal density integral. In detail, the exponent in that integral satisfies $$ tx-\frac{(x-\mu_i)^2}{2\sigma_i^2} =-\frac{(x-\mu_i-\sigma_i^2t)^2}{2\sigma_i^2} +\mu_i t+\frac{\sigma_i^2t^2}{2}. $$ The shifted normal density integrates to one, leaving the stated expression.

Now apply independence to $\bar X$. For every real $t$, the required expectations exist, and $$ \begin{aligned} M_{\bar X}(t) &=E\left[\exp\left(\frac{t}{n}\sum_{i=1}^nX_i\right)\right]\\ &=\prod_{i=1}^n M_{X_i}\left(\frac{t}{n}\right)\\ &=\exp\left(\frac{t}{n}\sum_{i=1}^n\mu_i+ \frac{t^2}{2n^2}\sum_{i=1}^n\sigma_i^2\right). \end{aligned} $$ This is the moment-generating function of the normal distribution with mean $\sum_i\mu_i/n$ and variance $\sum_i\sigma_i^2/n^2$. Since the moment-generating function exists on an open interval containing zero, it determines the distribution uniquely. This proves the general result. If each $\mu_i=\mu$ and each $\sigma_i^2=\sigma^2$, substituting gives mean $n\mu/n=\mu$ and variance $n\sigma^2/n^2=\sigma^2/n$, proving the iid formula. $\square$

Worked Example: An Independent Sample with Different Variances

Suppose four mutually independent observations have distributions $X_1\sim N(2,1)$, $X_2\sim N(4,4)$, $X_3\sim N(6,9)$, and $X_4\sim N(8,16)$. Find the distribution of their sample mean.

The mean of the sample mean is the average of the four means: $$ E[\bar X]=\frac{2+4+6+8}{4}=5. $$ The variance is the sum of the four variances divided by $4^2$: $$ \operatorname{Var}(\bar X)=\frac{1+4+9+16}{16}=\frac{30}{16}=\frac{15}{8}. $$ The theorem therefore gives $$ \bar X\sim N\left(5,\frac{15}{8}\right). $$ Its standard deviation is $\sqrt{15/8}=\sqrt{30}/4$, approximately $1.369$. The observations need not have the same means or variances for their mean to be normal; mutual independence and normality of each observation suffice for this result.

Worked Example: A Probability About an iid Sample Mean

Let $X_1,\ldots,X_{16}$ be iid $N(50,8^2)$. Find $P(\bar X>53)$.

Here $\mu=50$, $\sigma^2=64$, and $n=16$. Thus $$ \bar X\sim N\left(50,\frac{64}{16}\right)=N(50,4), $$ so the standard deviation of $\bar X$ is $2$. Standardizing the event gives $$ P(\bar X>53) =P\left(\frac{\bar X-50}{2}>\frac{53-50}{2}\right) =P(Z>1.5), $$ where $Z\sim N(0,1)$. Using the standard normal distribution, $P(Z>1.5)\approx 0.0668$. This is an exact distributional reduction: it does not rely on a large-sample approximation.

Affine Transformations: Shifting and Rescaling the Mean

A location change adds a constant; a scale change multiplies a random variable by a constant. Together, these form an affine transformation. When applied to the sample mean, the shift changes its center but not its spread, while the scale changes its variance by the square of its coefficient.

Theorem: If $Y\sim N(m,v)$, where $v>0$, and $a,b$ are constants with $b\ne 0$, then $$ a+bY\sim N(a+bm,b^2v). $$ If $b=0$, then $a+bY=a$ with probability one, so the result is a degenerate constant rather than a normal distribution with positive variance.

To verify the theorem, use the moment-generating function of $Y$: $$ \begin{aligned} M_{a+bY}(t) &=E[e^{t(a+bY)}]\\ &=e^{at}M_Y(bt)\\ &=\exp\left((a+bm)t+\frac{b^2v\,t^2}{2}\right). \end{aligned} $$ This is the moment-generating function of $N(a+bm,b^2v)$, establishing the claim. The variance also follows directly: $\operatorname{Var}(a+bY)=b^2\operatorname{Var}(Y)=b^2v$. The factor is $b^2$ even when $b$ is negative; the standard deviation is multiplied by $|b|$. $\square$

Combining this theorem with the iid sample-mean result, if $X_1,\ldots,X_n$ are iid $N(\mu,\sigma^2)$, then $$ a+b\bar X\sim N\left(a+b\mu,\frac{b^2\sigma^2}{n}\right) $$ for $b\ne0$. In particular, a negative $b$ reverses the direction of the scale but does not produce a negative variance.

Worked Example: Rescaling and Shifting a Sample Mean

Let $X_1,\ldots,X_{25}$ be iid $N(10,4)$, and define $Y=2\bar X-5$. Find the distribution of $Y$.

First, the sample mean has distribution $$ \bar X\sim N\left(10,\frac{4}{25}\right)=N(10,0.16). $$ Its standard deviation is $\sqrt{0.16}=0.4$. Apply the affine transformation with $a=-5$ and $b=2$. Its mean is $$ E[Y]=-5+2(10)=15, $$ and its variance is $$ \operatorname{Var}(Y)=2^2(0.16)=0.64. $$ Therefore $Y\sim N(15,0.64)$, with standard deviation $\sqrt{0.64}=0.8$. The standard deviation doubles because the random variable is multiplied by $2$, which multiplies its variance by $2^2$. Subtracting $5$ shifts the mean but does not affect the variance.

Common Mistakes and Why the Result Matters

The exact sampling distribution turns statements about a random average into ordinary normal probability calculations. It also clarifies how precision changes with sample size: under the iid model, multiplying the sample size by four halves the standard deviation of the mean, because $\sigma/\sqrt{4n}=(\sigma/\sqrt n)/2$. This result is useful whenever a statistical calculation depends on the distribution of an average of independent normal observations.

Using $\sigma/n$ as the standard deviation. The variance is $\sigma^2/n$, so its square root is $\sigma/\sqrt n$.
Confusing a shift with a rescaling. Adding or subtracting a constant changes the mean, not the variance. Multiplication by $b$ changes the variance by $b^2$.
Assuming identical distributions are required. Independent normal observations with different means or variances still have a normal sample mean; use the general formula.
Ignoring independence. The proof factors a joint moment-generating function using mutual independence. Normal marginal distributions alone do not justify that factorization or the stated variance formula.
Calling the result an approximation. Under the stated normal and independence assumptions, the sample mean is exactly normal for every positive integer $n$.

Key takeaway: For iid $N(\mu,\sigma^2)$ observations, $\bar X\sim N(\mu,\sigma^2/n)$ exactly. Under $a+b\bar X$, the mean becomes $a+b\mu$ and the variance becomes $b^2\sigma^2/n$.

Check Your Understanding

Answer each question using the distribution formulas and assumptions stated above.

If $X_1,\ldots,X_9$ are iid $N(3,25)$, what are the mean, variance, and standard deviation of $\bar X$?
Three independent normal observations have means $1,2,3$ and variances $4,1,4$. Find the distribution of their sample mean.
If $X_1,\ldots,X_n$ are iid $N(\mu,\sigma^2)$, what is the distribution of $-3+2\bar X$?
Why does the variance of a sample mean with independent observations involve $n^2$ in the denominator before the iid simplification?
What assumption in the moment-generating-function proof permits the product of the individual moment-generating functions?

What You'll Learn