Comparing Standardization with Known and Unknown Variance

Why the Denominator Matters

A sample mean varies from sample to sample. To calculate probabilities for that variation, standardize its distance from the population mean by dividing by a measure of its standard error. The choice of denominator matters: when the population standard deviation is known, standardization gives a standard normal variable; when it is estimated from the same sample, the resulting statistic follows Student’s t distribution under the normal-sample model.

The practical distinction is not simply that one distribution is used for “large samples” and the other for “small samples.” It is whether the population standard deviation is known or is replaced by the sample standard deviation. The t distribution accounts for the extra variability introduced by that estimate.

Definition: Let \(X_1,\ldots,X_n\) be an independent random sample from a normal population with mean \(\mu\) and standard deviation \(\sigma\). Write \(\bar{X}\) for the sample mean and \(S\) for the sample standard deviation, where \(S^2=\sum_{i=1}^{n}(X_i-\bar{X})^2/(n-1)\). If \(\sigma\) is known, the standardized sample mean is \(Z=(\bar{X}-\mu)/(\sigma/\sqrt{n})\), which has a standard normal distribution. If \(\sigma\) is unknown and \(S\) is used instead, \(T=(\bar{X}-\mu)/(S/\sqrt{n})\) has a Student t distribution with \(n-1\) degrees of freedom.

These are exact sampling distributions when the observations are independent and come from a normal population. “Exact” means that the stated distribution holds for the statistic across repeated samples under the model, not merely that it is a convenient approximation. If the population is not normal, the t result is not generally exact; a large-sample normal approximation may be appropriate under suitable conditions, but it is a different justification.

Known Population Standard Deviation: The Normal Statistic

When \(\sigma\) is known, the sample mean has mean \(\mu\) and standard deviation \(\sigma/\sqrt{n}\) for an independent sample from a normal population. This quantity is the standard error of \(\bar{X}\). Subtracting \(\mu\) and dividing by that standard error gives a standard normal variable, so probabilities can be evaluated using the standard normal cumulative distribution function \(\Phi\).

\bar{X}\sim N\left(\mu,\frac{\sigma^2}{n}\right), \qquad Z=\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\sim N(0,1).

For example, to find \(P(\bar{X}>c)\), first convert the cutoff to a z-score: \(z_c=(c-\mu)/(\sigma/\sqrt{n})\). Then \(P(\bar{X}>c)=1-\Phi(z_c)\). For an interval, standardize both endpoints and subtract their cumulative probabilities. The standard error is for the sample mean; using \(\sigma\) alone would omit the reduction in variability from averaging \(n\) observations.

Worked Example: A Two-Sided Probability with Known Variance

A hypothetical laboratory measures a normally distributed chemical concentration with population mean \(100\) units and known population standard deviation \(12\) units. An independent sample of \(16\) measurements is taken. Find the probability that the sample mean is between \(94\) and \(106\) units. The sampling distribution is normal with mean \(100\) and variance \(12^2/16\).

Method. Use the standard normal statistic because the population standard deviation is known. The standard error is \(\sigma/\sqrt{n}=12/\sqrt{16}=12/4=3\) units.

The lower endpoint has standardized value \((94-100)/3=-6/3=-2\). The upper endpoint has standardized value \((106-100)/3=6/3=2\). Therefore,

P(94<\bar{X}<106)=P(-2<Z<2) =\Phi(2)-\Phi(-2) =0.9772-0.0228 =0.9544\approx 0.9545.

Using standard normal values rounded to four decimals, the probability is \(0.9545\). Check: the interval is symmetric around the population mean, and the resulting probability is the familiar central area between \(-2\) and \(2\). A common mistake is to divide by \(12\) rather than by the standard error \(3\); that would standardize an individual observation, not the sample mean.

Unknown Population Standard Deviation: The Student t Statistic

When \(\sigma\) is unknown, replacing it with \(S\) makes the denominator random. Under the normal-sample assumptions, the standardized mean is no longer standard normal: it follows a Student t distribution. Its degrees of freedom are \(n-1\), and its tails are heavier than those of the standard normal distribution. Thus, for a given positive cutoff, a t variable generally has more probability beyond that cutoff than a standard normal variable.

T=\frac{\bar{X}-\mu}{S/\sqrt{n}}\sim t_{n-1}.

A t probability is calculated just as a normal probability is, except that the t cumulative distribution or a t table must be used with the correct degrees of freedom. For clarity, let \(t_{p,\nu}\) denote the positive value with upper-tail probability \(p\) for a t distribution with \(\nu\) degrees of freedom; that is, \(P(T_\nu>t_{p,\nu})=p\). A two-sided central probability of \(0.95\), for example, uses the two cutoffs \(-t_{0.025,\nu}\) and \(t_{0.025,\nu}\).

Why Studentizing Changes the Distribution

The difference follows from the relationship between the sample variance and the sample mean in a normal sample. Define \(U=(n-1)S^2/\sigma^2\). For a normal population, \(U\) has a chi-square distribution with \(n-1\) degrees of freedom, and it is independent of the standardized sample mean \(Z=(\bar{X}-\mu)/(\sigma/\sqrt{n})\). Studentizing replaces the fixed \(\sigma\) in \(Z\) with the random \(S\).

\frac{\bar{X}-\mu}{S/\sqrt{n}} = \frac{(\bar{X}-\mu)/(\sigma/\sqrt{n})}{S/\sigma} = \frac{Z}{\sqrt{U/(n-1)}}.

The standard t distribution is defined as the ratio of an independent standard normal variable to the square root of an independent chi-square variable divided by its degrees of freedom. Here \(Z\) and \(U\) have exactly those distributions and are independent. Consequently, the ratio has a t distribution with \(n-1\) degrees of freedom. Because its denominator can be smaller or larger than its typical value, the ratio is more variable than \(Z\), producing the heavier tails.

Worked Example: A Central Probability with an Estimated Standard Deviation

Suppose an independent sample of \(10\) observations is drawn from a normal population with mean \(72\) and unknown standard deviation. The sample standard deviation is \(S\). Find the probability that the sample mean is within \(2.262\) estimated standard errors of the population mean. Under the model, the statistic \(T=(\bar{X}-72)/(S/\sqrt{10})\) has a Student t distribution with \(9\) degrees of freedom.

Method. Express the event in terms of the t statistic: \(|\bar{X}-72|/(S/\sqrt{10})<2.262\) is equivalent to \(-2.262<T<2.262\). For \(9\) degrees of freedom, \(2.262\) is the t critical value with upper-tail probability \(0.025\).

P(-2.262<T<2.262) =1-P(T\leq -2.262)-P(T\geq 2.262) =1-0.025-0.025 =0.9500.

The probability is \(0.9500\). Check: the two excluded tails each have probability \(0.025\), so the central probability is \(1-0.050=0.950\). The mistake to avoid is using a standard normal critical value such as \(1.96\): the denominator here contains \(S\), so the statistic is t, not standard normal.

Comparing Probabilities on the Same Standardized Scale

The numerical cutoff alone does not determine a tail probability; the reference distribution matters. Comparing the two distributions at the same cutoff makes the effect of estimating \(\sigma\) visible. The following example uses the same underlying normal population for both calculations, but treats \(\sigma\) as known in one procedure and estimated in the other. The two standardized statistics have different sampling distributions.

Worked Example: The Same Cutoff Under Normal and t Distributions

A hypothetical normal population has mean \(40\) and standard deviation \(6\). Take an independent sample of \(6\) observations. Compare the two-sided probability beyond \(2\) for (a) the statistic using the known \(\sigma=6\), and (b) the statistic using the sample standard deviation \(S\). In (a), \(Z=(\bar{X}-40)/(6/\sqrt{6})\sim N(0,1)\); in (b), \(T=(\bar{X}-40)/(S/\sqrt{6})\sim t_5\).

Known \(\sigma\). By symmetry of the standard normal distribution, \(P(|Z|>2)=2[1-\Phi(2)]\). Using \(\Phi(2)=0.9772\),

P(|Z|>2)=2(1-0.9772)=2(0.0228)=0.0456\approx 0.0455.

Unknown \(\sigma\). For a t distribution with \(5\) degrees of freedom, the upper-tail probability at \(2\) is approximately \(0.0510\). By symmetry, the two tails together have probability

P(|T|>2)=2(0.0510)=0.1020\approx 0.1019.

Thus, the normal two-sided tail probability is approximately \(0.0455\), while the t probability is approximately \(0.1019\). The small difference in the last decimal comes from rounding table values. Check: the t tail is larger, as expected for a distribution with heavier tails. A common mistake is to use \(2[1-\Phi(2)]\) for both calculations. That is correct only for the statistic with known \(\sigma\); the statistic using \(S\) requires the t distribution with \(5\) degrees of freedom.

A Reliable Calculation Sequence

For a sample-mean probability, identify the model and denominator before looking up an area. This prevents a correct table lookup from being applied to the wrong statistic.

State the sampling assumptions.
For exact normal and t results, the observations must be an independent sample from a normal population.

Identify the standard deviation used.
If the population standard deviation \(\sigma\) is known, use \(\sigma/\sqrt{n}\). If it is unknown and replaced by \(S\), use \(S/\sqrt{n}\).

Select the distribution.
The known-\(\sigma\) statistic is standard normal. The unknown-\(\sigma\) statistic is Student t with \(n-1\) degrees of freedom.

Standardize endpoints and evaluate the area.
For a one-sided probability, take the appropriate tail. For an interval or two-sided event, include both endpoints or both tails as required.

Common Exam Traps and Interpretation

Both statistics measure how far the observed sample mean lies from \(\mu\), in estimated or known standard-error units. Their distributions differ because the unknown-\(\sigma\) statistic includes sample-to-sample variation in \(S\). In small samples, that extra variation has a noticeable effect on tail areas. As the degrees of freedom increase, the t distribution approaches the standard normal distribution, but that limiting behavior does not justify substituting a normal probability when an exact t calculation is requested.

Using \(\sigma\) instead of \(\sigma/\sqrt{n}\) for a sample mean. The standard deviation of an individual observation is not the standard error of the sample mean.
Choosing normal because the sample seems large. First check whether the problem specifies known \(\sigma\) or uses \(S\), and whether an exact normal-sample result is required.
Using the wrong t degrees of freedom. For the one-sample statistic defined here, the degrees of freedom are \(n-1\).
Forgetting a tail in a two-sided probability. A symmetric event such as \(P(|T|>c)\) includes both the lower and upper tails.
Treating an estimated denominator as fixed. \(S\) changes from sample to sample; replacing it with a fixed value does not produce the stated t sampling distribution.

Key takeaway: With a normal population, known \(\sigma\) gives a standard normal statistic; replacing \(\sigma\) by the sample standard deviation \(S\) gives a t statistic with \(n-1\) degrees of freedom. Choose the distribution from the denominator, then calculate the requested tail or interval probability.

Check Your Understanding

For each problem, identify the reference distribution before calculating the probability.

A normal population has mean \(120\) and known standard deviation \(18\). An independent sample of \(36\) observations is taken. Find \(P(117<\bar{X}<123)\). (Answer: \(0.6827\))
An independent sample of \(9\) observations comes from a normal population with mean \(50\) and unknown standard deviation. Find \(P(|\bar{X}-50|<2.306S/3)\). (Answer: \(0.9500\))
A normal population has mean \(30\) and known standard deviation \(4.2\). For an independent sample of \(7\), calculate the upper-tail probability \(P(Z>1.943)\). If the same cutoff is applied to the statistic using \(S\), state its upper-tail probability as well. (Answer: known \(\sigma\), \(0.0260\); estimated \(\sigma\), \(0.0500\))
An independent sample of \(5\) observations is drawn from a normal population with unknown standard deviation. Find the probability that its t statistic exceeds \(2.132\). (Answer: \(0.0500\))
A normal population has mean \(250\) and known standard deviation \(20\). For an independent sample of \(25\), find \(P(\bar{X}>258)\). (Answer: \(0.0228\))

What You'll Learn