Why Estimating Spread Changes the Statistic
A sample mean varies from sample to sample. If the population standard deviation \(\sigma\) is known, the standard error of the sample mean is \(\sigma/\sqrt{n}\), and under normal sampling the standardized sample mean follows a standard normal distribution. In practice, \(\sigma\) is often unknown. Replacing it with the sample standard deviation \(s\) makes the denominator an estimate rather than a fixed quantity.
That replacement matters. The resulting statistic is not exactly standard normal: it follows a t distribution under the usual normal-sampling assumptions. The t distribution has heavier tails, reflecting the additional uncertainty from estimating the population spread. This is the reason a t procedure is used when standardizing a sample mean with \(s\).
The formula resembles the z statistic used when \(\sigma\) is known: $$ z=\frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}}. $$ The difference is not just a change of letter. When \(\sigma\) is known, the denominator is fixed apart from the sample size. When \(\sigma\) is estimated, the observed \(s\) varies from sample to sample, and the standardized result has more uncertainty.
Deriving the t Distribution
The derivation makes the source of that extra uncertainty precise. Assume \(X_1,\ldots,X_n\) are independent observations from a normal population with mean \(\mu\) and standard deviation \(\sigma\). The sample mean then satisfies $$ Z=\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\sim N(0,1). $$ If \(\sigma\) were known, \(Z\) would be the standardized statistic. But the sample standard deviation is random too, so the statistic actually used is $$ T=\frac{\bar{X}-\mu}{S/\sqrt{n}}. $$
To see the distribution of \(T\), standardize each observation as \(Y_i=(X_i-\mu)/\sigma\). The vector of these \(n\) values has a joint standard normal distribution. Resolve that vector into orthogonal coordinates: one coordinate points in the direction \((1,\ldots,1)\), and the other \(n-1\) coordinates are perpendicular to it. The first coordinate is \(Z\). The sum of squares of the other coordinates is $$ Q=\sum_{i=1}^{n}\frac{(X_i-\bar{X})^2}{\sigma^2} =\frac{(n-1)S^2}{\sigma^2}. $$
An orthogonal change of coordinates preserves the joint standard normal distribution. Its resulting coordinates are independent standard normal variables. Therefore, \(Q\) is the sum of squares of \(n-1\) independent standard normal variables, so \(Q\sim\chi^2_{n-1}\), and \(Q\) is independent of \(Z\). Substituting the expression for \(Q\) into the statistic gives
A t distribution with \(n-1\) degrees of freedom is defined as the distribution of this ratio: an independent standard normal variable divided by the square root of an independent chi-square variable divided by its degrees of freedom. Thus the t statistic is not an arbitrary substitute for \(z\). It follows from the sampling model and the use of \(S\) in place of the unknown \(\sigma\).
The denominator \(\sqrt{Q/(n-1)}\) fluctuates around 1. When it is relatively small, dividing by it produces a value of \(T\) farther from zero than \(Z\) would be. When it is relatively large, the ratio is pulled closer to zero. These fluctuations add probability to the tails compared with a standard normal distribution. As \(n\) grows, the estimate \(S\) tends to be more stable, and the t distribution approaches the standard normal distribution.
Worked Example: Standardizing with an Estimated Spread
Suppose a hypothetical independent random sample of \(n=9\) measurements comes from a normal population. Its sample mean is \(\bar{x}=104\) units and its sample standard deviation is \(s=12\) units. Compare standardizing the difference from \(\mu_0=100\) units when the population spread is estimated with standardizing it if \(\sigma=12\) units were known.
With the estimated spread: The estimated standard error is $$ SE=\frac{s}{\sqrt{n}}=\frac{12}{\sqrt{9}}=\frac{12}{3}=4\text{ units}. $$ So the t statistic is $$ t=\frac{\bar{x}-\mu_0}{s/\sqrt{n}} =\frac{104-100}{4} =1.000. $$ Under the stated independent normal-sampling assumption, this statistic is compared with a t distribution with \(9-1=8\) degrees of freedom.
If the population spread were known to be 12 units: The standard error would also be \(12/\sqrt{9}=4\) units, and the standardized value would be $$ z=\frac{104-100}{12/\sqrt{9}}=1.000. $$ The observed numerical values match, but their reference distributions differ: the estimated-spread statistic is t-distributed, while the known-spread statistic is standard normal. The t distribution accounts for the fact that the value 12 was estimated from the same sample.
How the Heavier Tails Affect Probabilities
Heavier tails matter when interpreting how unusual a standardized value would be under a model. A value such as \(t=2\) is not assigned the same tail probability as \(z=2\). For a two-sided comparison, the probability is the chance, under the reference model, of a statistic at least as far from zero as the observed value. The t distribution’s extra tail probability is the mathematical cost of estimating spread.
Worked Example: Comparing t and Normal Tail Probabilities
In a hypothetical independent random sample of \(n=30\) observations from a normal population, suppose \(\bar{x}=22\) units, \(s=\sqrt{30}\) units, and the reference mean is \(\mu_0=20\) units. Calculate the statistic and compare the two-sided tail probabilities under the t and standard normal distributions.
The estimated standard error is $$ SE=\frac{s}{\sqrt{n}} =\frac{\sqrt{30}}{\sqrt{30}} =1\text{ unit}. $$ Therefore, $$ t=\frac{22-20}{1}=2.000. $$ The independent normal-sampling assumption gives \(df=30-1=29\). The two-sided t probability is $$ P(|T_{29}|\ge 2)=2P(T_{29}\ge 2)\approx 0.05494. $$ For comparison, the corresponding standard normal probability is $$ P(|Z|\ge 2)=2P(Z\ge 2)\approx 0.04550. $$
For the same standardized distance of 2, the t distribution assigns more probability to values at least that far from zero. This comparison concerns the probability under each reference distribution; it does not tell us the probability that a particular reference mean is true.
Worked Example: The Same t Value with a Smaller Sample
Now suppose a different hypothetical independent random sample of \(n=5\) observations is drawn from a normal population. Let \(\bar{x}=12\) units, \(s=\sqrt{5}\) units, and \(\mu_0=10\) units. Find the t statistic and its two-sided tail probability.
The estimated standard error is $$ SE=\frac{s}{\sqrt{n}} =\frac{\sqrt{5}}{\sqrt{5}} =1\text{ unit}. $$ Thus, $$ t=\frac{\bar{x}-\mu_0}{s/\sqrt{n}} =\frac{12-10}{1} =2.000. $$ For this sample, the reference distribution is \(t_4\). Its two-sided tail probability is approximately $$ P(|T_4|\ge 2)=2P(T_4\ge 2)\approx 0.1161. $$
This probability is larger than the \(0.05494\) for \(t_{29}\) at the same observed value of 2. With fewer observations, the sample standard deviation provides less information about the population spread, so the t distribution has heavier tails. The sample size affects the reference distribution even though the calculated t value is the same.
Conditions and Scope of the Result
The exact t distribution derived here depends on the sampling model, not just on the formula for \(t\). The observations must be independent and drawn from a normal population. Under these conditions, the sample mean and sample variance have the independence needed for the ratio result. If the population is not normal, the exact result does not automatically apply, especially in a small sample.
The derivation explains why one-sample t methods are used to reason about a mean when the population standard deviation is unknown. It does not mean that every calculation involving \(\bar{x}\) and \(s\) automatically follows a t distribution. The sampling design and the population model still matter. The essential distinction is:
| Population spread | Standardizing denominator | Reference distribution under normal sampling |
|---|---|---|
| \(\sigma\) known | \(\sigma/\sqrt{n}\) | Standard normal |
| \(\sigma\) estimated by \(s\) | \(s/\sqrt{n}\) | t distribution, with \(n-1\) degrees of freedom |
Common Mistake: Treating the t Statistic as a z Statistic
A common mistake is to calculate \((\bar{x}-\mu_0)/(s/\sqrt{n})\) and then read its tail probability from the standard normal distribution just because the formula looks like a z statistic. The reasoning misses the random denominator: \(s\) changes across samples, so the statistic has additional variability. Use a t distribution when the population standard deviation is estimated by \(s\), subject to the model conditions. The difference is particularly consequential when the sample is small; with larger samples, the t distribution becomes closer to standard normal.
Check Your Understanding
Answer each question using the definitions and reasoning in this tutorial.
- Why does replacing \(\sigma\) with \(s\) change the reference distribution, even if the computed denominator happens to have the same numerical value?
- Under the normal-sampling model, what two independent random quantities form the ratio that defines a t variable?
- For the same observed statistic of 2, why is the two-sided tail probability larger for \(t_4\) than for \(t_{29}\)?
- What sampling assumptions support the exact t distribution for the one-sample statistic?
- A student calculates a one-sample statistic using \(s/\sqrt{n}\) but uses the standard normal distribution for its tail probability. What is the mistake?