Student's T from a Normal Sample

From a Normal Sample to a T Statistic

When a population standard deviation is known, a normal sample mean can be standardized using that value. When the standard deviation is unknown, it is natural to estimate it from the same sample. The resulting statistic is not standard normal: its denominator is random. Under normal sampling, however, that randomness has an exact distribution, and the statistic follows Student’s t distribution.

The key structure is a ratio. The standardized sample mean is standard normal, and the sample variance, when scaled by its degrees of freedom and the population variance, is chi-square. These two quantities are independent. This tutorial derives the ratio law and shows how the sample size determines its degrees of freedom.

Definition: For \(\nu>0\), let \(Z\sim N(0,1)\) and \(U\sim\chi^2_\nu\) be independent. The random variable \(T=Z/\sqrt{U/\nu}\) has a Student’s t distribution with \(\nu\) degrees of freedom, written \(T\sim t_\nu\). In a normal sample of size \(n\), the usual t statistic has \(\nu=n-1\) degrees of freedom.

Here \(\chi^2_\nu\) denotes a chi-square distribution with \(\nu\) degrees of freedom. We will use its density \(f_U(u)=u^{\nu/2-1}e^{-u/2}/(2^{\nu/2}\Gamma(\nu/2))\) for \(u>0\), where \(\Gamma\) is the gamma function. The chi-square degrees of freedom \(\nu\) are a positive integer in the sample construction below; the ratio definition and density also make sense for any real \(\nu>0\).

The Normal-Sample Decomposition

Suppose \(X_1,\ldots,X_n\) are independent and identically distributed as \(N(\mu,\sigma^2)\), with \(\sigma>0\) and \(n\ge 2\). Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\) and \(S^2=(n-1)^{-1}\sum_{i=1}^n(X_i-\bar X)^2\), the unbiased sample variance. The normal sample mean result gives \(\bar X\sim N(\mu,\sigma^2/n)\). To see why the residual sum of squares has \(n-1\) degrees of freedom and is independent of the mean, standardize the observations by setting \(Y_i=(X_i-\mu)/\sigma\). The vector \(Y=(Y_1,\ldots,Y_n)^\mathsf{T}\) has independent standard normal coordinates.

Let \(e=n^{-1/2}(1,\ldots,1)^\mathsf{T}\). Choose an orthogonal matrix \(Q\) whose first row is \(e^\mathsf{T}\). The joint density of \(Y\) is proportional to \(\exp(-Y^\mathsf{T}Y/2)\); orthogonal transformation preserves \(Y^\mathsf{T}Y\) and has Jacobian of absolute value one. Thus the coordinates of \(QY\) are again independent standard normals. Its first coordinate is \(e^\mathsf{T}Y=\sqrt n(\bar X-\mu)/\sigma\). The residual sum of squares is the squared length of the part of \(Y\) orthogonal to \(e\), so it is the sum of squares of the other \(n-1\) coordinates of \(QY\).

Theorem: For this independent normal sample, \(Z=\sqrt n(\bar X-\mu)/\sigma\sim N(0,1)\), \(U=(n-1)S^2/\sigma^2\sim\chi^2_{n-1}\), and \(Z\) and \(U\) are independent.

Indeed, \(U=\sum_{j=2}^n (QY)_j^2\), a sum of squares of \(n-1\) independent standard normals, so it has the stated chi-square distribution. The first coordinate of \(QY\) is independent of the remaining coordinates, and therefore \(Z\) is independent of \(U\). This identifies exactly where the degrees of freedom come from: centering the observations removes the one direction corresponding to their common average, leaving an \((n-1)\)-dimensional residual component. \(\square\)

Deriving the Student’s T Density

The theorem gives the sample statistic directly. Since \(S=\sigma\sqrt{U/(n-1)}\), cancellation of the unknown \(\sigma\) yields

\frac{\bar X-\mu}{S/\sqrt n} = \frac{\sqrt n(\bar X-\mu)/\sigma} {\sqrt{((n-1)S^2/\sigma^2)/(n-1)}} = \frac{Z}{\sqrt{U/(n-1)}}.

To make the distributional claim explicit, derive the density of the general ratio \(T=Z/\sqrt{U/\nu}\), where \(Z\) and \(U\) are independent as in the definition. For fixed \(u>0\), the transformation from \(t\) to \(z\) is \(z=t\sqrt{u/\nu}\), with Jacobian \(\sqrt{u/\nu}\). Integrating out \(u\) gives

\begin{aligned} f_T(t) &=\int_0^\infty \frac{1}{\sqrt{2\pi}} \exp\!\left(-\frac{t^2u}{2\nu}\right) \frac{u^{\nu/2-1}e^{-u/2}}{2^{\nu/2}\Gamma(\nu/2)} \sqrt{\frac{u}{\nu}}\,du\\ &=\frac{1}{\sqrt{2\pi\nu}\,2^{\nu/2}\Gamma(\nu/2)} \int_0^\infty u^{(\nu+1)/2-1} \exp\!\left[-\frac{u}{2}\left(1+\frac{t^2}{\nu}\right)\right]du. \end{aligned}

The integral is a gamma integral: for \(a>0\) and \(b>0\), \(\int_0^\infty u^{a-1}e^{-bu}\,du=\Gamma(a)b^{-a}\). Here \(a=(\nu+1)/2\) and \(b=(1+t^2/\nu)/2\). Substituting these values and simplifying gives the density.

Theorem: If \(T=Z/\sqrt{U/\nu}\), where \(Z\sim N(0,1)\), \(U\sim\chi^2_\nu\), and \(Z\) and \(U\) are independent, then

f_T(t)= \frac{\Gamma((\nu+1)/2)} {\sqrt{\nu\pi}\,\Gamma(\nu/2)} \left(1+\frac{t^2}{\nu}\right)^{-(\nu+1)/2}, \qquad -\infty<t<\infty.

Proof: The change of variables and gamma integral above give the density. In simplifying the constant, the factor \(2^{(\nu+1)/2}\) from the gamma integral combines with \(2^{-\nu/2}/\sqrt{2\pi\nu}\) to give \(1/\sqrt{\nu\pi}\). The remaining power of \(1+t^2/\nu\) is \(-(\nu+1)/2\). This is the Student’s t density with \(\nu\) degrees of freedom. \(\square\)

The density is symmetric about zero because it depends on \(t\) only through \(t^2\). It has heavier tails than the standard normal density: unusually large values are more plausible because the random denominator can be small. As \(\nu\) increases, the chi-square quantity \(U/\nu\) concentrates around one, so the ratio increasingly resembles \(Z\). The exact distribution remains t for each finite sample size; the limiting normal approximation is not needed for the result.

Worked Example: Calculating a T Statistic From a Sample

Suppose the observations are \(2,4,6,8,10\), independent from \(N(\mu,\sigma^2)\), and we calculate the statistic for the reference value \(\mu=5\). Find the sample mean, sample variance, t statistic, and its degrees of freedom.

The sample size is \(n=5\), and \(\bar X=(2+4+6+8+10)/5=6\). The squared deviations from 6 sum to \((2-6)^2+(4-6)^2+(6-6)^2+(8-6)^2+(10-6)^2=16+4+0+4+16=40\). Therefore \(S^2=40/(5-1)=10\) and \(S=\sqrt{10}\).

The statistic is \(t=\sqrt{5}(6-5)/\sqrt{10}=1/\sqrt{2}\), approximately \(0.707\). Under the stated normal model with mean equal to the reference value, its exact distribution is \(t_4\), because \(n-1=4\). The degrees of freedom come from the residual sum of squares, not from the numerator’s sample size by itself.

Worked Example: One Degree of Freedom Gives the Cauchy Distribution

For \(\nu=1\), the t density simplifies to \(f_T(t)=\Gamma(1)/(\sqrt{\pi}\Gamma(1/2))(1+t^2)^{-1}\). Since \(\Gamma(1)=1\) and \(\Gamma(1/2)=\sqrt{\pi}\), this is \(1/(\pi(1+t^2))\), the standard Cauchy density.

For a normal sample this case corresponds to \(n=2\). The sample t statistic has one degree of freedom; it does not have a finite mean or variance. For example, its right-tail probability beyond 1 is \(\int_1^\infty dt/(\pi(1+t^2))\). Using the antiderivative \(\arctan(t)/\pi\), this equals \(\lim_{b\to\infty}[\arctan(b)-\arctan(1)]/\pi=(\pi/2-\pi/4)/\pi=1/4\).

Worked Example: A Sample Statistic and Its Exact Tail Expression

A normal sample has \(n=9\), sample mean \(\bar X=12\), and sample standard deviation \(S=3\). For the reference mean \(\mu=10\), calculate the observed statistic and express its two-sided probability under that reference model.

The observed statistic is \(t=\sqrt{9}(12-10)/3=3(2)/3=2\). The degrees of freedom are \(9-1=8\), so under independent normal sampling with \(\mu=10\), the statistic has a \(t_8\) distribution. By symmetry, the probability of a statistic at least as far from zero as the observed value is

P(|T|\ge 2) = 2\int_2^\infty \frac{\Gamma(9/2)}{\sqrt{8\pi}\,\Gamma(4)} \left(1+\frac{t^2}{8}\right)^{-9/2}dt, \qquad T\sim t_8.

This expression uses the density with \(\nu=8\) and doubles the upper-tail area because the density is symmetric. It specifies the exact probability; a numerical value can be obtained by evaluating the integral or using a t-distribution table or calculator.

Moments and a Common Pitfall

The tail behavior explains why not every t distribution has the same finite moments. For example, if \(T\sim t_\nu\) and \(\nu>2\), then its variance is \(\nu/(\nu-2)\). To verify this, use independence and \(T^2=Z^2/(U/\nu)\). The chi-square density gives

E[U^{-1}] = \frac{1}{2^{\nu/2}\Gamma(\nu/2)} \int_0^\infty u^{\nu/2-2}e^{-u/2}\,du = \frac{\Gamma(\nu/2-1)}{2\Gamma(\nu/2)} = \frac{1}{\nu-2}, \qquad \nu>2.

The integral is finite precisely when \(\nu>2\); the last equality follows from \(\Gamma(x)=(x-1)\Gamma(x-1)\). Since \(E[Z^2]=1\), independence gives \(E[T^2]=\nu E[Z^2]E[U^{-1}]=\nu/(\nu-2)\). Symmetry and integrability give \(E[T]=0\) when \(\nu>1\). Thus \(\operatorname{Var}(T)=\nu/(\nu-2)\) for \(\nu>2\). For \(0<\nu\le1\), the mean does not exist; for \(1<\nu\le2\), the mean exists but the variance does not.

Common pitfall: Using the population standard deviation \(\sigma\) in the denominator gives the standard normal statistic \(Z=\sqrt n(\bar X-\mu)/\sigma\), not the sample t statistic. Replacing \(\sigma\) with \(S\) introduces the random factor \(\sqrt{U/(n-1)}\). Another frequent error is to use \(n\) degrees of freedom for the sample variance. Because the residuals sum to zero, only \(n-1\) independent normal components contribute to the residual sum of squares.

Key takeaway: For an independent normal sample, the statistic \((\bar X-\mu)/(S/\sqrt n)\) is exactly \(t_{n-1}\). This follows because its numerator, standardized by \(\sigma\), is standard normal and its independent variance estimate contributes a chi-square quantity with \(n-1\) degrees of freedom.

Check Your Understanding

Use the normal-sample ratio and density derived above to answer these questions.

For a normal sample of size \(n=12\), what is the degrees of freedom of the statistic \((\bar X-\mu)/(S/\sqrt n)\), and what chi-square variable appears in its denominator?
State the independence condition required in the ratio definition of the Student’s t distribution.
Why does the residual sum of squares have \(n-1\), rather than \(n\), degrees of freedom?
What standard distribution does \(t_1\) have, and what is \(P(T>1)\) for that distribution?
For which values of \(\nu\) does a \(t_\nu\) variable have a finite variance?