Degrees of Freedom and t-Based Calibration

Why Prediction Intervals Use a t Critical Value

A prediction interval for a new outcome must allow for two sources of uncertainty: the fitted regression line is estimated from a sample, and the new outcome has its own random error. The simple-regression prediction-interval formula combines these sources in a standard error. But that standard error contains an estimated residual standard deviation, not the unknown population standard deviation. Accounting for that estimation is why the interval uses a t critical value.

The degrees of freedom for that t value matter. In simple linear regression, the fitted line estimates two coefficients: an intercept and a slope. The residual variance is therefore estimated using \(n-2\) degrees of freedom, where \(n\) is the number of observations used to fit the regression. This tutorial connects that residual degrees of freedom to the t calibration and shows how to select a quantile for a specified prediction level.

The Model Behind the Calibration

The t-based result relies on a model, not just on the shape of a formula. For the training observations, suppose the predictor values \(x_i\) are fixed and the outcomes follow \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), where the errors are independent and each has distribution \(N(0,\sigma^2)\). For the new case, suppose its predictor value \(x_0\) is fixed and its error is independent of the training errors with the same \(N(0,\sigma^2)\) distribution. These conditions say that the mean is linear in the predictor, errors have a common variance, and the errors are normal and independent.

Write \(\hat{y}_0\) for the fitted mean at \(x_0\), and define \(S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2\). The simple-regression prediction standard error is \(s\sqrt{h_0}\), where \(h_0=1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\). The \(1\) accounts for the new outcome’s own error; the other terms account for uncertainty in the fitted mean. The corresponding prediction interval is centered on \(\hat{y}_0\), with its margin determined by this standard error and the chosen t critical value.

Here \(1-\alpha\) is the requested two-sided prediction level, and \(t_{q,\nu}\) denotes the \(q\) quantile of a t distribution with \(\nu\) degrees of freedom: if \(T\) has that distribution, then \(P(T\leq t_{q,\nu})=q\). The quantile is positive when \(q>0.5\). A 95% two-sided prediction interval has \(\alpha=0.05\), so its critical value is \(t_{0.975,n-2}\), not \(t_{0.95,n-2}\).

Why Estimating Residual Variance Leads to t

If \(\sigma\) were known, the prediction error could be divided by its true standard deviation to form a standard normal quantity under the model. In practice, \(\sigma\) is not known, so the calculation replaces it with \(s\). Because \(s\) varies from sample to sample, the resulting standardized prediction error has heavier tails than a standard normal variable. Under the stated normal-error model, it has a t distribution with the residual degrees of freedom.

The connection follows from two standard facts about normal linear regression. First, after fitting an intercept and a slope, the residual sum of squares divided by \(\sigma^2\) has a chi-square distribution with \(n-2\) degrees of freedom. Second, this scaled residual sum of squares is independent of the standardized prediction error when that error is formed using the true \(\sigma\). The fitted line uses two independent directions in the data—one for the intercept and one for the slope—leaving \(n-2\) residual directions to estimate the error variance.

To see how these facts produce the t statistic, let \(Y_0\) be the new outcome at \(x_0\), and let \(Z\) be the prediction error divided by its true standard deviation. Under the model, \(Z\sim N(0,1)\). Its true standard deviation is \(\sigma\sqrt{h_0}\), with \(h_0\) as defined above. The scaled residual sum of squares, \(U=\mathrm{SSE}/\sigma^2\), is independent of \(Z\) and has a chi-square distribution with \(n-2\) degrees of freedom. Substituting \(s^2=\sigma^2U/(n-2)\) into the prediction standard error gives the following standardized error:

This is the defining construction of a t random variable: a standard normal variable divided by the square root of an independent chi-square variable divided by its degrees of freedom. The denominator adds uncertainty because the residual standard deviation was estimated. The degrees of freedom are \(n-2\) because that is the degrees of freedom of the residual sum of squares—not because the new case adds an observation to the regression fit.

Selecting the Quantile for the Requested Level

The prediction level and the number of tails determine the quantile. For a central two-sided prediction interval with level \(C\), the total probability placed outside the interval is \(\alpha=1-C\). Half of that probability is in each tail, so the upper quantile is \(t_{1-\alpha/2,n-2}\). For a one-sided 95% upper prediction bound, all of the excluded 5% is in the upper tail, so the quantile is \(t_{0.95,n-2}\). A one-sided lower bound uses the corresponding lower-tail quantile, \(t_{0.05,n-2}\), which is the negative of \(t_{0.95,n-2}\) by symmetry.

Always identify both the requested level and whether the result is one-sided or two-sided before looking up a critical value. The same degrees of freedom can give different critical values for different quantiles. The table below gives the quantile choice; the actual numerical value still depends on \(n\).

Common Mistakes and Why the Choice Matters

A frequent error is to use \(n-1\) degrees of freedom because a sample standard deviation often uses \(n-1\). That is not the residual variance calculation in simple regression. The fitted intercept and slope use two degrees of freedom, so the residual variance estimate divides \(\mathrm{SSE}\) by \(n-2\). Another error is to select a quantile by the stated percentage alone. A 95% two-sided interval needs the 0.975 quantile; a 95% one-sided upper bound needs the 0.95 quantile.

A third mistake is to use a standard-normal critical value simply because the prediction error would be normal if \(\sigma\) were known. The standard deviation is estimated in the usual procedure. The t distribution accounts for that estimation, and its heavier tails generally produce a larger critical value than the normal distribution at the same central level, particularly with few residual degrees of freedom.

The calibration is only as appropriate as its assumptions. The examples explicitly treated the model conditions as reasonable; the numerical summaries alone cannot establish independence, constant error variance, or normality. The t critical value does not correct a wrongly specified mean relationship or other failures of the model. Its role is narrower: it accounts for the sampling variability introduced by estimating \(\sigma\) from regression residuals.