Distinguishing Mean and Prediction Intervals

Two Different Questions, Two Different Intervals

A fitted regression model can be used to answer two related but distinct questions. First: what is the average response among observations with predictor values \(x_0\)? Second: what response might occur for one new observation at those same predictor values? The first concerns a conditional mean; the second concerns an individual response. Because individual responses vary around their mean, the second question requires a wider interval.

This distinction matters in applications such as estimating an average claim amount for a group of policies versus predicting the claim amount for one new policy. Both quantities are uncertain, but for different reasons. Estimating the mean is uncertain because the regression line is estimated from a finite sample. Predicting a new response has that same uncertainty, plus the new response’s own deviation from the mean.

The formulas below assume a correctly specified linear regression model with independent errors of mean zero and constant variance \(\sigma^2\). For exact \(t\)-intervals, also assume normally distributed errors. The predictor values are treated as fixed, and the model is treated as specified rather than selected using the same data. If the variance changes with predictors, observations are dependent, or the response model is misspecified, these standard intervals need not have their stated coverage.

Leverage and the Two Standard Errors

Write the multiple linear regression model as \(Y=X\beta+\varepsilon\), where \(X\) has \(p\) columns, including the intercept, and \(n\) rows. Let \(x_0\) be the predictor vector for the location of interest, including its intercept entry. The fitted conditional mean is \(\hat{Y}_0=x_0^\mathsf{T}\hat{\beta}\). Define the leverage at that location by

Let \(s=\sqrt{\mathrm{SSE}/(n-p)}\) be the residual standard error, with \(n-p\) residual degrees of freedom. For a two-sided interval with critical value \(t^*=t_{1-\alpha/2,n-p}\), the standard errors and intervals are:

For simple linear regression with an intercept and one predictor, \(p=2\), and the leverage has the convenient form

The term \(1/n\) reflects uncertainty in estimating the intercept or overall level. The second term reflects the extra uncertainty from estimating the slope when \(x_0\) is away from the sample mean. In multiple regression, the vector formula for \(h_0\) accounts for the location of all predictors together.

Why the Prediction Interval Has an Extra Term

The estimated mean has sampling variance \(\sigma^2h_0\). A new response can be written as its true conditional mean plus a new error: \(Y_{\text{new}}=x_0^\mathsf{T}\beta+\varepsilon_{\text{new}}\). Its prediction error is therefore

The new error has variance \(\sigma^2\). It is independent of the errors used to estimate \(\hat{\beta}\), so the covariance between the two terms is zero. The variance of the prediction error is consequently the sum of the two variances:

Replacing \(\sigma\) with \(s\) gives the prediction standard error. For the mean, there is no new response error to add, so its standard error uses \(s\sqrt{h_0}\) alone. Since \(h_0\geq 0\), \(1+h_0>h_0\), and thus the prediction interval is wider whenever the same positive critical value and residual standard error are used.

Predictor Location Changes Both Widths

In simple regression, leverage is smallest at \(x_0=\bar{x}\) and increases as \(x_0\) moves away from \(\bar{x}\). Consequently, both intervals widen away from the predictor center. This does not mean the mean and prediction intervals widen by the same amount: their standard errors involve \(\sqrt{h_0}\) and \(\sqrt{1+h_0}\), respectively.

Residual Variance Scales Interval Widths

For fixed predictors, \(h_0\), degrees of freedom, and critical value, each interval margin is proportional to \(s\). If \(s\) is multiplied by a factor \(c\), both widths are multiplied by \(c\). This is a statement holding the design and critical value fixed; in a refitted model, the estimated leverage or degrees of freedom might also change.

Interpretation and Common Exam Traps

A confidence interval for the conditional mean is not an interval containing a specified proportion of individual responses. A prediction interval is the appropriate standard interval for one new response, under the stated model assumptions. Neither interval is a guarantee about a particular future observation. Their coverage interpretation is repeated-sampling: under the model, the interval procedure covers the target mean or new response at its stated rate over repeated samples.

A frequent error is to use the mean standard error when asked for a new observation. For example, with \(h_0=0.2\), the incorrect prediction standard error \(s\sqrt{0.2}\) is smaller than the correct one \(s\sqrt{1.2}\). The correction is to include the extra \(1\) inside the prediction standard error’s square root. Another trap is to say that intervals always have the same width at every predictor value. In simple regression, the squared distance from \(\bar{x}\) changes leverage; in multiple regression, the vector location changes \(h_0\).

Finally, the standard formulas quantify uncertainty conditional on the regression model and its predictor values. They do not automatically account for model-selection uncertainty, omitted predictors, or violations of constant variance and independence. A numerically correct interval can still be misleading if these assumptions are substantially inappropriate.