Choosing the Inferential Target: Parameter, Mean, or Future Outcome

Start by Naming the Target

A linear-model question can ask about an unknown feature of the fitted mean structure, or about an outcome that has not yet been observed. Those are different inferential targets. Even when both are evaluated at the same design point, a regression mean and a future response have different variances and require different intervals. The first step is therefore not to look up a critical value; it is to state precisely what quantity is being estimated.

Consider the fixed-design normal linear model \(Y=X\beta+\varepsilon\), where \(Y\) is an \(n\)-vector, \(X\) is a known \(n\times p\) matrix of rank \(p\), \(\beta\) is an unknown \(p\)-vector, and \(\varepsilon\sim N_n(0,\sigma^2 I_n)\), with \(n>p\). Write \(\hat\beta=(X^{\mathsf T}X)^{-1}X^{\mathsf T}Y\), \(s^2=\mathrm{SSE}/(n-p)\), and \(\mathrm{SSE}=(Y-X\hat\beta)^{\mathsf T}(Y-X\hat\beta)\). All exact statements below are conditional on the fixed design matrix \(X\).

A regression mean is the expected response at the specified design point; it does not include the new observation's random error. In this tutorial, an exact prediction interval for \(Y_0\) assumes, in addition, that \(\varepsilon_0\sim N(0,\sigma^2)\) and that \(\varepsilon_0\) is independent of the original sample errors. That independence assumption matters: without it, the prediction pivot derived below need not have the stated distribution.

Match the Target to Its Variance

The least-squares estimator has covariance matrix \(\sigma^2(X^{\mathsf T}X)^{-1}\). Consequently, for any specified contrast \(c^{\mathsf T}\beta\), its estimator \(c^{\mathsf T}\hat\beta\) has variance \(\sigma^2 c^{\mathsf T}(X^{\mathsf T}X)^{-1}c\). For a mean at \(x_0\), use \(c=x_0\), giving variance \(\sigma^2 h_0\), where \(h_0=x_0^{\mathsf T}(X^{\mathsf T}X)^{-1}x_0\). A future response adds a new error with variance \(\sigma^2\), so its prediction-error variance is \(\sigma^2(1+h_0)\).

The coefficient contrast and the regression mean have the same mathematical form when the mean is written as the contrast \(x_0^{\mathsf T}\beta\). The distinction is one of the question being asked: a contrast may compare coefficients or combine them in a scientifically specified way, while a regression mean refers to an expected response at a design point. The future-response interval is different even when its center is the same as the mean interval.

Why the Exact t Intervals Work

The relevant exact result follows from normality and the orthogonality of fitted and residual components. Let \(P=X(X^{\mathsf T}X)^{-1}X^{\mathsf T}\) be the projection onto the column space of \(X\), and let \(M=I_n-P\) be the residual projection. The least-squares estimation error is \((X^{\mathsf T}X)^{-1}X^{\mathsf T}\varepsilon\), while the residual vector is \(M\varepsilon\).

Proof. Since \(\hat\beta-\beta=(X^{\mathsf T}X)^{-1}X^{\mathsf T}\varepsilon\), normality gives \(c^{\mathsf T}(\hat\beta-\beta)\sim N(0,\sigma^2 q)\), where \(q=c^{\mathsf T}(X^{\mathsf T}X)^{-1}c>0\). Also, \(\mathrm{SSE}=\varepsilon^{\mathsf T}M\varepsilon\). The two normal vectors \((X^{\mathsf T}X)^{-1}X^{\mathsf T}\varepsilon\) and \(M\varepsilon\) have zero cross-covariance because \(X^{\mathsf T}M=0\). Joint normality therefore makes them independent. The matrix \(M\) is an idempotent projection of rank \(n-p\), so \(\mathrm{SSE}/\sigma^2\sim\chi^2_{n-p}\); it is independent of the contrast estimator. Thus \(Z=[c^{\mathsf T}\hat\beta-c^{\mathsf T}\beta]/(\sigma\sqrt q)\sim N(0,1)\) is independent of \(U=\mathrm{SSE}/\sigma^2\sim\chi^2_{n-p}\). Since \(s^2=\sigma^2 U/(n-p)\), the displayed ratio is \(Z/\sqrt{U/(n-p)}\), which has a \(t_{n-p}\) distribution. \(\square\)

The theorem also gives the mean interval by taking \(c=x_0\), provided \(x_0\ne 0\). For the future response, the additional independent error changes the numerator, so it is useful to state and derive that result separately.

Proof. The prediction error is \(Y_0-x_0^{\mathsf T}\hat\beta=\varepsilon_0-x_0^{\mathsf T}(\hat\beta-\beta)\). The two terms on the right are independent centered normal variables: the first is independent of the original sample by assumption, and the second is a linear function of the original normal errors. Their variances are \(\sigma^2\) and \(\sigma^2h_0\), respectively. Hence their difference is normal with variance \(\sigma^2(1+h_0)\). The residual sum of squares, and therefore \(s^2\), depends only on the original sample residuals. The original sample's least-squares estimation error is independent of those residuals by the orthogonality argument in the preceding proof, and \(\varepsilon_0\) is independent of the entire original sample. Thus the prediction error is independent of \(s^2\). Dividing its standardized normal form by the independent square root of \((\mathrm{SSE}/\sigma^2)/(n-p)\), a scaled \(\chi^2_{n-p}\) variable, gives a \(t_{n-p}\) variable. \(\square\)

Worked Examples

A Practical Target-Selection Check

Before calculating, translate the question into one of three forms. A question about a slope, intercept, or specified linear combination of coefficients targets \(c^{\mathsf T}\beta\). A question about the expected response at a specified point targets \(x_0^{\mathsf T}\beta\). A question asking where one new observation is likely to fall targets \(Y_0\). The mean and future-response estimates have the same center, but their uncertainty is not the same.

A frequent mistake is to use a prediction interval whenever a problem mentions a particular design point. That reasoning confuses “the mean response at this point” with “a response that will be observed at this point.” The correction is to ask whether the target includes a new random error. If it does, include the \(1\) in \(1+h_0\); if it does not, use \(h_0\). Conversely, using a mean interval to describe a future individual omits the new outcome's variability and produces an interval that is too narrow for that target.

These formulas concern one scalar target at a time. When a question asks for simultaneous coverage of several contrasts or means, a collection of separate 95% intervals does not automatically give 95% joint coverage. That is a different inferential task. For a single target, however, naming the quantity correctly identifies the appropriate standard error and exact \(t\) interval.