Mean-Response Inference at Extrapolation Points

Why Extrapolation Affects Mean-Response Inference

A fitted regression model can produce a mean-response estimate at covariate values beyond those observed. The calculation is possible, but the estimate may be less precise: the design contains less direct information about the response mean at a distant point. To assess that uncertainty, look at the design-point variance factor, not simply at how large the fitted response is.

This tutorial focuses on that factor and its geometric interpretation. The inferential target is the mean response at a fixed design point. It is not an individual future outcome, which has a different uncertainty calculation.

The Exact Interval and Its Width

Under these assumptions, the exact two-sided \(100(1-\alpha)\%\) confidence interval for \(\mu_0\) is

Here \(t_{1-\alpha/2,n-p}\) is the indicated quantile of the \(t\) distribution. The standard error of the estimated mean response is \(s\sqrt{h_0}\), and the full interval width is

For a fixed fitted model, confidence level, and residual standard error, the width changes with \(x_0\) only through \(h_0\). This follows from the usual exact \(t\) result for a scalar linear contrast: \(x_0^T\hat{\beta}\) has variance \(\sigma^2h_0\), and its standardized version using the independent residual variance estimate has a \(t_{n-p}\) distribution. Thus, a larger \(h_0\) means a wider interval for the mean response.

Deriving the Centered-Design Formula

To interpret \(h_0\), first express it in terms of the predictor’s distance from the design center. Consider a model with an intercept and \(k\) predictors. Let \(Z\) be the \(n\times k\) matrix of uncentered predictor values, let \(\bar{z}\) be their vector of sample means, and define \(Z_c=Z-\mathbf{1}\bar{z}^T\). Set \(S=Z_c^TZ_c\), which is invertible when the full design has rank \(k+1\).

To prove the result, re-express the model with centered predictors. The design matrix is then \(X_c=[\mathbf{1},Z_c]\), and the intercept parameter changes to account for centering; the fitted mean does not change. Because the columns of \(Z_c\) sum to zero,

The design-point vector in these centered coordinates is \(x_{0c}=(1,(z_0-\bar{z})^T)^T\). Substitution gives

This is the variance factor for the estimated mean at the same covariate point under the centered parametrization, proving the formula. In particular, using \(z_0\) in the centered design would mix coordinate systems and would not establish this result. \(\square\)

The Geometry of the Variance Factor

The term \((z_0-\bar{z})^TS^{-1}(z_0-\bar{z})\) is a squared, design-dependent distance from the predictor center. It is not generally ordinary Euclidean distance: the matrix \(S^{-1}\) accounts for the scales and joint arrangement of the predictors. Equivalently, the term is the squared Euclidean length after transforming the centered coordinates by \(S^{-1/2}\).

Because \(S\) is positive definite, this squared distance is nonnegative and equals zero only at \(z_0=\bar{z}\). The factor \(h_0\) therefore takes its minimum value \(1/n\) at the predictor center. More generally, sets of points with the same distance term are ellipsoids centered at \(\bar{z}\). Moving outward in this geometry increases \(h_0\), and hence increases the mean-response interval width when \(s\) and the confidence level are fixed.

This monotonicity follows directly: if two points have distance terms \(q_1\) and \(q_2\), with \(q_2>q_1\geq0\), then \(h_2=1/n+q_2>1/n+q_1=h_1\). Since the square-root function is increasing on nonnegative numbers,

provided \(s>0\). If \(s=0\), both displayed widths are zero. This proves that, for positive residual standard error, greater centered-design distance yields a strictly wider interval. \(\square\)

In one predictor, the distance term is \((x_0-\bar{x})^2/S_{xx}\). A point just beyond the right endpoint has a larger factor than that endpoint; a point beyond the left endpoint has a larger factor than the left endpoint. But an outside point need not have a larger factor than every observed point: the opposite endpoint may be farther from the mean. With several predictors, being outside the convex hull of observed covariates also does not, by itself, guarantee a larger factor than at every observed point. The factor must be evaluated in the design’s geometry.

What Extrapolation Does and Does Not Establish

The interval width measures uncertainty about the mean response under the assumed linear model. It does not diagnose whether the model’s linear mean structure remains appropriate outside the observed covariate support. If the true mean bends, changes regime, or depends on omitted predictors, the exact normal-theory calculation does not protect against that misspecification. Extrapolation therefore calls for both a design-based uncertainty calculation and substantive judgment about the model.

A common mistake is to argue, “The fitted response is far from the observed responses, so its confidence interval must be wider.” The correction is that the width depends on \(s\), the degrees of freedom, the confidence level, and \(h_0\), not on the numerical value of \(x_0^T\hat{\beta}\). For example, adding a constant \(c\) to every response shifts the fitted mean by \(c\) but leaves the residuals, \(s\), and \(h_0\) unchanged; the interval shifts by \(c\) and keeps the same width. Conversely, a point with a large design factor has a wider mean-response interval even if its fitted mean happens to be near zero.

A practical assessment is to calculate \(h_0\), compare it with factors at relevant observed points, and report the resulting standard error or width. In one predictor, also state how far the point is beyond the observed range. With several predictors, describe the point relative to the joint design rather than checking each coordinate separately.