Confidence Regions for a Mean Response at a Fixed Design Point

What Is Being Estimated at a Fixed Design Point?

In a regression model, the mean response at a specified design point is a parameter, not a future observed outcome. For example, a model may describe the expected outcome for a patient with a particular covariate profile, or the expected measurement at a fixed experimental setting. The confidence interval for this mean should reflect uncertainty in the estimated regression coefficients. It should not include the additional residual variation that would affect an individual future response.

Let \(Y\) be the \(n\)-vector of responses, \(X\) an \(n\times p\) fixed design matrix of full column rank, and \(\beta\) the \(p\)-vector of regression coefficients. Assume the normal linear model

where \(\sigma^2>0\). Let \(x_0\) be a fixed \(p\)-vector specifying the design point of interest. The mean response there is \(x_0^T\beta\). The ordinary least-squares estimator of \(\beta\) is \(\hat{\beta}=(X^TX)^{-1}X^TY\), so the estimated mean response is \(x_0^T\hat{\beta}\).

The uncertainty in this estimate is determined by the design-dependent factor

Because \(X^TX\) is positive definite, \(q_0\geq 0\), and it is positive whenever \(x_0\neq 0\). The standard nondegenerate interval below assumes \(q_0>0\).

The Exact Confidence Interval

Write \(\nu=n-p\) for the residual degrees of freedom, let \(\mathrm{SSE}=(Y-X\hat{\beta})^T(Y-X\hat{\beta})\), and define \(\hat{\sigma}^2=\mathrm{SSE}/\nu\). For a confidence level \(1-\alpha\), the exact interval is

Here \(t_{\nu,1-\alpha/2}\) is the \(1-\alpha/2\) quantile of the Student \(t\) distribution with \(\nu\) degrees of freedom. The interval’s estimated standard error is \(\hat{\sigma}\sqrt{q_0}\). The factor \(q_0\) describes uncertainty from estimating the regression coefficients at the chosen design point; it is not a residual variance added for a future outcome.

Proof. The least-squares estimator is a linear transformation of \(Y\), so

Consequently, \(x_0^T\hat{\beta}\) is normal with mean \(x_0^T\beta\) and variance \(\sigma^2q_0\). Thus

Let \(H=X(X^TX)^{-1}X^T\), the orthogonal projection onto the column space of \(X\). The residual vector is \((I_n-H)Y=(I_n-H)\varepsilon\), since \((I_n-H)X=0\). The matrices \(H\) and \(I_n-H\) project onto orthogonal subspaces. For a multivariate normal vector, these orthogonal projections are independent: their cross-covariance is \(\sigma^2H(I_n-H)=0\), and zero covariance implies independence for jointly normal vectors. Therefore \(\hat{\beta}\) and the residual vector are independent. Also, \(I_n-H\) has rank \(n-p=\nu\), so the squared length of the projected normal error, divided by \(\sigma^2\), has a chi-squared distribution with \(\nu\) degrees of freedom. In particular,

Since \(\hat{\sigma}^2=\mathrm{SSE}/\nu\), it follows that

By the definition of the two-sided \(t\) quantile, \(P(-t_{\nu,1-\alpha/2}\leq T\leq t_{\nu,1-\alpha/2})=1-\alpha\). Rearranging these inequalities to isolate \(x_0^T\beta\) gives exactly the stated interval. \(\square\)

Why the Standard Error Has This Form

The interval’s standard error follows directly from the covariance matrix of \(\hat{\beta}\). This calculation also shows how uncertainty in the estimated mean depends on the particular \(x_0\).

Derivation. For any fixed vector \(a\), the variance of \(a^T\hat{\beta}\) is \(a^T\operatorname{Var}(\hat{\beta})a\). Taking \(a=x_0\) and using \(\operatorname{Var}(\hat{\beta})=\sigma^2(X^TX)^{-1}\) gives

Taking the square root gives the standard deviation \(\sigma\sqrt{q_0}\); estimating \(\sigma\) by \(\hat{\sigma}\) gives the standard error. Multiplying the interval’s half-width by two gives its stated total width. \(\square\)

This is a scalar confidence interval for one specified linear combination of coefficients. It is not generally the same as taking a joint confidence ellipsoid for all of \(\beta\) and projecting that ellipsoid onto \(x_0^T\beta\). That projection is a valid region if the joint ellipsoid has the required coverage, but its calibration is for simultaneous coefficient coverage and generally produces a different, wider interval. The direct \(t\) calculation uses the exact distribution of the particular contrast \(x_0^T\hat{\beta}\).

Worked Confidence Intervals

Mean Uncertainty Is Not Future-Response Variation

The distinction is easiest to see by writing the two sources of randomness separately. The fitted mean \(x_0^T\hat{\beta}\) varies from sample to sample because \(\hat{\beta}\) varies. A future response at the same design point also varies around its mean because of the model error. The interval derived here is for \(x_0^T\beta\); its estimated standard error is \(\hat{\sigma}\sqrt{q_0}\). It does not add a separate \(\hat{\sigma}^2\) term for the future response’s residual.

A common mistake is to use a standard error involving \(\hat{\sigma}\sqrt{1+q_0}\) when the target is the mean response. That extra \(1\) represents residual variation for a future individual observation, not uncertainty in the mean itself. Using it for a mean-response interval answers a different question and makes the interval unnecessarily wide for the stated target. Conversely, omitting residual variation when the target is a future observation would understate uncertainty; prediction intervals are treated separately.

The point \(x_0\) need not be one of the observed design rows for the algebraic interval formula to apply. The model assumptions must still be credible at that point. In particular, the formula does not make an extrapolated mean reliable merely because it produces a numerical interval. The value of \(q_0\) captures the design-matrix contribution to standard error, while assessing an extrapolation’s plausibility requires attention to the model and design.