Prediction Intervals for New Gaussian Responses

Why Predicting a New Response Requires More Uncertainty

A confidence interval for the mean response at a covariate value \(x_0\) describes uncertainty about \(x_0^\mathsf{T}\beta\). A prediction interval asks a different question: where might one new response at \(x_0\) fall? Even if the mean were known exactly, a new Gaussian response would still vary around it. A prediction interval must account for both that variation and the uncertainty in estimating the mean.

We work with the identity-link Gaussian model and a fixed design matrix. The covariate vector \(x_0\) is treated as fixed, or the calculation is understood conditionally on its value. The interval is pointwise: it targets one specified \(x_0\), not simultaneous coverage over many covariate values.

Model and Prediction Variance

Let \(Y=(Y_1,\ldots,Y_n)^\mathsf{T}\) follow the Gaussian regression model

where \(X\) is a fixed \(n\times p\) design matrix of full column rank, \(n>p\), \(\beta\) is a \(p\)-vector, \(\sigma^2>0\) is the dispersion, and \(W=\operatorname{diag}(w_1,\ldots,w_n)\) has known positive precision weights. Thus, conditionally on the design, \(Y_i\) has variance \(\sigma^2/w_i\). The unweighted model is the special case \(w_i=1\) for every observation.

The weighted least-squares estimator and residual sum of squares are

For a specified new covariate vector \(x_0\), suppose \(Y_0\) is independent of the observed responses and follows \(N(x_0^\mathsf{T}\beta,\sigma^2/w_0)\), where \(w_0>0\) is its known precision weight. Write

The fitted mean \(x_0^\mathsf{T}\hat{\beta}\) has variance \(\sigma^2h_0\). The new response contributes a separate variance \(\sigma^2/w_0\). Because the new response and the fitted mean are independent, their difference has variance \(\sigma^2(h_0+1/w_0)\).

Exact Prediction Intervals

If \(\sigma^2\) is known, the prediction error is normal, so a standard normal quantile gives an exact interval. In most applications \(\sigma^2\) is estimated. For an exact finite-sample Student \(t\) interval, use the degrees-of-freedom-corrected residual variance estimate

The maximum-likelihood estimate \(Q/n\) is not the estimate used in this exact \(t\) pivot: its scaled form does not have the required chi-square degrees of freedom.

has a Student \(t\) distribution with \(n-p\) degrees of freedom. Consequently, an exact \(100(1-\alpha)\%\) prediction interval is

where \(t_{1-\alpha/2,\,n-p}\) is the \(1-\alpha/2\) quantile of the \(t\) distribution with \(n-p\) degrees of freedom.

Proof. Let \(A=W^{1/2}X\) and \(Z=W^{1/2}Y\). Then \(Z\sim N(A\beta,\sigma^2I)\), and the projection matrix onto the column space of \(A\) is \(P=A(A^\mathsf{T}A)^{-1}A^\mathsf{T}\). The residual sum of squares is \(Q=Z^\mathsf{T}(I-P)Z\). Since \(I-P\) is an idempotent matrix of rank \(n-p\), the normal projection result gives \(Q/\sigma^2\sim\chi^2_{n-p}\). The fitted coefficient is a linear function of \(Z\), while the residual vector is its orthogonal projection onto the column space’s complement. These projections of a multivariate normal vector are independent; therefore \(\hat{\beta}\) and \(Q\) are independent.

The estimation error \(x_0^\mathsf{T}\hat{\beta}-x_0^\mathsf{T}\beta\) is normal with mean zero and variance \(\sigma^2h_0\). Independently, \(Y_0-x_0^\mathsf{T}\beta\) is normal with mean zero and variance \(\sigma^2/w_0\). Their difference \(Y_0-x_0^\mathsf{T}\hat{\beta}\) is therefore normal with mean zero and variance \(\sigma^2(h_0+1/w_0)\). It is independent of \(Q\), because the future response is independent of the observed data and the fitted-mean error is independent of \(Q\). Finally, \(s^2/\sigma^2=(Q/\sigma^2)/(n-p)\). Dividing the standard normal prediction error by the square root of this independent chi-square quantity divided by its degrees of freedom gives a \(t_{n-p}\) variable. Inverting its central probability statement gives the displayed interval. \(\square\)

When \(\sigma^2\) is known, the same normal prediction error directly gives the exact interval

where \(z_{1-\alpha/2}\) is the standard normal quantile. This is a separate result from the \(t\) interval: replacing \(s\) by a known \(\sigma\) removes the extra uncertainty due to estimating dispersion.

Worked Calculations

Common Mistake and Interpretation

A frequent error is to use only the standard error for the fitted mean, \(s\sqrt{h_0}\), when predicting a new response. That quantity reflects uncertainty in \(x_0^\mathsf{T}\hat{\beta}\), but it omits the new response’s residual variance. The correct prediction standard error is \(s\sqrt{h_0+1/w_0}\). In an unweighted model, this becomes \(s\sqrt{h_0+1}\); omitting the 1 can make a prediction interval much too narrow.

The reverse mistake is to use the response variance alone and omit \(h_0\). That treats the fitted mean as if it were known. Both sources of uncertainty belong in the interval. A useful check is that \(h_0\geq0\), so the prediction standard error is at least as large as the new response’s residual standard deviation when dispersion is known. Prediction intervals also need not stay within a scientifically plausible range; their coverage claim follows from the Gaussian model, not from any extra restriction on response values.

These are model-based intervals for a single specified covariate vector. They do not automatically account for uncertainty in the covariates, model selection, or simultaneous prediction at multiple values of \(x_0\). Their exact \(t\) coverage relies on Gaussian errors, the stated variance structure, known precision weights, and the full-rank fixed-design assumptions used in the derivation.