Prediction Intervals for a Single Future Observation

Predicting One Future Response

At a specified set of predictor values, the mean response describes the model’s average outcome. A future observation at those same values will generally differ from that mean: it has its own random error. An interval for the mean therefore answers a different question from an interval for one future response.

A prediction interval must account for both sources of uncertainty: the fitted mean is estimated from a sample, and the future response varies around that mean. Under the normal linear model, these sources combine to give an exact finite-sample interval.

Model and notation: Let $Y=X\beta+\varepsilon$, where $Y$ is an $n$-vector of responses, $X$ is a fixed $n\times k$ design matrix of full column rank, and $\varepsilon\sim N_n(0,\sigma^2 I_n)$, with $\sigma^2>0$. Let $\hat{\beta}=(X'X)^{-1}X'Y$, $s^2=\mathrm{SSE}/(n-k)$, and $\mathrm{SSE}=(Y-X\hat{\beta})'(Y-X\hat{\beta})$. At the fixed design vector $x_0$, a single future response is $Y_{\mathrm{new}}=x_0'\beta+\varepsilon_{\mathrm{new}}$, where $\varepsilon_{\mathrm{new}}\sim N(0,\sigma^2)$ is independent of the errors used to fit the model.

Assume $n>k$, so there are residual degrees of freedom. Write $h_0=x_0'(X'X)^{-1}x_0$ for the leverage of the prediction point. The fitted response there is $\hat{Y}_0=x_0'\hat{\beta}$. For $0<\alpha<1$, let $t_{1-\alpha/2,\,n-k}$ denote the $1-\alpha/2$ quantile of the $t$ distribution with $n-k$ degrees of freedom.

Exact prediction interval: Under the stated model assumptions, an exact $100(1-\alpha)\%$ prediction interval for one future response at $x_0$ is $$ \hat{Y}_0 \ \pm\ t_{1-\alpha/2,\,n-k}\,s\sqrt{1+h_0}. $$ Equivalently, its endpoints are $\hat{Y}_0-t_{1-\alpha/2,\,n-k}s\sqrt{1+h_0}$ and $\hat{Y}_0+t_{1-\alpha/2,\,n-k}s\sqrt{1+h_0}$.

Why the Interval Has This Form

The quantity to predict is $Y_{\mathrm{new}}$, not just its model mean $x_0'\beta$. The prediction error is $Y_{\mathrm{new}}-\hat{Y}_0=\varepsilon_{\mathrm{new}}-x_0'(\hat{\beta}-\beta)$. The new error contributes variance $\sigma^2$, while estimating the mean contributes variance $\sigma^2h_0$. The next result establishes the exact distribution needed for the interval.

Result: Conditional on the fixed design matrix $X$ and fixed vector $x_0$, $$ \frac{Y_{\mathrm{new}}-\hat{Y}_0}{s\sqrt{1+h_0}}\sim t_{n-k}. $$

Proof. Put $H=X(X'X)^{-1}X'$ and $M=I_n-H$. Then $\hat{\beta}-\beta=(X'X)^{-1}X'\varepsilon$, while $\mathrm{SSE}=\varepsilon'M\varepsilon$. The matrices $H$ and $M$ are orthogonal projection matrices, $M X=0$, and $M$ has rank $n-k$. Since $\varepsilon$ is multivariate normal, the standard normal linear-model results give $\mathrm{SSE}/\sigma^2\sim\chi^2_{n-k}$.

The prediction error is a linear combination of independent normal errors, so it is normal with mean zero. Its variance is $$ \operatorname{Var}(Y_{\mathrm{new}}-\hat{Y}_0) =\operatorname{Var}(\varepsilon_{\mathrm{new}})+\operatorname{Var}\{x_0'(\hat{\beta}-\beta)\} =\sigma^2+\sigma^2x_0'(X'X)^{-1}x_0 =\sigma^2(1+h_0). $$ Here the covariance term is zero because $\varepsilon_{\mathrm{new}}$ is independent of the fitting errors. Also, $\hat{\beta}-\beta$ and $M\varepsilon$ are independent: their covariance is zero since $X'M=0$, and they are jointly normal. The future error is independent of both. Thus the prediction error is independent of $\mathrm{SSE}$. Dividing the normal prediction error by $\sigma\sqrt{1+h_0}$ gives a standard normal variable; dividing that variable by the independent square root of $(\mathrm{SSE}/\sigma^2)/(n-k)$ gives the stated $t$ distribution. $\square$

Coverage follows by inversion. The central $t$ quantile satisfies $P(-t_{1-\alpha/2,\,n-k}\leq T\leq t_{1-\alpha/2,\,n-k})=1-\alpha$ for $T\sim t_{n-k}$. Substituting the pivot and multiplying by the positive quantity $s\sqrt{1+h_0}$ yields $$ P\!\left(\hat{Y}_0-t_{1-\alpha/2,\,n-k}s\sqrt{1+h_0} \leq Y_{\mathrm{new}}\leq \hat{Y}_0+t_{1-\alpha/2,\,n-k}s\sqrt{1+h_0}\,\middle|\,X,x_0\right)=1-\alpha. $$ This is exact conditional coverage under the model, not an approximation that depends on a large sample.

Worked Calculations

Worked Example: An Intercept-Only Model

Suppose four independent observations $2,4,6,8$ come from the normal model $Y_i=\beta+\varepsilon_i$, with independent errors $N(0,\sigma^2)$. Construct a 95% prediction interval for one new observation from the same model.

There is one coefficient, so $k=1$ and the residual degrees of freedom are $4-1=3$. The fitted mean is $\hat{\beta}=(2+4+6+8)/4=5$. The residuals are $-3,-1,1,3$, giving $\mathrm{SSE}=9+1+1+9=20$ and $s^2=20/3$.

For an intercept-only model, $x_0=1$, $X'X=4$, and $h_0=1/4$. The estimated prediction standard error is therefore \[ s\sqrt{1+h_0} =\sqrt{\frac{20}{3}\left(1+\frac14\right)} =\sqrt{\frac{25}{3}} \approx 2.887. \] The 97.5th percentile of $t_3$ is approximately $3.182$, so the margin is $3.182(2.887)\approx9.19$. The interval is $5\pm9.19$, or approximately $(-4.19,14.19)$.

The wide range is possible even though the observed values run only from 2 to 8: it describes the uncertainty for one future draw, not a range that must contain only values already observed.

Worked Example: Prediction in a Simple Regression

Consider the five observations $(x,y)=(0,1),(1,3),(2,2),(3,5),(4,4)$, fitted with an intercept and a linear slope. Find a 95% prediction interval at $x_0=3$.

The sample means are $\bar{x}=2$ and $\bar{y}=3$. The centered predictor sum of squares is $S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10$. The centered cross-product is $(-2)(-2)+(-1)(0)+(0)(-1)+(1)(2)+(2)(1)=8$. Thus $\hat{\beta}_1=8/10=0.8$ and $\hat{\beta}_0=3-0.8(2)=1.4$. The fitted response at $x_0=3$ is $1.4+0.8(3)=3.8$.

The fitted values at $x=0,1,2,3,4$ are $1.4,2.2,3.0,3.8,4.6$. The residuals are $-0.4,0.8,-1,1.2,-0.6$, so $\mathrm{SSE}=0.16+0.64+1+1.44+0.36=3.60$. There are $5-2=3$ residual degrees of freedom, hence $s^2=3.6/3=1.2$.

For simple regression, $h_0=1/n+(x_0-\bar{x})^2/S_{xx}=1/5+(3-2)^2/10=0.3$. The prediction standard error is $\sqrt{1.2(1.3)}=\sqrt{1.56}\approx1.249$. Using $t_{0.975,3}\approx3.182$, the margin is approximately $3.182(1.249)=3.97$. The interval is $3.8\pm3.97$, or approximately $(-0.17,7.77)$.

Worked Example: Predicting Beyond the Observed Predictor Values

Fit a straight line to the four observations $(-1,0),(-1,2),(1,2),(1,4)$, and predict one response at $x_0=2$. This point lies outside the observed predictor range $[-1,1]$.

The predictor mean is $\bar{x}=0$, and $S_{xx}=(-1)^2+(-1)^2+1^2+1^2=4$. The response mean is $\bar{y}=2$, and $\sum (x_i-\bar{x})(y_i-\bar{y})=(-1)(-2)+(-1)(0)+(1)(0)+(1)(2)=4$. Therefore, the fitted slope is $4/4=1$, the intercept is $2$, and $\hat{Y}_0=2+1(2)=4$.

The fitted values are 1 at $x=-1$ and 3 at $x=1$. The residuals are $-1,1,-1,1$, giving $\mathrm{SSE}=4$. With $n-k=4-2=2$, $s^2=4/2=2$. The leverage at the prediction point is $h_0=1/4+(2-0)^2/4=5/4$.

The prediction standard error is $\sqrt{2(1+5/4)}=\sqrt{9/2}\approx2.121$. Using $t_{0.975,2}\approx4.303$, the margin is $4.303(2.121)\approx9.13$. The resulting 95% interval is $4\pm9.13$, or approximately $(-5.13,13.13)$. Its large width reflects the limited data and the prediction point’s distance from the observed predictor values; exact coverage still depends on the stated model assumptions.

Interpretation and a Common Mistake

A prediction interval concerns the random outcome of one future observation at the specified design point. Its coverage statement is conditional on the fixed design and the model assumptions: if the same procedure were repeated under the model, the interval would contain the future response with probability $1-\alpha$. It does not assert that a particular future response has probability $1-\alpha$ of lying in the already calculated interval.

Common mistake: using the mean-response standard error. The interval for the mean response uses $s\sqrt{h_0}$. Replacing $s\sqrt{1+h_0}$ with that quantity leaves out the future observation’s error variance, $\sigma^2$, and does not give the stated prediction interval.
Common mistake: using the wrong degrees of freedom. When $\sigma^2$ is estimated from the residuals, the quantile uses $n-k$, not $n$.
Common mistake: treating a prediction interval as assumption-free. The exact $t$ calibration relies on independent normal errors with a common variance and on the specified linear mean model.

At a fixed design point, the interval’s center is the fitted mean, but its purpose is to cover a new observation. Including both sources of variation is essential to matching the interval to that target.

Check Your Understanding

Use the model assumptions and interval construction above to answer these questions.

In the exact prediction pivot, what random quantity represents the new observation’s error?
For a model with $n=18$ observations and $k=4$ coefficients, what degrees of freedom are used for the $t$ quantile?
In an intercept-only model with $n$ observations, what is $h_0$ for predicting another observation from the same population?
Why is $s\sqrt{h_0}$ not the appropriate standard error for one future response?
Which model assumptions are needed for the stated finite-sample coverage to be exact?

Prediction Intervals for a Single Future Observation

What You'll Learn