Deriving a Prediction Interval for a Future Response

Why Predicting a Future Response Requires Two Sources of Uncertainty

A confidence interval for a mean response concerns the unknown average response at a specified predictor value. A prediction interval concerns a different quantity: the response of one new individual or unit at that predictor value. Even if the regression mean were known exactly, a new response would still vary around it because of its own error. A prediction interval must account for both that new error and the uncertainty in the estimated mean.

This tutorial derives the standard prediction interval for one future response in simple linear regression. The derivation is aligned with the SOA VEE Mathematical Statistics syllabus. It assumes the usual normal-error regression model and a future observation independent of the data used to fit the regression.

Model and assumptions: For observed responses $Y_i$ at fixed predictor values $x_i$, $i=1,\ldots,n$, assume $$ Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad \varepsilon_i\ \text{independent and identically distributed as }N(0,\sigma^2). $$ The predictor values are treated as fixed, the conditional mean is linear, and the errors have a common variance. For an exact t-based interval, the errors must be normal. The future response at the specified value $x_0$ is $Y_{\mathrm{new}}=\beta_0+\beta_1x_0+\varepsilon_{\mathrm{new}}$, where $\varepsilon_{\mathrm{new}}\sim N(0,\sigma^2)$ is independent of the observed errors. Require $n\geq 3$ and at least two distinct observed predictor values; these conditions give a rank-2 regression design and positive residual degrees of freedom.

Notation and the Target of Prediction

Let $\bar{x}$ be the sample mean of the predictor values and let $S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2$. The least-squares estimates of the intercept and slope are $b_0$ and $b_1$. At a specified predictor value $x_0$, the estimated mean response is $\hat{Y}_0=b_0+b_1x_0$. The observed residual sum of squares is $\mathrm{SSE}=\sum_{i=1}^n(Y_i-b_0-b_1x_i)^2$, and the residual mean square and residual standard error are

\mathrm{MSE}=\frac{\mathrm{SSE}}{n-2}, \qquad s=\sqrt{\mathrm{MSE}}.

The target is $Y_{\mathrm{new}}$, not just its conditional mean. The prediction error is therefore $Y_{\mathrm{new}}-\hat{Y}_0$. It contains a component from estimating the regression mean and a separate component from the new response’s error.

Definition: A two-sided pointwise $100(1-\alpha)\%$ prediction interval for one future response at $x_0$ is an interval constructed to have probability $1-\alpha$ of containing $Y_{\mathrm{new}}$, under the stated model and for that specified $x_0$. It is not an interval for the mean response and is not a simultaneous interval covering future responses at many predictor values.

Deriving the Prediction Standard Error

First consider uncertainty in the estimated mean response. The fitted value at $x_0$ can be expressed as a weighted sum of the observed responses. This representation makes its variance visible.

\hat{Y}_0 =\bar{Y}+b_1(x_0-\bar{x}) =\sum_{i=1}^n \left[ \frac{1}{n}+ \frac{(x_0-\bar{x})(x_i-\bar{x})}{S_{xx}} \right]Y_i.

The equality follows from the least-squares slope formula $b_1=\sum_i(x_i-\bar{x})Y_i/S_{xx}$ and $\bar{Y}=\sum_iY_i/n$. Define the weight in brackets as $w_i$. Since the observed responses are independent and each has error variance $\sigma^2$, the variance of their weighted sum is $\sigma^2\sum_iw_i^2$. Expanding that sum gives the variance of the estimated mean.

\begin{aligned} \sum_{i=1}^n w_i^2 &=\sum_{i=1}^n \left[ \frac{1}{n}+ \frac{(x_0-\bar{x})(x_i-\bar{x})}{S_{xx}} \right]^2\\ &=\frac{1}{n} +\frac{2(x_0-\bar{x})}{nS_{xx}}\sum_{i=1}^n(x_i-\bar{x}) +\frac{(x_0-\bar{x})^2}{S_{xx}^2} \sum_{i=1}^n(x_i-\bar{x})^2\\ &=\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}. \end{aligned}

The middle term vanishes because deviations from $\bar{x}$ sum to zero; the final sum of squared deviations is $S_{xx}$. Thus the estimated mean has variance $\sigma^2[1/n+(x_0-\bar{x})^2/S_{xx}]$. For a future observation, subtracting the fitted mean also leaves the independent error $\varepsilon_{\mathrm{new}}$. Its variance adds to the variance of the estimated mean, giving the prediction-error variance.

\begin{aligned} \operatorname{Var}(Y_{\mathrm{new}}-\hat{Y}_0) &=\operatorname{Var}(\varepsilon_{\mathrm{new}}) +\operatorname{Var}(\hat{Y}_0)\\ &=\sigma^2\left[ 1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}} \right]. \end{aligned}

The addition is justified by independence: the future error is independent of the observed errors and therefore independent of $\hat{Y}_0$. The first $1$ in the brackets is the future observation’s own error variance, in units of $\sigma^2$. The other terms describe uncertainty in the estimated mean.

Replacing the Unknown Error Variance

In practice, $\sigma$ is unknown, so the prediction standard error uses $s=\sqrt{\mathrm{MSE}}$. Under the normal-error model, $\mathrm{SSE}/\sigma^2$ has a chi-square distribution with $n-2$ degrees of freedom and is independent of the fitted-value errors. The degrees of freedom are $n-2$ because estimating the intercept and slope uses two independent regression parameters. The future error is also independent of the data used to calculate $s$.

Consequently, the prediction error divided by its estimated standard error has a Student t distribution with $n-2$ degrees of freedom. If $t_{1-\alpha/2,n-2}$ denotes the $1-\alpha/2$ quantile of that distribution, the interval is:

Theorem: Under the stated model, an exact two-sided $100(1-\alpha)\%$ prediction interval for one future response at $x_0$ is $$ b_0+b_1x_0 \ \pm\ t_{1-\alpha/2,n-2}\, s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}. $$ The critical value uses $n-2$ degrees of freedom, and the square-root factor includes the additional $1$ for the future response’s own error.

Equivalently, the pivot is

\frac{Y_{\mathrm{new}}-\hat{Y}_0} {s\sqrt{1+1/n+(x_0-\bar{x})^2/S_{xx}}} \sim t_{n-2}.

The result is exact when the model assumptions hold. If the errors are not normal, this exact small-sample t result need not hold; a large-sample approximation may sometimes be appropriate, but its validity depends on the setting. A prediction interval also relies on the stated linear mean and constant-variance assumptions for the future response. It does not repair a misspecified model.

Worked Prediction Intervals

Worked Example: Predicting at the Sample Mean

A hypothetical equipment study fits a simple linear regression using $n=5$ units. Its predictor mean is $\bar{x}=4$, $S_{xx}=10$, fitted line $b_0=12$, $b_1=3$, and residual standard error $s=2$. Under the normal-error model, find a 95% prediction interval for one new response at $x_0=4$. Use $t_{0.975,3}=3.182$.

Method. Substitute the fitted response and prediction standard error into the t-based prediction formula. The residual degrees of freedom are $n-2=5-2=3$.

Fitted response. At $x_0=4$, $\hat{Y}_0=b_0+b_1x_0=12+3(4)=24$. Since $x_0=\bar{x}$, the predictor-location term is $(x_0-\bar{x})^2/S_{xx}=0^2/10=0$.

Prediction standard error. The factor under the square root is $1+1/n+0=1+1/5=1.2$. Therefore the standard error is $2\sqrt{1.2}=2.1909$, rounded here for display.

Margin and endpoints. The margin is $3.182(2\sqrt{1.2})=6.97$ to two decimals. The interval is $24\pm6.97$, or $(17.03,30.97)$, with endpoints rounded to two decimals.

Check and trap avoided. The interval is centered on the fitted response $24$, as it must be. Using only the mean-response factor $1/5$ would omit the new response’s error; the required factor here is $1+1/5$.

Worked Example: Accounting for a Predictor Value Away from the Mean

A hypothetical maintenance analysis has $n=6$, $\bar{x}=10$, $S_{xx}=20$, fitted line $b_0=5$, $b_1=2$, and $s=3$. Assuming the model conditions hold, find a 95% prediction interval for one new response at $x_0=12$. Use $t_{0.975,4}=2.776$.

Method. Calculate the fitted response and include both the future-error term and the predictor-location contribution in the standard error. The residual degrees of freedom are $6-2=4$.

Fitted response. $\hat{Y}_0=5+2(12)=29$. The squared distance contribution is $(12-10)^2/20=4/20=0.2$.

Prediction standard error. The factor is $1+1/6+0.2=1+0.1667+0.2=1.3667$, or $41/30$ before rounding. Thus the standard error is $3\sqrt{41/30}=3.51$, rounded to two decimals.

Margin and endpoints. Using the unrounded standard error, the margin is $2.776(3\sqrt{41/30})=9.74$ to two decimals. The 95% prediction interval is $29\pm9.74=(19.26,38.74)$, with endpoints rounded to two decimals.

Check and trap avoided. The result is centered at $29$. The location term is $0.2$, not zero, because $x_0$ differs from $\bar{x}$. Omitting the leading $1$ would calculate uncertainty for an estimated mean response rather than for a new response.

Worked Example: Two Distinct Predictor Values Are Sufficient

A hypothetical calibration sample has predictor values $2,2,5,5$ and responses $7,9,13,15$. Find a 90% prediction interval for one future response at $x_0=3.5$, assuming independent normal errors with constant variance and a linear mean. Use $t_{0.95,2}=2.920$.

Method. Fit the least-squares line, calculate the residual standard error, and then use the prediction formula. This example also checks the rank condition: there are only two distinct predictor values, but the regression can still be fitted.

Fit and degrees of freedom. The predictor mean is $\bar{x}=(2+2+5+5)/4=3.5$, and the response mean is $\bar{Y}=(7+9+13+15)/4=11$. The predictor sum of squares is $S_{xx}=4(1.5^2)=9$. The cross-product sum is $(-1.5)(-4)+(-1.5)(-2)+(1.5)(2)+(1.5)(4)=15$, so $b_1=15/9=5/3$. The intercept is $b_0=11-(5/3)(3.5)=31/6$. The residual degrees of freedom are $4-2=2$.

Residual standard error. At $x=2$, the fitted response is $31/6+(5/3)(2)=10.1667$; at $x=5$, it is $31/6+(5/3)(5)=15.1667$. The residuals are $-19/6,-7/6,-13/6,-1/6$, so $\mathrm{SSE}=(361+49+169+1)/36=580/36=145/9$. Hence $\mathrm{MSE}=(145/9)/2=145/18$ and $s=\sqrt{145/18}=2.84$ to two decimals.

Prediction and standard error. At $x_0=\bar{x}=3.5$, the fitted response is $\hat{Y}_0=11$, and the location term is zero. The prediction standard error is $\sqrt{145/18}\sqrt{1+1/4}=\sqrt{725/72}=3.17$ to two decimals.

Margin and endpoints. Using unrounded values, the margin is $2.920\sqrt{725/72}=9.27$ to two decimals. The 90% prediction interval is $11\pm9.27=(1.73,20.27)$, with endpoints rounded to two decimals.

Check and trap avoided. There are $n=4\geq3$ observations and two distinct predictor values, so the design has rank 2 and the residual degrees of freedom are positive. Three distinct predictor values are not required. Treating the repeated predictor values as a failure of the regression fit would incorrectly reject a valid design.

Common Exam Traps and Interpretation

Using the mean-response standard error. The mean-response factor is $1/n+(x_0-\bar{x})^2/S_{xx}$. For one future response, add $1$ inside the square root.
Using the wrong degrees of freedom. The residual standard error is based on $n-2$ degrees of freedom because two coefficients, the intercept and slope, are estimated.
Requiring three distinct predictor values. The correct requirements are $n\geq3$ and at least two distinct $x_i$. Repeated predictor values are permitted as long as the design has rank 2.
Confusing the target with the center. The interval is centered at $\hat{Y}_0$, but it covers an individual future response, not the unknown mean response.
Claiming exact coverage without the model. The exact t construction depends on the normal, independent, constant-variance error assumptions and on the future error being independent of the fitted sample.

For an actuarial application, the response might represent a future claim-related measurement predicted from a risk characteristic. Such a use is hypothetical: the interval describes uncertainty under the specified regression model, not a guarantee that an individual outcome falls within its endpoints. Model assumptions and the meaning of the response must be considered before interpreting the result.

Check Your Understanding

For each problem, identify the future-response standard error and use the stated critical value where one is supplied. Round interval endpoints to two decimals.

A regression has $n=8$, $\bar{x}=6$, $S_{xx}=28$, $\hat{Y}_0=20$ at $x_0=6$, and $s=1.5$. Find the 95% prediction interval using $t_{0.975,6}=2.447$. (Answer: $(16.11,23.89)$)
A fitted regression has $n=10$, $\bar{x}=5$, $S_{xx}=45$, $b_0=4$, $b_1=1.5$, and $s=2$. Find a 95% prediction interval at $x_0=8$, using $t_{0.975,8}=2.306$. (Answer: $(10.74,21.26)$)
A sample has $n=3$ and predictor values $1,1,3$. Does it have positive residual degrees of freedom and a rank-2 regression design? State both the degrees of freedom and the number of distinct predictor values. (Answer: Yes; 1 residual degree of freedom and 2 distinct predictor values.)
At $x_0=\bar{x}$, a regression has $n=15$ and $s=2$. What is the estimated prediction standard error? (Answer: $2\sqrt{1+1/15}=2.07$ to two decimals.)
A future response is to be predicted at $x_0=9$, with $n=12$, $\bar{x}=7$, $S_{xx}=40$, fitted value $\hat{Y}_0=30$, and $s=2.5$. Calculate the prediction standard error using the supplied quantities. (Answer: $2.5\sqrt{1+1/12+4/40}=2.79$ to two decimals.)

What You'll Learn