Calculating Prediction Intervals

From a Fitted Value to a Prediction Interval

A fitted regression line gives a point prediction at a chosen predictor value. But a future observation can differ from that fitted value for two reasons: the line itself is estimated from a sample, and an individual response varies around the true regression mean. A prediction interval accounts for both sources of uncertainty.

This tutorial constructs a two-sided prediction interval for one future response in simple linear regression. The calculations use a fitted intercept and slope, a residual standard error, the sample size, and the predictor value. They also require information about where that predictor value sits relative to the observed predictors. That information may be given as the sample mean and sum of squares of the predictors, or included in a model-output prediction standard error.

Model and assumptions: For observed data $(x_i,Y_i)$, $i=1,\ldots,n$, assume the simple linear regression model $$ Y_i=\beta_0+\beta_1x_i+\varepsilon_i, $$ where the predictor values $x_i$ are fixed, and the errors are independent with $\varepsilon_i\sim N(0,\sigma^2)$. A future response at a specified predictor value $x_0$ follows the same model, with an error independent of the data used to fit the line. The usual exact t prediction interval relies on these assumptions, including constant error variance and normal errors.

Write $b_0$ and $b_1$ for the fitted intercept and slope, and let $\hat y_0=b_0+b_1x_0$ be the fitted value at $x_0$. Define $\bar x=\sum_{i=1}^n x_i/n$, $S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2$, and $s$ as the residual standard error. For simple linear regression, the residual degrees of freedom are $n-2$.

Theorem: Under the stated model, a two-sided $100(1-\alpha)\%$ prediction interval for one future response at $x_0$ is $$ \hat y_0\ \pm\ t_{1-\alpha/2,\,n-2}\, s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar x)^2}{S_{xx}}}. $$ Here $t_{1-\alpha/2,\,n-2}$ is the $1-\alpha/2$ quantile of the t distribution with $n-2$ degrees of freedom. The square-root factor is the prediction standard error divided by $s$.

The Calculation in Four Steps

The interval is easiest to calculate in a fixed order. Keep the fitted value, the prediction standard error, the critical value, and the final endpoints distinct; this makes it easier to spot arithmetic and input errors.

Find the fitted response.
Substitute the specified $x_0$ into $\hat y_0=b_0+b_1x_0$.

Calculate the prediction standard error.
Evaluate $s\sqrt{1+1/n+(x_0-\bar x)^2/S_{xx}}$, using the supplied predictor summaries or an equivalent prediction standard error from the model output.

Choose the t critical value.
For a two-sided confidence level $1-\alpha$, use $t_{1-\alpha/2,n-2}$, not a t quantile with $n$ degrees of freedom.

Calculate both endpoints.
Multiply the prediction standard error by the critical value, then subtract the margin from and add it to $\hat y_0$.

A list of $b_0$, $b_1$, $s$, $n$, and $x_0$ alone is generally not enough to calculate the interval: the factor involving $x_0$, $\bar x$, and $S_{xx}$ is not determined by those five values. If a problem omits $\bar x$ and $S_{xx}$, check whether it provides the prediction standard error or the leverage factor in another form. If it does not, the numerical interval cannot be uniquely found.

Worked Example: Predicting a Future Claim-Processing Time

A hypothetical insurer models processing time $Y$, in days, against a claim-complexity score $x$. Assume the normal-error regression model above. A fitted model has $b_0=12$, $b_1=3$, and residual standard error $s=2$ days. The training sample has $n=11$, $\bar x=3$, and $S_{xx}=20$. Calculate a 95% prediction interval for one future claim with $x_0=4$. Use $t_{0.975,9}=2.262$.

Method: fitted value plus the t prediction margin. First, the fitted processing time is $$ \hat y_0=12+3(4)=24\text{ days}. $$

Next, the factor under the square root is $$ 1+\frac{1}{11}+\frac{(4-3)^2}{20} =1+0.090909+0.05 =1.140909. $$ Its square root is approximately $1.0681$, so the prediction standard error is $$ 2(1.0681)=2.1362\text{ days}. $$

There are $11-2=9$ residual degrees of freedom. The 95% margin of error is $$ 2.262(2.1362)=4.8322\text{ days}. $$ Therefore, the endpoints are $24-4.8322=19.1678$ and $24+4.8322=28.8322$. Rounding the endpoints to two decimal places gives the interval $(19.17,28.83)$ days.

Check and mistake avoided. The endpoints are equally far from the fitted value 24. Using only $s=2$ as the prediction standard error would omit both the fitted-mean estimation term and the future-response variation; the correct calculation uses the entire square-root factor.

Why the Formula Has This Form

The prediction interval is based on how far a future response can be from the fitted value. Let the true mean at $x_0$ be $\beta_0+\beta_1x_0$, and define the fitted-mean estimation error by $$ \bar\varepsilon_{\mathrm{fit}}=\hat y_0-(\beta_0+\beta_1x_0). $$ If the new response is $Y_{\mathrm{new}}=\beta_0+\beta_1x_0+\varepsilon_{\mathrm{new}}$, then subtracting the fitted value gives $$ Y_{\mathrm{new}}-\hat y_0 =\varepsilon_{\mathrm{new}}-\bar\varepsilon_{\mathrm{fit}}. $$ The minus sign follows directly from the stated definition of $\bar\varepsilon_{\mathrm{fit}}$.

The new error is independent of the data used to fit the line, so it is independent of the fitted-mean estimation error. Under the model, their variances are $\sigma^2$ and $\sigma^2[1/n+(x_0-\bar x)^2/S_{xx}]$, respectively. Therefore, $$ \operatorname{Var}(Y_{\mathrm{new}}-\hat y_0) =\sigma^2\left(1+\frac{1}{n}+\frac{(x_0-\bar x)^2}{S_{xx}}\right). $$ The leading $1$ comes from the new response’s own error; the other terms describe uncertainty in the estimated mean. Replacing $\sigma$ by the residual standard error $s$, and using the t distribution with $n-2$ degrees of freedom, yields the stated interval. Under the normal-error assumptions, this studentized prediction error has that t distribution.

Worked Example: Prediction at the Sample Mean Score

A hypothetical insurer relates a policy’s annual service-cost score $Y$, in hundreds of dollars, to a risk index $x$. Assume independent normal errors with constant variance and a future policy independent of the fitted sample. The fitted coefficients are $b_0=8$ and $b_1=0.8$; the residual standard error is $s=1.5$. The model used $n=16$ policies, with $\bar x=5$ and $S_{xx}=40$. Calculate a 95% prediction interval at $x_0=5$, using $t_{0.975,14}=2.145$.

Method: evaluate at the predictor mean, then apply the prediction standard error. The fitted value is $$ \hat y_0=8+0.8(5)=12 $$ hundreds of dollars. Since $x_0=\bar x$, the squared-displacement term is zero. The prediction standard error is $$ 1.5\sqrt{1+\frac{1}{16}+0} =1.5\sqrt{1.0625} =1.5(1.0308) =1.5462. $$

The degrees of freedom are $16-2=14$. The margin is $$ 2.145(1.5462)=3.3165. $$ Thus, the interval is $12\pm3.3165$, or $(8.6835,15.3165)$ hundreds of dollars. To two decimal places, this is $(8.68,15.32)$ hundreds of dollars.

Check and mistake avoided. The interval is centered at 12, the fitted value at the requested score. Being at $\bar x$ makes the predictor-displacement term zero, but it does not remove the $1$ for future-response variation or the $1/n$ term for estimating the mean.

Using Model Output and Interpreting the Result

Some regression output provides the prediction standard error directly rather than listing $\bar x$ and $S_{xx}$. If that value is explicitly for a single future response at $x_0$, multiply it by the appropriate t critical value and add and subtract the result from $\hat y_0$. Do not substitute a standard error labeled for the estimated mean response: it does not include the full future-response uncertainty.

The resulting interval is intended to cover the one future response at the specified predictor value in repeated samples, under the model assumptions. It is not a statement that the future response has a specified probability of being inside an already calculated interval once the data and interval are fixed. The confidence level describes the long-run coverage of the interval procedure.

Check the response scale before interpreting endpoints. If $Y$ is measured in hundreds of dollars, both the fitted value and endpoints are in hundreds of dollars; convert to dollars only after calculating the interval. If $Y$ is a log-transformed response, the interval produced by this formula is on the log scale. Transforming its endpoints requires separate care and does not make the original-scale interval symmetric.

Worked Example: Predicting a Future Repair Duration

A hypothetical maintenance provider models repair duration $Y$, in hours, using a machine-age index $x$. Assume independent normal errors with constant variance, and assume the future repair is independent of the fitted data. The fitted line is $\hat y=50-2x$, the residual standard error is $s=4$ hours, $n=9$, $\bar x=10$, and $S_{xx}=18$. Find a 90% prediction interval at $x_0=13$, using $t_{0.95,7}=1.895$.

Method: calculate the fitted value and the full prediction margin. The fitted duration is $$ \hat y_0=50-2(13)=24\text{ hours}. $$ The factor under the square root is $$ 1+\frac{1}{9}+\frac{(13-10)^2}{18} =1+0.111111+0.5 =1.611111. $$ Thus, the prediction standard error is $$ 4\sqrt{1.611111}=4(1.2693)=5.0772\text{ hours}. $$

The residual degrees of freedom are $9-2=7$. The 90% margin is $$ 1.895(5.0772)=9.6213\text{ hours}. $$ The endpoints are $24-9.6213=14.3787$ and $24+9.6213=33.6213$. Rounded to two decimal places, the prediction interval is $(14.38,33.62)$ hours.

Check and mistake avoided. The negative slope is used when calculating the fitted value, giving 24 hours. The interval is nevertheless formed by adding and subtracting a positive margin. Using $50+2(13)$ would incorrectly reverse the slope’s sign.

Common Calculation Traps

Using the wrong critical value. For a two-sided 95% interval, use the 0.975 t quantile, not the 0.95 quantile. Use $n-2$ degrees of freedom for simple linear regression.
Omitting the leading 1. The formula for one future response includes $1+1/n+(x_0-\bar x)^2/S_{xx}$. Omitting the leading 1 leaves out the new observation’s error.
Using the mean-response standard error. The mean-response factor lacks the leading 1. It is not the standard error for predicting one future observation.
Assuming the given quantities always suffice. Coefficients, $s$, $n$, and $x_0$ do not determine the predictor-displacement term. Look for predictor summaries or a prediction standard error.
Rounding too early. Keep several digits in the square root and margin, then round the final endpoints to the requested precision.

Key takeaway: For a two-sided prediction interval for one future response, calculate the fitted value, multiply the residual standard error by the square root of $1+1/n+(x_0-\bar x)^2/S_{xx}$, and then use the t critical value with $n-2$ degrees of freedom. The model assumptions and the required predictor information matter as much as the arithmetic.

Check Your Understanding

For each question, use the stated normal-error regression assumptions and round final endpoints to two decimal places. Answers are included for checking.

A hypothetical insurer fits a claim-cost model with $b_0=5$, $b_1=2$, $s=1$, $n=12$, $\bar x=2$, and $S_{xx}=10$. Find a 95% prediction interval at $x_0=3$, using $t_{0.975,10}=2.228$. (Answer: fitted value $11$; interval $(8.58,13.42)$.)
A hypothetical service-cost model has fitted value $40$ at $x_0=\bar x$, $s=3$, and $n=21$. Find a 95% prediction interval using $t_{0.975,19}=2.093$. (Answer: prediction standard error $3\sqrt{22/21}\approx3.0706$; interval $(33.57,46.43)$.)
A hypothetical repair-time model has $b_0=20$, $b_1=1.5$, $s=2$, $n=8$, $\bar x=4$, and $S_{xx}=8$. Find a 90% prediction interval at $x_0=6$, using $t_{0.95,6}=1.943$. (Answer: fitted value $29$; interval $(24.05,33.95)$.)
A fitted simple linear regression provides $b_0=7$, $b_1=1$, $s=2$, $n=10$, and $x_0=4$, but gives neither $\bar x$, $S_{xx}$, nor a prediction standard error. Can a unique numerical prediction interval be calculated from the information given? (Answer: No; the predictor-displacement term, or an equivalent prediction standard error, is missing.)

What You'll Learn