Common Errors in Interpreting Prediction Intervals

Why Interpretation Matters

A regression prediction interval is intended to describe plausible values for a new individual outcome at a specified predictor value. Its interpretation is easy to blur with that of an interval for the mean response. Those intervals answer different questions, and confusing them can make a prediction sound much more precise than it is.

Two other errors are common: treating the interval’s width as if it reflected only uncertainty in the fitted regression coefficients, and reading a nominal coverage level as a promise that the interval will contain the next outcome. Interpreting the interval correctly requires keeping its target, sources of uncertainty, and model assumptions in view.

The Prediction-Interval Target

For a simple linear regression, write the model for an outcome at predictor value \(x_i\) as \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). Here \(\beta_0+\beta_1x_i\) is the mean response at \(x_i\), and \(\varepsilon_i\) is the individual’s deviation from that mean. For the usual exact prediction interval, assume the mean relationship is linear, the errors are independent and normally distributed with mean zero and common variance \(\sigma^2\), and the predictor values are treated as fixed. A future outcome must come from the same process represented by the model.

Let \(x_0\) be the predictor value for a new case, and let \(\hat{y}_0\) be the fitted mean response there. In simple linear regression, define \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\) and \(h_0=1/n+(x_0-\bar{x})^2/S_{xx}\). The usual two-sided \(100(1-\alpha)\%\) prediction interval is:

\hat{y}_0 \pm t_{1-\alpha/2,\,n-2}\,s\sqrt{1+h_0}

In this formula, \(s\) estimates the residual standard deviation, and \(t_{1-\alpha/2,\,n-2}\) is the relevant critical value from the \(t\) distribution with \(n-2\) degrees of freedom. The interval’s center is the predicted mean; its target, however, is a single new outcome \(Y_{\text{new}}\) at \(x_0\). The interval’s units are the units of the response.

Definition: A prediction interval for a new outcome at \(x_0\) is an interval calculated from the fitted regression and data, intended to cover \(Y_{\text{new}}\) at that predictor value at a stated repeated-sampling rate, provided the model assumptions hold. It is not an interval for the unknown mean response alone.

Why the Interval Is Wider Than a Mean-Response Interval

The distinction between an individual outcome and a mean response can be seen in the uncertainty terms. The fitted mean \(\hat{y}_0\) varies across training samples; its estimated standard error is \(s\sqrt{h_0}\). But even if that mean were known exactly, a new individual outcome would still vary around it because of the new error \(\varepsilon_{\text{new}}\).

Under the stated assumptions, the estimation error in \(\hat{y}_0\) and the new error are independent. The difference between the new outcome and the fitted mean can be written as:

Y_{\text{new}}-\hat{y}_0 = (\beta_0+\beta_1x_0-\hat{y}_0)+\varepsilon_{\text{new}}

The first term represents estimation error in the fitted mean, with variance \(\sigma^2h_0\); the second has variance \(\sigma^2\). Independence means the variances add:

\operatorname{Var}(Y_{\text{new}}-\hat{y}_0) = \sigma^2h_0+\sigma^2 = \sigma^2(1+h_0)

Replacing \(\sigma\) with \(s\), and using the \(t\) critical value because the variance is estimated, gives the prediction-interval margin in the formula above. By contrast, a confidence interval for the mean response uses \(s\sqrt{h_0}\), without the extra \(1\), because it targets the mean and not a new individual deviation. This is the key reason a prediction interval is wider.

Worked Example: Individual Outcome or Mean Response?

Suppose a hypothetical regression of a device’s operating time on a measured input is based on \(n=20\) devices. At the average input, the fitted mean operating time is \(50\) hours, the residual standard deviation is \(s=4\) hours, and \(h_0=1/n=0.05\). Assume the linearity, independence, constant-variance, and normal-error conditions are reasonable for this hypothetical setting, and that a new device follows the same process. For a 95% interval, \(t_{0.975,18}=2.101\).

For one new device, the prediction-interval margin is:

2.101(4)\sqrt{1+0.05} = 8.404\sqrt{1.05} \approx 8.61\text{ hours}

Thus the 95% prediction interval is \(50\pm8.61\), or approximately \(41.39\) to \(58.61\) hours. In context, this procedure is intended to produce intervals covering the operating time of a new device at the average input at the stated rate over repeated use under the model assumptions.

The mean-response interval at the same input has margin:

2.101(4)\sqrt{0.05} \approx 1.88\text{ hours}

That interval, approximately \(48.12\) to \(51.88\) hours, concerns the mean operating time for devices at that input. It does not aim to contain the operating time of every individual device. Reporting it as a range for one new device would confuse the two targets.

What Interval Width Does—and Does Not—Measure

The margin \(t^*s\sqrt{1+h_0}\) reflects more than uncertainty in the estimated regression coefficients. The factor \(s\) reflects residual scatter around the fitted mean; the \(1\) accounts for the variation of an individual new outcome around its mean; \(h_0\) reflects uncertainty in estimating the mean at \(x_0\); and \(t^*\) depends on the confidence level and degrees of freedom. Therefore, an interval can be wide even when the fitted mean is estimated quite precisely.

The value of \(h_0\) also depends on predictor location. In simple linear regression, it is smallest at \(\bar{x}\) and increases as \(x_0\) moves away from \(\bar{x}\). That changes the mean-estimation contribution to the prediction interval, but it does not remove the individual-outcome variation represented by the \(1\). A wide interval is not, by itself, evidence that the slope or intercept is especially uncertain.

Worked Example: Separating the Sources of Width

Continue with a hypothetical fit having \(n=20\), \(s=4\) hours, \(df=18\), and \(t^*=2.101\). Suppose the residual checks and study context support the stated regression assumptions, and the new devices are from the same process. Compare a prediction at the average input, where \(h_0=0.05\), with one farther from the average input, where \(h_0=0.45\). Assume the fitted means are both \(50\) hours to keep the comparison focused on width.

At the average input, the prediction standard error is \(4\sqrt{1.05}\), so the margin is:

2.101(4)\sqrt{1.05}\approx8.61\text{ hours}

Farther from the average input, it is \(4\sqrt{1.45}\), giving:

2.101(4)\sqrt{1.45}\approx10.12\text{ hours}

The latter interval is wider because estimating the mean at that predictor location is less precise. But the coefficient-uncertainty contribution is not the whole margin. At the average input, its standard-error component is \(s\sqrt{h_0}=4\sqrt{0.05}\approx0.894\) hours, while the individual outcome also has residual variation of about \(4\) hours. Farther from the average, the mean-estimation component grows to \(4\sqrt{0.45}\approx2.68\) hours, while the residual component remains \(4\) hours. These contributions combine through the square root of the sum of variances; they are not added as separate margins.

The interval width therefore reflects both new-outcome scatter and uncertainty in the fitted mean, as well as the chosen confidence level and degrees of freedom. Saying that the far-from-average interval is wider “because the coefficients are uncertain” is incomplete: it leaves out the substantial outcome variation that remains even if the mean were known.

Nominal Coverage Is Not a Guarantee for One Case

A 95% prediction interval does not promise to contain the next outcome. Its nominal level describes a repeated-sampling property: if the model assumptions hold and the process is repeated with new training data and a new outcome at the specified fixed predictor value, the interval procedure will cover that outcome in about 95% of repetitions. In the normal-error simple linear model, the usual \(t\)-based interval has this exact coverage under the stated assumptions.

That statement is conditional on the model being appropriate. If the mean relationship is misspecified, errors are not independent, the error variance changes in a way the model does not account for, or the new case comes from a different process, the actual coverage may differ from the nominal level. A nominal percentage is not a guarantee for any particular person, device, or other case, and it is not a probability assigned to the fixed outcome after the interval has been calculated.

Worked Example: An Outcome Outside a 95% Interval

Suppose a hypothetical regression fit meets the assumptions described above, and its 95% prediction interval for a new device at a specified input is \(41.39\) to \(58.61\) hours, centered at \(50\) hours. A new device from the same process has an operating time of \(61\) hours. The observed value is outside the interval because \(61>58.61\).

This outcome does not contradict the meaning of a 95% prediction interval. The interval procedure is designed for about 95% coverage over repetitions under the model; it allows some outcomes to fall outside. It would be incorrect to say the interval guaranteed this device’s operating time would be between \(41.39\) and \(58.61\) hours. It would also be incorrect to use this single result alone to conclude that the procedure’s long-run coverage is exactly 95% or that it is not 95%. One case either falls inside or outside; coverage is a property assessed over repeated cases under the model conditions.

Common Mistakes and Better Wording

Clear interpretation starts by naming what is being predicted and the conditions under which the interval is intended to work. Avoid statements that blur an individual outcome with a mean or turn long-run coverage into a case-specific promise.

Mistake: “The interval gives the range for the mean.” Correction: a prediction interval targets one new outcome. Use a confidence interval for the mean response when the target is the mean.
Mistake: “The width measures only uncertainty in the regression coefficients.” Correction: prediction-interval width includes individual residual variation as well as uncertainty in the fitted mean, and it depends on the confidence level and predictor location.
Mistake: “There is a 95% chance this particular outcome will be in the calculated interval.” Correction: under the assumptions, 95% describes the interval procedure’s coverage across repeated samples and new outcomes at the specified predictor value. It does not guarantee coverage for this case.
Mistake: “An outcome outside the interval proves the model is wrong.” Correction: occasional misses are expected even when a procedure has its stated coverage. A pattern of misses or evidence that assumptions fail may warrant closer investigation; one miss alone does not establish that conclusion.

Key takeaway: Interpret a prediction interval as a range intended for a new individual outcome, not the mean response. Its width includes individual-outcome variation and fitted-mean uncertainty, and its nominal coverage is a repeated-sampling statement under model assumptions—not a guarantee for a particular case.

Check Your Understanding

Answer each question by identifying the target and the relevant uncertainty or coverage statement.

A 95% prediction interval for a new patient’s measurement is reported as 12 to 20 units. What outcome does the interval target, and why is it not an interval for the mean measurement?
Why does the usual prediction-interval formula contain \(1+h_0\), while a mean-response interval contains \(h_0\)?
Two prediction intervals use the same residual standard deviation and confidence level, but the second is farther from \(\bar{x}\). Give a reason its width might differ.
What does nominal 95% coverage mean across repetitions under the model assumptions?
A new outcome falls outside a 95% prediction interval. Explain why this does not, by itself, contradict the interval’s stated coverage.