Why Prediction Intervals Are Wider Than Mean Intervals

Two Targets at the Same Design Point

At a specified design point, an interval for the mean response answers a different question from an interval for one future observation. The mean response is a fixed, though unknown, quantity. A future outcome varies around that mean, even if the mean were known exactly. A prediction interval must account for both the uncertainty in estimating the mean and this new outcome variation.

The difference is visible in the variance terms: the mean-response variance contains a design-point leverage, while the prediction variance contains that leverage plus one. This tutorial derives the exact intervals and traces how that additional unit changes their widths.

Model and notation: Let $Y=X\beta+\varepsilon$, where $X$ is a fixed $n\times p$ design matrix of full column rank, and $\varepsilon\sim N_n(0,\sigma^2 I_n)$ with $\sigma^2>0$. Write $\hat{\beta}=(X'X)^{-1}X'Y$, $s^2=\mathrm{SSE}/(n-p)$, and $\nu=n-p$. At a fixed design vector $x_0$, the mean response is $x_0'\beta$; a future observation is $Y_0=x_0'\beta+\varepsilon_0$, where $\varepsilon_0\sim N(0,\sigma^2)$ is independent of the observed errors. Define $h_0=x_0'(X'X)^{-1}x_0$.

The quantity $h_0$ is the leverage for estimating the mean at $x_0$. It is nonnegative because $X'X$ is positive definite. For a confidence level $1-\alpha$, let $t_{\nu,1-\alpha/2}$ denote the $1-\alpha/2$ quantile of the $t$ distribution with $\nu$ degrees of freedom. Both intervals below are centered at the fitted mean $x_0'\hat{\beta}$, but their standard-error factors differ.

Why the Prediction Variance Has an Extra One

The least-squares fitted mean at $x_0$ has variance $\sigma^2h_0$. For a future outcome, the prediction error separates into two sources:

Y_0-x_0'\hat{\beta} =\varepsilon_0-x_0'(\hat{\beta}-\beta).

The new error $\varepsilon_0$ has variance $\sigma^2$, and it is independent of the estimation error $x_0'(\hat{\beta}-\beta)$, whose variance is $\sigma^2h_0$. Therefore the prediction-error variance is $\sigma^2(1+h_0)$. The “one” is the variance of one new outcome’s error, expressed in units of $\sigma^2$.

Theorem: Under the stated normal linear model, conditional on the fixed design matrix $X$, exact $1-\alpha$ intervals are $$ x_0'\hat{\beta}\ \pm\ t_{\nu,1-\alpha/2}s\sqrt{h_0} $$ for the mean response $x_0'\beta$, and $$ x_0'\hat{\beta}\ \pm\ t_{\nu,1-\alpha/2}s\sqrt{1+h_0} $$ for one independent future observation $Y_0$.

Proof. Under the model, $\hat{\beta}-\beta=(X'X)^{-1}X'\varepsilon$ is normal with mean zero and covariance matrix $\sigma^2(X'X)^{-1}$. Thus $x_0'(\hat{\beta}-\beta)$ is normal with variance $\sigma^2h_0$. Also, $\mathrm{SSE}/\sigma^2$ has a chi-square distribution with $\nu$ degrees of freedom and is independent of $\hat{\beta}$, by the orthogonal-projection decomposition of a normal error vector into fitted and residual components. Consequently,

\frac{x_0'\hat{\beta}-x_0'\beta}{s\sqrt{h_0}}\sim t_\nu

when $h_0>0$, which gives the stated mean-response interval. If $h_0=0$, the mean is estimated with zero variance at that point and the interval collapses to $x_0'\hat{\beta}=x_0'\beta$ under this model.

For prediction, $\varepsilon_0$ is independent of the observed data, so the numerator $Y_0-x_0'\hat{\beta}$ is normal with mean zero and variance $\sigma^2(1+h_0)$. It is independent of $s^2$: the new error is independent of the data, and the fitted coefficient estimator is independent of the residual sum of squares under normality. Since $\mathrm{SSE}/\sigma^2\sim\chi^2_\nu$, division by $s\sqrt{1+h_0}$ gives a $t_\nu$ variable. The central $t$ quantiles therefore give the stated prediction interval. $\square$

Comparing the Exact Widths

For $s>0$, the half-widths are

H_{\mathrm{mean}}=t_{\nu,1-\alpha/2}s\sqrt{h_0}, \qquad H_{\mathrm{pred}}=t_{\nu,1-\alpha/2}s\sqrt{1+h_0}.

The critical value, estimated error scale, and fitted center are the same. Only the variance factor changes. Since $1+h_0>h_0$, the prediction half-width is strictly larger. More precisely,

H_{\mathrm{pred}}-H_{\mathrm{mean}} =t_{\nu,1-\alpha/2}s\left(\sqrt{1+h_0}-\sqrt{h_0}\right)>0.

This is a comparison of standard errors, not a claim that the prediction interval adds a fixed amount to the mean interval’s half-width. The additional variance is exactly $\sigma^2$, but square roots turn the two variances into widths.

Worked Example: A Design-Point Leverage of 0.2

Take five observations with an intercept and predictor values $-2,-1,0,1,2$. Thus $n=5$, $p=2$, $\nu=3$, and $S_{xx}=10$. Let the model coefficients be $\beta=(10,1)'$, and set $x_0=(1,0)'$, the point with predictor value zero. For this design, $h_0=1/5+0^2/10=0.2$.

To verify a particular residual scale, take residuals $r=\sqrt{1.2}(1,-2,0,2,-1)'$. Their sum is zero, and their inner product with the predictor vector is $-2+2+2-2=0$, so they are orthogonal to both columns of $X$. Also, $r'r=1.2(1+4+0+4+1)=12$. With responses $Y=X\beta+r$, least squares gives $x_0'\hat{\beta}=10$, $\mathrm{SSE}=12$, and $s=\sqrt{12/3}=2$.

At 95% confidence, $t_{3,0.975}\approx3.182446$. The mean-response half-width is $3.182446(2)\sqrt{0.2}\approx2.8465$, giving the interval $(7.1535,12.8465)$. The prediction half-width is $3.182446(2)\sqrt{1.2}\approx6.9724$, giving $(3.0276,16.9724)$, or approximately $(3.028,16.972)$. The larger interval accounts for the error in a new outcome, not just uncertainty about the fitted mean.

Worked Example: Comparing Both Intervals at Leverage 0.6

Keep the same five predictor values, but take $x_0=(1,2)'$, so $h_0=1/5+2^2/10=0.6$. Let $\beta=(10,2)'$, and use residuals $r=\sqrt{2}(1,-2,1,0,0)'$. The residual sum is zero, and its inner product with the predictor vector is $-2+2=0$; hence it is orthogonal to the intercept and predictor columns. Its squared length is $2(1+4+1)=12$. Thus the fitted mean at $x_0$ is $10+2(2)=14$, $\mathrm{SSE}=12$, and again $s=2$ with three degrees of freedom.

Using the same 95% critical value, the mean-response half-width is $3.182446(2)\sqrt{0.6}\approx4.9302$. Its interval is approximately $(9.0698,18.9302)$. For one future observation, the factor is $\sqrt{1+0.6}=\sqrt{1.6}$, so the half-width is $3.182446(2)\sqrt{1.6}\approx8.0510$, and the prediction interval is approximately $(5.9490,22.0510)$. Both intervals use the same center and critical value; the different widths come from $0.6$ versus $1.6$ inside the square root.

Worked Example: The Added Noise Matters Less in Relative Terms at High Leverage

Return to the five-point design above, with $s=2$ and $\nu=3$, and consider the same fitted model with $\beta=(10,0)'$ and residual vector $r=\sqrt{1.2}(1,-2,0,2,-1)'$. At $x_0=(1,\sqrt{40})'$, the fitted mean is 10. The leverage is $h_0=1/5+40/10=4.2$. The mean and prediction half-widths have ratio

\frac{H_{\mathrm{pred}}}{H_{\mathrm{mean}}} =\sqrt{\frac{1+h_0}{h_0}} =\sqrt{\frac{5.2}{4.2}} =\sqrt{\frac{26}{21}} \approx1.1127.

Here the prediction interval is about 11.27% wider than the mean interval. By contrast, at leverage $0.2$ the ratio is $\sqrt{1.2/0.2}=\sqrt{6}$, about 2.45. The unit contribution remains present at either point, but it is a smaller share of total variance when leverage is already large.

Common Mistake and Practical Meaning

A common mistake is to use $s\sqrt{h_0}$ for a future observation because that is the standard error used for the mean response. This omits the independent error of the new outcome. The correction is to add its variance, not its standard deviation: the future-outcome variance is $\sigma^2h_0+\sigma^2$, and its estimated standard error is $s\sqrt{h_0+1}$. In general, $s\sqrt{h_0}+s$ is not the correct prediction standard error.

The distinction matters whenever the inferential target is an individual outcome rather than an average response. Even with a very large sample and a precisely estimated mean, individual outcomes can still vary around that mean. Conversely, the intervals are not interchangeable: a prediction interval is designed to cover one future outcome, while a mean-response interval is designed to cover the fixed mean at the specified point.

Key takeaway: At the same design point, both exact intervals share the fitted mean as center and the same $t$ critical value. The prediction interval is wider because its variance factor is $1+h_0$, rather than $h_0$: the added one is the variance contribution of new outcome noise.

Check Your Understanding

Use the formulas and assumptions above to answer the following.

At a design point with leverage $h_0=0.4$, what variance factors appear in the mean-response and prediction standard errors?
Why is the new outcome error independent of the uncertainty in the fitted mean under the stated model?
If $h_0$ increases while the other quantities remain fixed, what happens to the ratio of prediction half-width to mean-response half-width?
Explain why the prediction standard error is not $s\sqrt{h_0}+s$.
Which interval targets a fixed quantity: the mean-response interval or the prediction interval? What does the other interval target?

Why Prediction Intervals Are Wider Than Mean Intervals

What You'll Learn