Intervals at Specific Predictor Settings

Why the Predictor Setting Matters

A regression model can be used to estimate an average response at a chosen predictor setting, or to predict one future response there. These are different targets: an average is less variable than an individual observation. In either case, uncertainty depends on the predictor setting. A setting near the center of the observed predictors generally gives a more precise estimate of the mean than a setting far from that center.

The formulas below apply to a linear regression model with a full-rank design matrix, independent errors with mean zero and common variance, and normally distributed errors when exact $t$-based intervals are required. A prediction interval also assumes that the future response is generated by the same model and has an error independent of the data used to fit it. If these conditions do not hold, the stated intervals need not have their advertised coverage.

Definition: Let $x_0$ denote the predictor vector for a setting of interest, including the intercept component. The leverage quantity at that setting is $h_0=x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0$, where $X$ is the regression design matrix. It measures how much the estimated mean at that setting varies because the regression coefficients were estimated from a finite sample.

For a model with $n$ observations and $p$ estimated coefficients, let $b$ be the least-squares coefficient vector, let $s^2=\mathrm{SSE}/(n-p)$ be the residual mean square, and let $\nu=n-p$ be the residual degrees of freedom. The fitted mean at $x_0$ is $x_0^\mathsf{T}b$. The two intervals use the same fitted value and the same $t$ critical value, but different standard errors.

Theorem: Under the linear model assumptions, a two-sided $100(1-\alpha)\%$ confidence interval for the mean response at $x_0$ is $$ x_0^\mathsf{T}b \ \pm\ t_{1-\alpha/2,\nu}s\sqrt{h_0}. $$ A two-sided $100(1-\alpha)\%$ prediction interval for one new response at that setting is $$ x_0^\mathsf{T}b \ \pm\ t_{1-\alpha/2,\nu}s\sqrt{1+h_0}. $$

Where the Width Formulas Come From

The distinction between the two standard errors follows from the variance of the fitted mean and the extra error in a new observation. Under the model, the coefficient estimator has covariance matrix $\operatorname{Cov}(b)=\sigma^2(X^\mathsf{T}X)^{-1}$. Applying the variance formula for a linear combination gives

\operatorname{Var}(x_0^\mathsf{T}b) =x_0^\mathsf{T}\operatorname{Cov}(b)x_0 =\sigma^2 x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0 =\sigma^2 h_0.

Thus the standard error for estimating the mean response is estimated by $s\sqrt{h_0}$. For a new response $Y_{\mathrm{new}}$, write $Y_{\mathrm{new}}=x_0^\mathsf{T}\beta+\varepsilon_{\mathrm{new}}$, where $\varepsilon_{\mathrm{new}}$ has variance $\sigma^2$ and is independent of the fitted data. The prediction error is the sum of the new observation's error and the error in estimating the mean:

Y_{\mathrm{new}}-x_0^\mathsf{T}b =\varepsilon_{\mathrm{new}}+x_0^\mathsf{T}(\beta-b), \qquad \operatorname{Var}(Y_{\mathrm{new}}-x_0^\mathsf{T}b) =\sigma^2+\sigma^2h_0 =\sigma^2(1+h_0).

Replacing $\sigma$ by $s$ gives the prediction standard error $s\sqrt{1+h_0}$. Under normal errors, each standardized interval statistic has a $t$ distribution with $\nu=n-p$ degrees of freedom, which justifies using the same $t$ critical value in both intervals. The additional $1$ is essential: it accounts for the random variation of the future individual response, not just uncertainty in the fitted mean.

Distance from the Center in Simple Linear Regression

In simple linear regression with an intercept and one predictor $x$, the leverage quantity at $x_0$ has a particularly useful form. Let $\bar{x}$ be the sample mean of the predictor values and $S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2$. Then

h_0=\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}.

The first term is the contribution from estimating the intercept. The second is the contribution from estimating the slope. It is nonnegative and increases with the squared distance from $\bar{x}$, so the mean-response interval is narrowest at $x_0=\bar{x}$. The prediction interval also is narrowest there, but it remains wider because its standard error includes the additional $1$. In multiple regression, “far from the center” is captured by $h_0$, which depends on the design and the joint predictor setting; ordinary distance in one predictor alone may not describe it.

Worked Example: Comparing Mean Intervals at Two Settings

A hypothetical portfolio model uses a simple linear regression to relate an exposure measure $x$ to an annual response $Y$, in thousands of dollars. Assume independent normal errors with constant variance. The fitted line is $b_0=20$, $b_1=4$, with $n=12$, $\bar{x}=4$, $S_{xx}=12$, and $s=3$. Find 95% confidence intervals for the mean response at $x_0=4$ and $x_0=7$. Use $t_{0.975,10}=2.228$.

Method. There are two estimated coefficients, so $\nu=12-2=10$. For each setting, calculate $h_0$, the fitted mean $20+4x_0$, and the margin $2.228(3\sqrt{h_0})$.

At $x_0=4$, the predictor is at its sample mean: $$ h_0=\frac{1}{12}+\frac{(4-4)^2}{12}=\frac{1}{12}=0.0833333. $$ The fitted mean is $20+4(4)=36$. The standard error is $3\sqrt{1/12}=0.8660254$, and the margin is $2.228(0.8660254)=1.9295046$. Therefore the interval is $[36-1.9295046,\ 36+1.9295046]=[34.0705,\ 37.9295]$ thousand dollars.

At $x_0=7$, the predictor is three units from its sample mean: $$ h_0=\frac{1}{12}+\frac{(7-4)^2}{12} =\frac{10}{12}=\frac{5}{6}=0.8333333. $$ The fitted mean is $20+4(7)=48$. The standard error is $3\sqrt{5/6}=2.7386128$, and the margin is $2.228(2.7386128)=6.1016294$. The interval is $[48-6.1016294,\ 48+6.1016294]=[41.8984,\ 54.1016]$ thousand dollars.

Check and common mistake. The second interval is wider because $h_0=5/6$ exceeds $1/12$. The fitted means also differ, so width must be compared using the margins, not by comparing the interval endpoints. A common error is to use only $1/n$ at both settings; that would omit the slope-estimation term and incorrectly give equal widths.

Prediction at a Specified Setting

A prediction interval is for an individual future response, not for the average of all responses at that setting. Its center is still the fitted mean, but its standard error is $s\sqrt{1+h_0}$. Consequently, even when the fitted mean is estimated very precisely, the interval for a new observation does not collapse to zero width: the new observation itself varies around the mean.

Worked Example: Prediction Interval with a Given Leverage

A hypothetical multiple regression analysis has $n=18$ observations and $p=4$ fitted coefficients. At a specified predictor setting, the fitted mean is 36 response units, the leverage is $h_0=0.4625$, and the residual standard error is $s=4$ units. Assume the model conditions for exact $t$-intervals hold. Construct a 95% prediction interval for one new response, using $t_{0.975,14}=2.145$, and compare it with the confidence interval for the mean response at the same setting.

Method. The residual degrees of freedom are $18-4=14$. First calculate the mean-response and prediction standard errors separately; the prediction calculation must include $1+h_0$.

For the mean response, the standard error is $$ 4\sqrt{0.4625}=2.7202941. $$ The margin is $2.145(2.7202941)=5.8350309$, so the confidence interval for the mean is $[36-5.8350309,\ 36+5.8350309]=[30.1650,\ 41.8350]$.

For one new response, the standard error is $$ 4\sqrt{1+0.4625}=4\sqrt{1.4625}=4.8373546. $$ The margin is $2.145(4.8373546)=10.3761258$. Thus the prediction interval is $[36-10.3761258,\ 36+10.3761258]=[25.6239,\ 46.3761]$, or $[25.62,\ 46.38]$ to two decimal places.

Check and common mistake. The prediction interval is wider than the mean-response interval because its variance includes the future observation's error. Omitting the $1$ would incorrectly produce the mean-response standard error and interval. Using the unrounded prediction standard error also matters: rounding $4.8373546$ too early can shift the reported endpoints.

Leverage with More Than One Predictor

In multiple regression, a predictor setting is represented by a vector. The leverage formula combines all its components and their relationships in the design matrix. This matters when predictors have different scales or are correlated: settings that look close by eye can have quite different leverage, and settings that look numerically far apart can have modest leverage if they follow the design's usual pattern.

Worked Example: A Mean Interval from a Multiple Regression Design

A hypothetical regression has an intercept and two predictors, $z_1$ and $z_2$, with $n=25$. Suppose the predictor columns are centered and orthogonal, and the given inverse cross-product matrix is $$ (X^\mathsf{T}X)^{-1}= \begin{pmatrix} 0.04&0&0\\ 0&0.05&0\\ 0&0&0.05 \end{pmatrix}. $$ At $(z_1,z_2)=(2,-1)$, the fitted mean is 100 units and $s=5$. Construct a 95% confidence interval for the mean response, using $t_{0.975,22}=2.074$.

Method. The predictor vector, including its intercept component, is $x_0=(1,2,-1)^\mathsf{T}$. There are $25-3=22$ residual degrees of freedom. The leverage is $$ h_0=x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0 =(1)(0.04)(1)+(2)(0.05)(2)+(-1)(0.05)(-1) =0.04+0.20+0.05=0.29. $$

The standard error of the estimated mean is $5\sqrt{0.29}=2.6925824$. The margin is $2.074(2.6925824)=5.5844159$. The interval is $[100-5.5844159,\ 100+5.5844159]=[94.4156,\ 105.5844]$ units.

Check and common mistake. The leverage is positive and includes contributions from both predictor coordinates as well as the intercept. Treating the setting as if it were a single predictor value, or leaving out the intercept coordinate, would not compute $x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0$ and could give the wrong interval width.

Interpreting Width and Using the Intervals Carefully

For fixed $s$, degrees of freedom, and confidence level, the mean-response interval half-width is proportional to $\sqrt{h_0}$, while the prediction interval half-width is proportional to $\sqrt{1+h_0}$. A larger leverage therefore widens both intervals. In simple regression, moving away from $\bar{x}$ increases leverage; in multiple regression, use the full design-based quantity.

A setting beyond the range or combinations represented in the observed data may have high leverage. The formulas can still be evaluated algebraically if the design matrix has full rank, but a narrow-looking numerical result does not make unsupported extrapolation reliable. The linear relationship and constant variance assumptions may be less credible outside the observed predictor region.

A confidence interval for a mean response describes uncertainty in the model's average response at a fixed setting.
A prediction interval describes the range of uncertainty for one new response and includes individual-level error.
The fitted value is the center of both intervals; changing $x_0$ can change both the center and the width.
Use $n-p$ residual degrees of freedom and the corresponding $t$ critical value when the usual exact normal-error assumptions apply.

These distinctions are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus: identify the target, calculate leverage at the stated setting, select the appropriate standard error, and interpret the interval in context.

Check Your Understanding

For each problem, identify whether the target is a mean or an individual response, and use the stated setting and interval type.

A simple regression has $n=16$, $\bar{x}=5$, $S_{xx}=20$, fitted line $12+3x$, and $s=2$. Find the 95% confidence interval for the mean response at $x_0=5$, using $t_{0.975,14}=2.145$. (Answer: $[25.93,\ 28.07]$.)
Using the same model and critical value, find the 95% confidence interval for the mean response at $x_0=9$. (Answer: approximately $[35.02,\ 42.98]$.)
Using that model, find a 95% prediction interval for one new response at $x_0=5$. (Answer: approximately $[22.58,\ 31.42]$.)
A regression has $n=20$, $p=3$, $s=6$, fitted mean 50 at a setting with $h_0=0.25$, and $t_{0.975,17}=2.110$. Give both the 95% mean-response interval and the 95% prediction interval. (Answer: mean-response interval $[43.67,\ 56.33]$; prediction interval approximately $[35.85,\ 64.15]$.)
In a centered, orthogonal two-predictor model with $(X^\mathsf{T}X)^{-1}=\operatorname{diag}(0.05,0.01,0.04)$, consider $x_0=(1,2,1)^\mathsf{T}$. If $n=20$, the fitted mean is 80, $s=4$, and $t_{0.975,17}=2.110$, calculate the 95% confidence interval for the mean response. (Answer: $h_0=0.05+4(0.01)+0.04=0.13$; interval approximately $[76.96,\ 83.04]$.)

What You'll Learn