Worked Problems in Mean Response and Individual Prediction

Two Questions at a Predictor Value

A fitted regression line gives a predicted response at any chosen predictor value, but that fitted value is not itself a statement of certainty. At a specified value \(x_0\), we may want to estimate the mean response for all individuals with that predictor value. Or we may want to predict the response of one particular individual. These are different questions, so they require different intervals.

The distinction matters in practice. A clinic might want the average change in a health measure among patients receiving a specified dose, or it might want to predict the change for one patient. The first concerns a group mean; the second concerns an individual outcome, which includes both uncertainty about the fitted regression line and the natural variation among individuals.

Definition: In simple linear regression, the mean response at \(x_0\) is the population average response for individuals whose predictor equals \(x_0\). A prediction interval at \(x_0\) estimates a range of plausible values for the response of one new individual with predictor \(x_0\). A confidence interval estimates the mean response; it does not estimate the range containing most individual responses.

Model and Interval Formulas

The formulas below use the simple linear regression model \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). For the usual exact \(t\)-based intervals, assume the mean response is linear in \(x\), errors are independent with mean zero and constant variance \(\sigma^2\), and errors at each predictor value are normally distributed. The predictor values are treated as fixed, or the analysis is conditional on the observed predictor values. The residual standard deviation is \(s\), and \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). There are \(n-2\) degrees of freedom because two regression coefficients are estimated.

For a specified \(x_0\), the fitted response is \(\hat{y}_0=b_0+b_1x_0\), where \(b_0\) and \(b_1\) are the estimated intercept and slope. Let \(t^*=t_{n-2,\,1-\alpha/2}\) denote the positive critical value with cumulative probability \(1-\alpha/2\) from a \(t\) distribution with \(n-2\) degrees of freedom. Then:

\begin{aligned} SE_{\mathrm{mean}}(x_0)&=s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}},\\ \text{confidence interval for the mean response} &:\quad \hat{y}_0\ \pm\ t^*SE_{\mathrm{mean}}(x_0),\\[4pt] SE_{\mathrm{pred}}(x_0)&=s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}},\\ \text{prediction interval for an individual response} &:\quad \hat{y}_0\ \pm\ t^*SE_{\mathrm{pred}}(x_0). \end{aligned}

Both intervals are centered on the same fitted value. The prediction standard error includes an additional \(1\) under the square root to account for the variation of a new individual around the mean response. The intervals are most precise near \(\bar{x}\), since the distance term \((x_0-\bar{x})^2/S_{xx}\) increases as \(x_0\) moves away from the mean predictor value. Intervals at extreme predictor values can be wide, and predictions outside the observed predictor range require extrapolation and may be unreliable.

Why the Prediction Standard Error Has an Extra Term

A brief variance argument explains the difference between the intervals. The model mean at \(x_0\) is \(\beta_0+\beta_1x_0\). Its estimated value is \(b_0+b_1x_0\), so uncertainty about the mean comes from estimating the regression coefficients. Under the model assumptions, the variance of that fitted mean is \(\sigma^2[1/n+(x_0-\bar{x})^2/S_{xx}]\).

For a new individual, write its response as \(Y_0=\beta_0+\beta_1x_0+\varepsilon_0\). The prediction error is \(\ Y_0-\hat{y}_0=\varepsilon_0-(b_0-\beta_0)-(b_1-\beta_1)x_0\). The new error \(\varepsilon_0\) is independent of the errors used to fit the line and has variance \(\sigma^2\). Thus the prediction-error variance is the fitted-mean variance plus \(\sigma^2\):

\begin{aligned} \operatorname{Var}(Y_0-\hat{y}_0) &=\sigma^2+\sigma^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right)\\ &=\sigma^2\left(1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right). \end{aligned}

Replacing \(\sigma\) by \(s\) gives the stated standard errors. Since the prediction standard error is strictly larger than the mean-response standard error for the same \(x_0\), the prediction interval is wider when both use the same confidence level and model fit.

Worked Example: Both Intervals at an Off-Center Value

Suppose hypothetical data on a measured response \(Y\), in response units, and predictor \(x\), in predictor units, produce the fitted line \(\hat{y}=50+3x\). The data summaries are \(n=12\), \(\bar{x}=5\), \(S_{xx}=110\), and \(s=4\) response units. At \(x_0=8\), calculate 95% intervals for the mean response and for one individual response. Assume residual plots support linearity and constant variance, the observations are independent, and the errors are reasonably normal. Also suppose \(8\) is within the observed predictor range.

The fitted response is \(\hat{y}_0=50+3(8)=74\) response units. With \(df=12-2=10\), the 95% critical value is \(t^*=2.228\).

For the mean response, the standard error is

SE_{\mathrm{mean}}(8) =4\sqrt{\frac{1}{12}+\frac{(8-5)^2}{110}} =4\sqrt{0.16515} \approx 1.626.

The margin of error is \(2.228(1.626)\approx3.622\) response units. Therefore, the 95% confidence interval is \(74\pm3.622\), or approximately \([70.38,77.62]\) response units. In repeated samples under the model, this method would capture the true mean response at \(x=8\) in about 95% of intervals. This particular interval estimates that mean, not the response of any one individual.

For an individual response, include the additional \(1\) in the prediction standard error:

SE_{\mathrm{pred}}(8) =4\sqrt{1+\frac{1}{12}+\frac{9}{110}} =4\sqrt{1.16515} \approx 4.318.

The prediction margin is \(2.228(4.318)\approx9.620\) response units. The 95% prediction interval is \(74\pm9.620\), or approximately \([64.38,83.62]\) response units. It gives a range of plausible values for one new individual response at \(x=8\), under the stated model. It is wider than the mean-response interval because individual outcomes vary around their mean.

Comparing Predictor Locations

The distance term in each standard error makes precision depend on where the response is requested. At \(x_0=\bar{x}\), that term is zero and both intervals have their smallest standard errors. Moving away from \(\bar{x}\) increases both standard errors by the same location-related amount, but the prediction standard error remains larger because it also includes individual variation.

Worked Example: Intervals at the Mean and Away From It

A hypothetical regression of a response in points on a predictor measured in minutes has fitted line \(\hat{y}=82+0.40x\), with \(n=20\), \(\bar{x}=30\) minutes, \(S_{xx}=4000\) minutes squared, and \(s=6\) points. Assume the regression conditions are reasonably met, the predictor values 30 and 50 are within the observed range, and observations are independent. Find 95% intervals at both values. Here \(df=18\) and \(t^*=2.101\).

At \(x_0=30\), the fitted response is \(82+0.40(30)=94\) points. Since this is \(\bar{x}\), the location term is zero. For the mean response, \(SE_{\mathrm{mean}}(30)=6\sqrt{1/20}=1.342\) points, so the margin is \(2.101(1.342)\approx2.819\). The confidence interval is approximately \([91.18,96.82]\) points.

For one individual at \(x_0=30\), \(SE_{\mathrm{pred}}(30)=6\sqrt{1+1/20}=6\sqrt{1.05}\approx6.148\) points. The margin is \(2.101(6.148)\approx12.917\), giving a prediction interval of approximately \([81.08,106.92]\) points.

At \(x_0=50\), the fitted response is \(82+0.40(50)=102\) points, and the location contribution is \((50-30)^2/4000=0.10\). Thus \(SE_{\mathrm{mean}}(50)=6\sqrt{0.05+0.10}=6\sqrt{0.15}\approx2.324\) points. The margin is \(2.101(2.324)\approx4.882\), so the mean-response confidence interval is approximately \([97.12,106.88]\) points.

For an individual at 50 minutes, \(SE_{\mathrm{pred}}(50)=6\sqrt{1+0.05+0.10}=6\sqrt{1.15}\approx6.434\) points. The margin is \(2.101(6.434)\approx13.518\), giving a prediction interval of approximately \([88.48,115.52]\) points.

The intervals centered at 102 are wider than their counterparts centered at 94 because 50 minutes is farther from \(\bar{x}\) than 30 minutes. At either value, the individual prediction interval is much wider than the mean-response interval. These widths address uncertainty and variability under the fitted model; they do not imply that responses outside an interval are impossible.

Worked Example: A Response Near the Edge of the Observed Range

In a separate hypothetical regression, suppose \(n=10\), \(\bar{x}=4\), \(S_{xx}=45\), \(s=2.5\) millimeters, and \(\hat{y}=18+1.2x\). Consider \(x_0=1\), which is assumed to lie within the observed predictor range. Suppose the observations are independent and residual checks support a linear mean, constant variance, and approximate normality. For a 95% interval, \(df=8\) and \(t^*=2.306\).

The fitted response is \(18+1.2(1)=19.2\) millimeters. The location term is \((1-4)^2/45=9/45=0.20\). Therefore, \(SE_{\mathrm{mean}}(1)=2.5\sqrt{0.10+0.20}=2.5\sqrt{0.30}\approx1.369\) millimeters. The margin is \(2.306(1.369)\approx3.158\), so the 95% confidence interval for the mean response is approximately \([16.04,22.36]\) millimeters.

For an individual response, \(SE_{\mathrm{pred}}(1)=2.5\sqrt{1+0.10+0.20}=2.5\sqrt{1.30}\approx2.850\) millimeters. The margin is \(2.306(2.850)\approx6.573\), so the 95% prediction interval is approximately \([12.63,25.77]\) millimeters. Both intervals are wider than they would be at \(x_0=4\), where the location term is zero. In context, the first interval estimates the average response at \(x=1\); the second predicts a response for one individual at that predictor value.

Assumptions and Common Pitfalls

A numerical interval is only as informative as the model supporting it. Check whether the relationship between the mean response and predictor is reasonably linear, whether residual spread is roughly constant, and whether observations can reasonably be treated as independent. Normality of errors is especially relevant to the exact small-sample \(t\) procedures. Also check whether the requested predictor value is within the range of the observed data. A calculation is possible outside that range, but it does not make extrapolation trustworthy.

Confusing the two targets. A confidence interval for a mean response is about an average at \(x_0\); it is not a range expected to contain most individual responses.
Using the same standard error for both intervals. The individual prediction standard error has the extra \(1\) inside the square root. Omitting it understates uncertainty for an individual.
Assuming equal widths at all predictor values. The standard errors increase with the squared distance from \(\bar{x}\), so intervals generally widen away from the center of the observed predictor values.
Interpreting a 95% interval as a 95% chance for this fixed interval. In the frequentist interpretation, the method has 95% long-run coverage when its assumptions hold; the parameter is fixed, not random.

For any requested predictor value, first calculate the fitted response, then use the standard error matching the target: the mean response or one individual. Use the same critical value when the confidence level and degrees of freedom are the same, and report the interpretation with the response units and context.

Check Your Understanding

Answer each question by identifying the target and, where relevant, showing the interval calculation.

At a particular \(x_0\), why is a prediction interval for one individual wider than a confidence interval for the mean response at the same \(x_0\)?
If \(x_0=\bar{x}\), what part of each standard error becomes zero, and what part remains for individual prediction?
A 95% mean-response confidence interval is reported as 20 to 24 units at \(x_0=6\). What does this interval estimate, and what does it not estimate?
What happens to the location contribution to the standard errors as \(x_0\) moves farther from \(\bar{x}\)?
Name two model conditions that should be checked before relying on the usual regression response intervals.